Archive

Archive for the ‘big data’ Category

2012: A year of Cloud Coalescence (whatever that means)

January 5, 2012 Leave a comment

This post is a collaboration between three Dell Cloud activists: Rob Hirschfeld (@zehicle), Joseph B George (@jbgeorge) and Stephen Spector (@SpectoratDell).

We’re not making predictions for the “whole” Cloud market, this is a relatively narrow perspective based on technologies that on our daily radar. These views are strictly our own and based on publicly available data. They do not reflect plans, commitments, or internal data from our employer (Dell).

The major 2012 theme is cloud coalescence.  However, Rob worries that we’ll see slower adoption due to lack of engineers and confusing names/concepts.

Here are our twelve items for 2012:

  1. Open sourcecontinues to be a disruptive technology delivery model. It’s not “free” software – there’s an emerging IT culture that is doing business differently, including a number of large enterprises. The stable of sleeping giant vendors are waking up to this in 2012 but full engagement will take time.
      
  2. Linux. It is the cloud operating system and had a great 2012. It seems silly pointing this out since it seems obvious, but it’s the foundation for open source acceleration.
      
  3. Tight market for engineering and product development talent will get tighter. The catch-22 of this is that potential mentors are busy breaking new ground and writing code, making it hard for new experts to be developed.
      
  4. On track, OpenStack moves into its awkward adolescence. It is still gangly and rebelling against authority, but coming into its own. Expect to see a groundswell of installations and an expected wave of issues and challenges that will drive the community. By the “F” release, expect to see OpenStack cement itself as a serious, stable contender with notable public deployments and a significant international private deployment foot print.
      
  5. We’ll start seeing OpenStack Quantum (networking) in near-production pilots by year end.OpenStack Quantum is the glue that holds the big players in OpenStack Nova together. The potential for next generation cloud networking based on open standards is huge, but it will emerge without a killer app (OpenStack Nova in this case) pushing it forward. The OpenStack community will pull together to keep Quantum on track.
      
  6. Hadoop will cross into mainstream awareness as the need for big data analysis grows exponentially along with the data. Hadoop is on fire in select circles and completely obscure in others. The challenge for Hadoop is there are not enough engineers who know how to operate it. We suspect that lack of expertise will throttle demand until we get more proprietary tools to simplify analysis. We also predict a lot of very rich entrepreneurs and VCs emerging from this market segment.
      
  7. DevOps will enter mainstream IT discussions. Marketers from major IT brands will struggle and fail to find a better name for the movement. Our prediction is that by 2015, it will just be the way that “IT” is done and the name won’t matter.
      
  8. KVM continues to gain believers as the open source hypervisor. In 2011, I would not have believed this prediction but KVM making great strides and getting a lot of love from the OpenStack community, though Xen is also a key open source technology as well. I believe that Libvirt compatibility between LXE & KVM will further accelerate both virtualization approaches. 
      
  9. Big Data and NoSQL will continue to converge. While NoSQL enthusiasm as a universal replacement for structured databases appears to be deflating, real applications will win.
     
  10. Java will continue to encounter turbulenceas a software platform under Oracle’s overly heady handed management.
     
  11. PaaS continues to be a confusing term. Cloud players will struggle with a definition but I don’t think a common definition will surface in 2012. I think the big news will be convergence between DevOps and PaaS; however, that will be under the radar since most of the market is still getting educated on both of those concepts.
      
  12. Hybrid cloud will continue to make strides but will not truly emerge in 2012 – we’ll try to develop this technology, and expose gaps that will get us there ultimately (see PaaS and Quantum above)

Thoughts?  We’d love to hear your comments.

Rob, JBG, and Stephen

You can follow Rob at www.RobHirschfeld.com or @zehicle on Twitter.
You can follow Joseph at www.JBGeorge.net or @jbgeorge on Twitter.

You can follow Stephen at http://en.community.dell.com/members/dell_2d00_stephen-sp/blogs/default.aspx or @SpectoratDell on Twitter.

THIS JUST IN: Dell Open Sources Hadoop Barclamps for Crowbar

November 29, 2011 Leave a comment

Have you been plugged into what Dell is doing in the open source big data space, specifically around Hadoop?

A few months ago, Dell (the company I work for) announced the Dell | Cloudera Hadoop Solution to provide our customers with superfast data mining, processing and analysis, and featured the Dell-developed Crowbar software framework to deploy / manage it.  

To make it even cooler, we went ahead and open sourced Crowbar (Apache 2).

And earlier this month, at Hadoop World, we announced our intent to open source a few Hadoop barclamps (Crowbar modules that perform a specific function like configure BIOS).

Well, that day has arrived – the Hadoop barclamps are now open sourced.

.

The details

Specifically, the newly open sourced barclamps help with the following Hadoop projects:

  • Cloudera CDH / Enterprise enables Hadoop administrators to first deploy CDH then easily move to Cloudera Enterprise if/when the business needs warrant.
  • Zookeeper allows Apache Hadoop administrators to track and coordinate distributed applications.
  • Apache Pig provides a compiler that produces sequences of Map-Reduce programs.
  • Flume provides an agent for collecting data and putting into the Hadoop environment.
  • Sqoop allows rapid connection to external data sources including relational databases. E.g., data can be moved from Oracle to Hadoop and back again for different types of analysis.
      

Videos

Our team has put together a number of videos to help provide you more detail on what these new barclamps can do – check them out.

Get those barclamps!

Ready to get the barclamps?  Head over to https://github.com/dellcloudedge/barclamp-hadoop to download them.

  
If you’re interested in learning more about the Dell innovation happening in the Hadoop space, visit us at www.Dell.com/Hadoop or email us at Hadoop@Dell.com

Until next time,

JBGeorge
@jbgeorge