Archive

Posts Tagged ‘crowbar’

THIS JUST IN: Dell Open Sources Hadoop Barclamps for Crowbar

November 29, 2011 Leave a comment

Have you been plugged into what Dell is doing in the open source big data space, specifically around Hadoop?

A few months ago, Dell (the company I work for) announced the Dell | Cloudera Hadoop Solution to provide our customers with superfast data mining, processing and analysis, and featured the Dell-developed Crowbar software framework to deploy / manage it.  

To make it even cooler, we went ahead and open sourced Crowbar (Apache 2).

And earlier this month, at Hadoop World, we announced our intent to open source a few Hadoop barclamps (Crowbar modules that perform a specific function like configure BIOS).

Well, that day has arrived – the Hadoop barclamps are now open sourced.

.

The details

Specifically, the newly open sourced barclamps help with the following Hadoop projects:

  • Cloudera CDH / Enterprise enables Hadoop administrators to first deploy CDH then easily move to Cloudera Enterprise if/when the business needs warrant.
  • Zookeeper allows Apache Hadoop administrators to track and coordinate distributed applications.
  • Apache Pig provides a compiler that produces sequences of Map-Reduce programs.
  • Flume provides an agent for collecting data and putting into the Hadoop environment.
  • Sqoop allows rapid connection to external data sources including relational databases. E.g., data can be moved from Oracle to Hadoop and back again for different types of analysis.
      

Videos

Our team has put together a number of videos to help provide you more detail on what these new barclamps can do – check them out.

Get those barclamps!

Ready to get the barclamps?  Head over to https://github.com/dellcloudedge/barclamp-hadoop to download them.

  
If you’re interested in learning more about the Dell innovation happening in the Hadoop space, visit us at www.Dell.com/Hadoop or email us at Hadoop@Dell.com

Until next time,

JBGeorge
@jbgeorge

Sneak Peek: Dell at the Fall 2011 OpenStack Design Summit and Conference

September 29, 2011 Leave a comment

.

It’s that time of year again – leaves changing color, kids back in school, Halloween preparations…

…and time for the the Fall OpenStack Design Summit and Conference!

For Dell, the company I work for, this is our fourth time at bat with the OpenStack summit – and if you do the math, that means we’ve been at EVERY OpenStack Design Summit since the initiative was announced.Dell Crowbar!   If you’ve never been, this is a great time to come out and check out what’s happening in this innovative open source community. 
  

Since Dell is among a few vendors that have been a part of OpenStack since Day 1, I can truly say I’m amazed at how far we’ve come.  From a fledgling group of partners with a vision for an open source project to a thriving community of over 100 partners and multiple projects in just over a year is an incredible feat.

  
And there’s no chance that this community is stopping any time soon.

  
With the release of Diablo, new work ALREADY starting on Essex, and a LOT of customer interest in the Dell OpenStack-Powered Cloud Solution, this is an amazing time to be a participant in the community!

  
So what’s Dell got in store for the Design Summit and Conference next week?

    
Glad you asked.
  

  • Conference Sponsor – Dell’s been an active part of every Design Summit to date, including sponsorship, gear donation, speaking sessions, etc.  And this time around, we’re continuing that tradition as a a top level sponsor of the event.Dell
      
  • Keynote – We’ll have one of our fearless leaders get up on stage and talk about what we’re seeing in the community and the market place, some insight into how OpenStack has positively impacted how we all do cloud, and where we see OpenStack going next.
      
  • Speaking Sessions – A few of the OpenStack veteran celebrities at Dell – Rob Hirschfeld, Greg Althaus, and others – will be speaking at various times throughout the conference, leading sessions, participating in panels, etc.  (Please no autographs or flash photography.)
      
  • Live Demos – The Dell OpenStack-Powered Cloud Solution is the only HW + SW + Services OpenStack solution on the market, and we’ll be showcasing it LIVE at the conference.  And we’ve got a few surprises up our sleeves – Crowbar barclamps anyone???  🙂

Get a backstage pass!  We’ve been working on some new tech to better enable our customers with OpenStack, but some of it’s not quite ready for prime time demos yet.
  
HOWEVER….
  
If you’re interested in getting a sneak peek at some of the innovation while you’re at the Conference, email us ASAP at OpenStack@Dell.com – we’ll schedule some time for you to meet with our engineers while they’re out in Boston next week, and you’ll get a front row seat to some of the coolness we’re working on.  

  
I plan on blogging a bit next week while I’m at the summit, so be sure to stay tuned for the latest and greatest from Boston.

See you in the Cradle of Liberty!

More info:

Until next time,

JOSEPH
@jbgeorge

The Dell OpenStack-Powered Cloud Solution: An Update!

September 18, 2011 Leave a comment

.

Hello fellow cloud fans!

The Cloud is Here

The Cloud is Here (Note: I took this pic outside one day - no doctoring at all! Two jet trails created an arrow pointing at a cloud!)

There’s been a lot going on with the OpenStack team at Dell – and its been wreaking havoc with my blogging time!  

It’s been about six weeks since we announced our Dell OpenStack-Powered Cloud Solution, the market’s first OpenStack cloud solution featuring hardware, software and services. 

And the movement has been fast and furious!

Here’s a quick update…
  

The Real Deal

This is the most exciting development from our team.

Real customers solving real problems with a real OpenStack solution from Dell… TODAY. 

  
Many of you have reached out, learned about the power and agility of the Dell OpenStack-Powered Cloud Solution, and have stepped up to working with Dell to get OpenStack going in their environments.  It’s awesome that we’re able to partner with a number of you out there on some of the coolest, new technology out there.

With our long history with OpenStack (since Day 1), our cloud optimized PowerEdge C server line, and the power of the Dell Crowbar software enabling customers to get a multi-node OpenStack cloud on bare metal in less than two hours, this solution is just making a lot of sense for many of our customers.

In fact, looking for some “rubber meets the road” info? 

Look no further than Dell OpenStack customer, DreamHost – they’re a great example of how partnering with Dell can get you REAL progress on OpenStack. 

DreamHost and the Dell OpenStack-Powered Cloud Solution

DreamHost working with Dell on OpenStack!

Check out the FULL CASE STUDY here – http://i.dell.com/sites/content/shared-content/solutions/en/Documents/dreamhost-openstack-cloud-case-study.pdf
  

  
  
Crowbar

Crowbar has really taken off since  we announced it with the release of the Dell OpenStack-Powered Cloud Solution. 

In case you don’t know what Crowbar is, it’s a Dell- developed software framework designed to enable customers to deploy OpenStack quickly, as well as be a continuous integration engine for your sustaining OpenStack cloud.  It’s based on Opscode’s Chef tool, and is available as open source code for anyone to download (link to site at end of blog).

  • I mentioned DreamHost as a Dell OpenStack customer – they are actually using Crowbar as a means to deploy their Ceph storage technology.  They have taken Crowbar and developed their own “barclamp” to deploy their technology.  (A barclamp is a Crowbar module that performs a function like BIOS update, network discovery, etc.)
      
  • The Cloud Foundry team at VMware has done the same – taken the Crowbar software framework and written barclamps that will deploy the open source PaaS.
      
  • Check out Rob Hirschfeld’s Blog to learn more about some of the tech details on what’s happening with Crowbar.  Link at the end of this blog.
      

And a whole lot more – Diablo, the Essex Summit, OpenStack training at Dell

Our team’s are also feverishly working on Diablo support for our OpenStack offering.  (FYI – the Diablo release of OpenStack is slated to go public next week on Sep 22.)  We’ll actually be cooperating with a number of partners over the next week on enabling Diablo – keep an eye out – you may see some tech blogging going on about this soon.

And then there’s the upcoming OpenStack Conference and Design Summit, to be held in Boston the week of Oct 3.  Dell has been a major part of EVERY OpenStack design summit, and this one is no different, as Dell is a conference sponsor of the event.  All the usual Dell OpenStack celebrities will be there – along with a few new ones – participating in a variety of capacities at the event, so buckle up for some real insight based on real world OpenStack feedback.

Finally, if you haven’t heard, the OpenStack community announced a number of trainings to get users up to speed on the cloud technology.  There are a number of trainings planned, including sessions in Boston, London, and Dell headquarters in Austin.  If you’re interested in learning more about the training happening at Dell in Austin, check it out here http://www.cvent.com/events/rackspace-training-for-openstack-austin/event-summary-77838e10ba284088b4628026f2cd9158.aspx

More info

Being the marketing director, I want to be sure you have access to a number of the wicked cool stuff out there on our OpenStack-Powered Cloud Solutiona, Crowbar, and more.

Whew!  Told ya it would be a lot…

OK, that should get you up to speed. 

As always, feel free to contact us at OpenStack@Dell.com or leave a comment.

On my next blog, I’ll get you an update on where we are with our new Apache Hadoop solution!

Until next time,

JBG
@jbgeorge

Big News: #Dell and #Cloudera Partner on #Hadoop Solution

August 4, 2011 1 comment

.

Big news!

DellToday, Dell (the company I work for) has announced a solution offering in conjunction with Cloudera, called the Dell | Cloudera Solution for Apache Hadoop!  It’s a fully deployable Hadoop solution made up of Dell PowerEdge C hardware, Dell and Cloudera software, and Dell and Cloudera services – the complete package.

(Note that this is the second solution that my team has put out in the last two weeks – if you missed last week’s OpenStack announcement, you can get more info here.)

But on to the star of today’s show!

We’d often hear from our customers that they were experimenting with Hadoop on Dell hardware to analyze sales data, marketing information, etc in order to better serve their own customers.  And pretty soon, we were being asked, “What’s Dell’s point of view?  What recommendations do you have for distributions, architecture, and implementation?”

ClouderaWell, here’s the answer.  The Dell | Cloudera Solution for Apache Hadoop.
  

What’s Under the Hood

    • Hardware – similar to OpenStack, the reference architecture that this solution is built on is the innovative Dell PowerEdge C server line.  Based out of Dell’s well known Data Center Solutions team (aka DCS), PowerEdge C server technology was born out of learnings from custom engagements with some of the biggest cloud and hyperscale environments in the world.  What we discovered is that, despite custom requirements and specs, there was some commonality among the requests.  Concepts like density, power efficiency, etc were at the top of the list when it came to these environments.
        
      So we took what was common and built a line of servers that was ideal for cloud and hyperscale environments, which became the Dell PowerEdge C server family.  And it is an ideal platform for our Hadoop solution, as all the tenets of scale apply when building out an optimal Hadoop instance.
        
    • Software – Our partners, Cloudera, who are among the most established Hadoop vendors in the market, will be providing their distributions of the Hadoop software, as well as their CDH management tools, which are top notch.  In addition, Dell will be providing our Crowbar software framework and a specific Cloudera barclamp as part of the solution.  (As a refresher, we announced availability of Crowbar last week as part of our OpenStack announcement, which helps speed deployment CONSIDERABLY.  In the case of the Dell Cloudera Solution for Apache Hadoop, it can speed bare metal deployment of a Hadoop cluster from days or weeks to less than one day!)
        
    • Services – Dell and Cloudera are partnering to enable our customers to get going on a validated instance of Hadoop, and offer a variety of services from support of the entire stack of hardware and software, as well as training on the technologies that will enable our customers on Hadoop. 
        

Now’s the time to get started on Hadoop, and the Dell | Cloudera Solution for Apache Hadoop can get you there.

Find out more about this new solution at www.Dell.com/Hadoop or get in touch with us at Hadoop@Dell.com.
     

More info:

Until next time,

JBG
@jbgeorge

The #Dell #Crowbar Software Framework: Who, What, When, Where, and How

August 1, 2011 1 comment

.

It’s been about a week since Dell (the company I work for) officially announced the Dell OpenStack Cloud Solution, the market’s first HW / SW / Services solution for OpenStack – and the response has been great! 

(Check out my last blog post about it here for more details.)

Crowbar from Dell

Crowbar from Dell

Since this wasn’t your average announcement with a big company like Dell actually contributing to the community, I received a number of questions, comments, and accolades around our contributed software framework – Crowbar.

So I thought I’d pass on some of the people, history, functionality, and background behind Dell’s Crowbar software.
  

History

Since Dell had been a committed partner of OpenStack’s since it’s announcement in July of 2010 (over a year ago), we were among the first to begin working with the OpenStack code as we began developing the Dell solution for it.   Since it was very early, very raw code, there was a lot of installing / testing / coding / re-testing / blowing it away / reinstalling / testing / coding  / re-testing / blowing it away… you get the idea. 

And as any cloud technology goes, deploying OpenStack on bare metal took time, effort, and expertise

It wasn’t long before our solutions development team decided we’d automate this process to make our efforts quicker.  But lo and behold, we discovered that others in the community were dealing with the same thing.   A few early users indicated that it was taking them a full day to deploy OpenStack from start to finish, and in most cases, it was taking multiple days.

So we decided build out a quick, automated method of deploying OpenStack on bare metal technology that the whole community could use.

And Crowbar was born.
     

Functionality

Extending the capabilities of the Chef configuration management framework from Opscode, our team began developing Crowbar with the goal of installing a multi-node OpenStack cloud on bare metal PowerEdge C servers in less than four hours.  “Multi-node” was a key requirement as almost every use case we encountered required that. 

We announced Crowbar’s existence and development in early 2011, with our intent to field test, and then release to the open source community.  Over the course of time, we began testing it out in various instances and scenarios, as well as demo-ing it at various cloud events.  Many of you may remember us showcasing our progress at SCaLE, CloudConnect, and the Apr OpenStack Design Summit (which Dell sponsored).  

With time and experience, we came across a number of requirements for this type of a tool – automated BIOS configuration, RAID configuration, network discovery and set up, installation of the open source system monitoring tool Nagios, installation of the open source performance monitoring tool Ganglia, and more.  So we built it all in.

And based on the extensive cloud cloud experience on our team, we knew that cloud development and management is an ever evolving exercise – as the Dell OpenStack lead architect and OpenStack community celeb Rob Hirschfeld would say, “Cloud is always ready, and never finished.”  So Crowbar is designed in a way to manage change.  Add in a new supported server to the mix, and Crowbar detects it and proceeds to add it to the OpenStack pool.  When software needs to be updated, Crowbar can be leveraged to update once, and deploy to all.
     

Barclamps

Dell Crowbar featuring Barclamps!

Dell Crowbar featuring Barclamps!

And what about new components that come out?  There are a number of new OpenStack capabilities coming out of various projects – how do those get rolled in?

Barclamps.  

(Yes, we designed Crowbar with modularity in mind.)

Think of it this way:  Crowbar is made up of two general functionality categories – the Crowbar core and Crowbar barclamps.  The Crowbar core contains all the main functionality that allows Crowbar to perform tasks.  Barclamps are modules that actually perform a function.  So need to deploy OpenStack Nova (aka cloud compute)?  Leverage the Nova barclamp.  Need to deploy OpenStack Swift (aka cloud storage)?  Leverage the Swift barclamp.

Need to deploy some software that doesn’t have a barclamp?

You can create your own barclamp!

And that’s the beauty of Crowbar – we created (and will continue to create) barclamps that work for our customers, but we’re encouraging / training our customers to develop their own barclamps, and we want the community at large to do the same.
  

What real customers are doing with the Crowbar and the Dell OpenStack Cloud Solution

In our press release, one of our OpenStack customers, DreamHost, spoke about their success with the Dell OpenStack Cloud Solution.  Along with Dell server technology and services, the innovative DreamHost team really picked up on the Crowbar tool.  In fact, as they are working on hosted options for their Ceph technology, the DreamHost team has begun developing a barclamp for Ceph!

In fact, here’s what the DreamHost GM of Emerging Technologies, Ben Cherian, had to say about Crowbar:

“If Dell had not developed Crowbar, we would have been forced to write a similar tool ourselves. It’s a fundamental goal that we have in the company: automate, automate, automate. That’s how we drive down prices.”

Ben also went on to say that since Dell had already developed a technology like Crowbar, we ended up saving them four to six months of development time.
  

Common Questions

  1. Is Crowbar for Dell products only?
      
    The Crowbar software that we’ve developed at Dell will obviously work with Dell supported platforms.  But there is no reason that this framework will not work for any other vendor technology – all that needs to happen is for that vendor / user / community member to create a barclamp for that technology.
       
  2. Is Crowbar for OpenStack only?
      
    Obviously, our original use case is for OpenStack – it has been tested and validated for that model.  However, again, it is a flexible software framework.  We can see a number of use cases for OpenStack, and welcome the community to use it where it makes sense.
      
  3. What do I get with Crowbar as a part of the Dell OpenStack Cloud Solution vs open source Crowbar?
      
    There are two main differences at this point: 
      
    First, open source Crowbar has everything that the Dell OpenStack Cloud Solution Crowbar has, except for the BIOS and RAID capabilities, where we’re trying to clear some legal hurdles before we open source.  Once those are cleared, we expect to open source those capabilities.
      
    Second, when you get Crowbar as a part of the Dell OpenStack Cloud Solution, you will get full support from Dell via Dell Services.  Any questions, support, training you need on Crowbar, Dell delivers it.
      

Where do I get Crowbar?

OK, the question of the hour.   How do you get Crowbar?

  1. You get Dell-supported Crowbar when you purchase the Dell OpenStack Cloud Solution.
      
  2. You get open source Crowbar from our github site – http://www.github.com/dellcloudedge.
      

You can head over to http://www.RobHirschfeld.com and see much more TECHNICAL content on the Dell OpenStack Cloud Solution and Crowbar.  In addition to being our lead architect on OpenStack, Rob is actually one of the developers of the actual Crowbar software.  (You can also follow him on Twitter at @zehicle.)

And as always, if you want to learn more about the Dell OpenStack Cloud Solution, you can head to http://www.Dell.com/OpenStack.  Or if you’d like to reach out to us to learn more, drop me and the gang a line at OpenStack@Dell.com.
  

Until next time,

JOSEPH
@jbgeorge
  

Dell OpenStack Cloud Solution

Dell OpenStack Cloud Solution