Time for another hackathon.

There are alot of companies who contribute to BigTop.  Pivotal, Cloudera, Red Hat, WanDisco, Amazon and so on... if I left yours out feel free to leave a comment below and I'll update this post.  And today, we are proud  to announce that Red Hat is hosting the next bigtop hackathon, immediately following Hadoop Summit 2014, in San Jose.  Hadoop is about alot more than just soucre code - its about packaging, deployment, configuration, and so on.  And BigTop has embraced the difficult task of tying all this together.

Apache BigTop makes Hadoop deployment transparent

All source code is complex, regardless of the language.  But whats even more complex is the deployment of code on a distributed system.  While vendors have come a long way making it easy to DEPLOY hadoop with black box administrative or cloud tools, nobody has really opened up hadoop by building a culture around the deployment of it.

Enter Apache BigTop. 

BigTop contains all the stuff thats not in the hadoop docs.  For example:

  • Puppet modules for installation and configuration of hadoop without using a tarball.
  • Vagrant recipes for deployment-from-zero.
  • Smoke tests for fine grained testing.
  • The intersection of Java and RPM .
  • bigpetstore app for demonstrating to the business community how to actually use hadoop, gradle, pig, and google's javascript visualization widgets tools to build, test, and deploy a reference "hadoop app".

BigTop is working to embrace HCFS

The community has put alot of work into testing different hadoop stacks on different file systems (https://wiki.apache.org/hadoop/HCFS/Progress), and the bigtop community has embraced this effort - to their own higher cost of having to support a generic filesystem deployment, and also, at the cost of alot of JIRA reviewing.  For example, with the recent BIGTOP-952 and BIGTOP-1200 JIRAs, we're now packaging HDFS independent artifacts into BigTop.  That paves the way for more competition, more choice, and more hadoop hacking - which ultimately translates to a better end-user experiences, around hadoop.

BigTop builds OS+Admin freindly packages for emerging ecosystem projects, fast !

If you compare apache bigtop with other hadoop vendor distributions, you'll find that it is the bleeding edge.  For example, you can watch this recent video demonstration of spinning up Storm on BigTop, from ApacheCon 2014: https://www.youtube.com/watch?v=VZzJxsMJahc, to see just how easy it is to deploy spark out of the box using BigTop's deployment recipes.  As new projects come forward in the upstream, the first place to put them is into apache BigTop.  This means that if you want to try out a new animal in hadoop's stack, you can easily do so with the bigtop stack.  And again : the infrastructure around vagrant makes it easy to build maintainable VM workflows around hadoop app and distribution development tasks, which easily be modified to include/exlude whichever bleeding edge packages.   Think of bigtop's approach to packaging and deployment as a lower-level version of apache ambari.

Sounds Interesting? come to the hackathon in Mountain View after Hadoop Summit  ! 

So this is all prelude to the BigTop hackathon that we are hosting at  Red Hat.  The focus will be on hacking - not presentations.  But that doesn't mean you have to be an expert to get involved.  Coming to this hackathon will give you a chance to pair program with the BigTop commiters, and try your hand at a working directly on a JIRA.  I think most would agree that hacking around on apache bigtop is an excellent introduction to that hadoop ecosystem.  

The NEXT HACKATHON will be from June 6th - JUNE 9th at the Red Hat Offices in Mountain View California.  For more details, ping us on the BigTop mailing list and check the meetup URL : http://www.meetup.com/Bay-Area-Bigtop-Meetup/events/184893732/ .

See you there !