Apache Bigtop 1.0 release was announced two weeks ago and it is time to share what we were doing in these 10 months since 0.8 has came out. New stack has latest and most stable versions of Bigdata and machine learning applications like Apache Hadoop, HBase, Spark, and Hive as well as representatives of quickly growing in-memory computing segment like Apache Ignite.
Release 1.0 delivers a number of new and upgraded data processing
components. Here's the full list, which one can retrieve by running ./gradlew all-components on the release branch-1.0
As you see the stack is based on the most stable Hadoop 2.6.0, delivers stability releases of most of the standard components and adds some major new ones like Spark 1.3.1 The release brings in a number of new features aimed to improved developers and users experience with the build system, smoke tests that can be ran without any preliminary maven installations, and provisioning of the VMs, containers, and real distributed environments. Setting up of a standard development environment is now a breeze and can be done with simple ./gradlew toolchain command
One of the exciting development in this new version of the stack is the acceleration of tradition Hadoop MR and HDFS that can be easily and transparently achieved through Apache Ignite Hadoop Accelerator. With Bigtop's Puppet recipes one can stand up the cluster with Ignite Hadoop Accelerator in it and get 30x better performance out of the legacy MR code without re-compiling or touching the code. Here you can take a quick look for yourself how easy it is
As always the convenience binary artifacts are available for immediate installation: just pick a repo file for your packaging format from the release repository, set it up according to the rules of your distribution, and fire up the puppet recipe to get your cluster up and running.
As usual you can find tons of the information on our website bigtop.apache.org or by subscribing to dev@ and/or user@ lists.