Open Source distributed Big Data logging service and publish/subscribe system used to reliably log streams of records

Forest Hill, MD –27 January 2015– The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today that Apache™ BookKeeper™ has graduated to become a Top-Level Project (TLP), signifying that the project's community and products have been well-governed under the ASF's meritocratic process and principles.
Apache BookKeeper was established in 2011 as a sub-project of Apache ZooKeeper™ (Open Source API for highly reliable distributed coordination) to reliably log streams of records. It serves as a building block for reliable system consistency and recovery, and can be used to turn any standalone service into a highly available replicated service.
With disk/server failure rates up to 10% annually, replication is a must in today's always-on Cloud and Big Data services. One way to build a replicated service is to ensure that all write operations to the service are copied to all replicas; Apache BookKeeper's replicated logging service is well suited for this purpose. A database may have two replicas to ensure availability: if one crashes, the other can continue to serve traffic. However, ensuring that the data in these two replicas is consistent is not an easy problem to solve. Unlike naive solutions that run into problems like deadlock and inconsistency when one or both of the replicas fail, BookKeeper uses a combination of quorum writes, fencing, and, when necessary, outsourcing of consensus to ZooKeeper to ensure no state will be lost in the case of a replica failure. BookKeeper can similarly be applied to different classes of systems, such as messaging systems, filesystems and transaction processing systems.

Apache BookKeeper is highly available (no single point of failure), and scales horizontally as more storage nodes are added. BookKeeper is used in production in many web scale companies. At Yahoo, it is used as the persistence layer for its Cloud messaging infrastructure, which delivers tens of billions of messages in a day. BookKeeper is used at Twitter as the replicated persistence backend for different messaging use cases, and is also used by Huawei as a shared storage in their solution for HDFS Namenode High Availability. 

"We're very proud to have BookKeeper become a Top-Level Project. It is a testament to the hard work that my fellow committers have put in over the years that the ASF would give us their stamp of approval," said Ivan Kelly, Vice President of Apache BookKeeper. "We hope that the increased exposure will bring even more contributions and use cases to the community."
Availability and Oversight
As with all Apache products, Apache BookKeeper software is released under the Apache License v2.0, and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For documentation and ways to become involved with Apache BookKeeper, visit and
About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 500 individual Members and 4,500 Committers successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Budget Direct, Citrix, Cloudera, Comcast, Facebook, Google, Hortonworks, HP, Huawei, IBM, InMotion Hosting, Matt Mullenweg, Microsoft, Pivotal, Produban, WANdisco, and Yahoo. For more information, visit or follow @TheASF on Twitter.
© The Apache Software Foundation. "Apache", "Apache BookKeeper", "BookKeeper", ApacheCon", and the Apache BookKeeper logo are trademarks of The Apache Software Foundation. All other brands and trademarks are the property of their respective owners.
# # #