Open Source distributed publish-and-subscribe messaging system in use at MercadoLibre, Oath, One Click Retail, STICorp, TaxiStartup, Yahoo Japan Corporation, and Zhaopin.com, and more.
Wakefield, MA —25 September 2018— The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 Open Source projects and initiatives, announced today Apache® Pulsar™ as a Top-Level Project (TLP).
Apache Pulsar is a next-generation, Open Source distributed publish-and-subscribe messaging system designed for scalability, flexibility, and no data loss.
"We are very proud of Pulsar reaching this important milestone. This is the testament to all work done over the years by all the contributors, before and after starting our journey within The Apache Software Foundation," said Matteo Merli, Vice President of Apache Pulsar. "During the incubation process, it has been amazing to see the community grow and the project mature at such a high pace. The last year has seen the evolution of Pulsar from a its original messaging core into an integrated platform for data in motion. We are thrilled to continue drive the innovation in this exciting and fast moving space."

Pulsar is a highly scalable, low latency messaging platform running on commodity hardware. It provides simple pub-sub and queue semantics over topics, lightweight compute framework, automatic cursor management for subscribers, and cross-datacenter replication. The project was originally developed at Yahoo (now part of Oath), and was submitted to the Apache Incubator June 2017. 

The initial goal for Pulsar was to create a multi-tenant scalable messaging system that could serve as a unified platform for a wide set of very demanding use cases. Thanks to the original design we have been able to iterate and expand the scope of the project, adding lightweight compute and a connector frameworks that allow users to process data and integrate with external systems, all from within Pulsar. 

The unique architecture of Pulsar, which separates the serving and storage layers, leveraging Apache BookKeeper as the storage component, has proven to be a key strong point. The two layers architecture enables Pulsar to offer a vastly simplified approach to the cluster operations, allowing operators to easily expand clusters and replace failed nodes, or by providing a much higher write and read availability. 

Apache Pulsar is in use at MercadoLibre, Oath, One Click Retail, STICorp, TaxiStartup, Yahoo Japan Corporation, and Zhaopin.com, among others.

"Launching Pulsar at Yahoo in 2015, our goal has always been to make Pulsar widely used and well-integrated with other large-scale open source software," said Joe Francis, Director, Storage and Messaging, Oath. "We are excited for Pulsar's graduation and to see the growth of its vibrant open-source developer community within The Apache Software Foundation. At Oath we run Apache Pulsar at scale across many major products — including Yahoo Mail, Yahoo Finance, Yahoo Sports and Oath Ad Platforms — and in multiple data centers across the globe, with full mesh replication. Pulsar will continue to be an integral part of our tech stack, in streaming and also as a bridge between public and private clouds in our hybrid cloud strategy."

"At Zhaopin.com, we have used Apache Pulsar to build our enterprise event bus, because it has many enterprise features to address the shortcomings of existing messaging systems, such as message durability, low latency," said Hui Li, Director of Infrastructure Group at Zhaopin. "We also contributed a few exciting features to Pulsar, and are planning to work with the community to contribute more. It's been thrilling to watch the community grow, and I'm very proud and excited to see that the project is graduating. Pulsar has a bright future, and I'm looking forward to what's to come."

"We have used Apache Pulsar as a centralized pub-sub messaging platform for many of our services/applications. It has remarkable features; multi-tenancy and horizontal scalability that allows us to deal with a large number of services/applications on a single system, and durability, high throughput and low latency bring reliable real-time pub-sub messaging to users," said Nozomi Kurihara, Manager of Messaging Platform team at Yahoo Japan Corporation. "We're very excited with the graduation of Pulsar, and strongly believe it will play a significant role in the next generation of stream processing. We will continue to contribute by sending more pull-requests and by holding community events etc. in our aspirations for continued growth."
"After two years of struggling with the complexity of other technologies, we turned to Apache Pulsar for a new platform that would simplify our data pipeline," said Jowanza Joseph, Principal Software Engineer at One Click Retail. "Because of Pulsar's unique combination of messaging and stream processing, we've been able to replace multiple systems with one solution that works seamlessly in our Kubernetes environment. Pulsar functions has allowed us to dramatically simplify our stream processing pipeline and to reduce the cost associated with production grade stream processing systems. Seeing Pulsar become a top-level Apache project is a great milestone that validates our confidence in the current and future innovations of Pulsar and the Pulsar community."

"With the graduation, we hope to take the Apache Pulsar project and community to the next level and to reach a wider set of users and contributors, with the ultimate goal of building a strong ecosystem," added Merli. "We welcome anyone to join our efforts by helping with code, documentation or technical discussions in our forums."

Catch Apache Pulsar in action at ApacheCon North America 24-27 September 2018.

Availability and Oversight

Apache Pulsar software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache Pulsar, visit http://pulsar.apache.org/ and https://twitter.com/apache_pulsar .

About The Apache Software Foundation (ASF)
Established in 1999, the all-volunteer Foundation oversees more than 350 leading Open Source projects, including Apache HTTP Server --the world's most popular Web server software. Through the ASF's meritocratic process known as "The Apache Way," more than 730 individual Members and 6,800 Committers across six continents successfully collaborate to develop freely available enterprise-grade software, benefiting millions of users worldwide: thousands of software solutions are distributed under the Apache License; and the community actively participates in ASF mailing lists, mentoring initiatives, and ApacheCon, the Foundation's official user conference, trainings, and expo. The ASF is a US 501(c)(3) charitable organization, funded by individual donations and corporate sponsors including Aetna, Anonymous, ARM, Bloomberg, Budget Direct, Capital One, Cerner, Cloudera, Comcast, Facebook, Google, Hortonworks, Huawei, IBM, Indeed, Inspur, LeaseWeb, Microsoft, Oath, ODPi, Pineapple Fund, Pivotal, Private Internet Access, Red Hat, Target, and Union Investment. For more information, visit http://apache.org/ and https://twitter.com/TheASF

© The Apache Software Foundation. "Apache", "Pulsar", "Apache Pulsar", "BookKeeper", "Apache BookKeeper", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.

# # #