Open source one-stop integration framework for big data in use at organizations in advertising, payment, social, gaming, AI, and more 


Wilmington, DE – June 22, 2022 – The Apache Software Foundation (ASF), the all-volunteer developers, stewards, and incubators of more than 350 open source projects and initiatives, announced today Apache® InLong™ as a Top-Level Project (TLP).


Apache InLong is a one-stop integration framework for massive data that provides automatic, secure, and reliable data transmission capabilities. The project was originally developed at Tencent in June 2013 and entered the Apache Incubator in November 2019.


The project name carries significant meaning. InLong (应龙) is a divine beast in Chinese mythology who guides the river into the sea, and it is regarded as a metaphor for the InLong system for reporting data streams.


"The InLong community aims to help enterprises simplify the process of data ingestion, ETL, and distribution," said Charles Zhang, vice president of Apache InLong. "The graduation of InLong marks the successful establishment of an open, diverse, and mature open source community. The project will continue to practice the Apache way and help the digital transformation of enterprises through a joint community effort."


InLong has been widely adopted across various industries, including advertising, payment, social, gaming, AI, and others. The Apache InLong project was originally called TubeMQ, focusing on high-performance, low-cost message queuing services. 


To further release the surrounding ecological capabilities of TubeMQ, the community upgraded the project to InLong. It integrates the entire processes of collecting, aggregating, storing, and sorting data processing. It is simple, flexible, stable, and reliable. Features include:


  • Ease of Use: Apache InLong is a SaaS-based service platform. Users can easily and quickly report, transfer, and distribute data by publishing and subscribing to data based on topics.

  • Stability & Reliability: Apache InLong is derived from the online production environment. It delivers high-performance processing capabilities for 100 trillion-level data streams and highly reliable services for 100 billion-level data streams.

  • Comprehensive Features: Apache InLong supports various data access methods and can be integrated with different types of Message Queue (MQ). It also provides real-time data extract, transform, and load (ETL) and sorting capabilities based on rules. Apache InLong also allows users to plug features to extend system capabilities.

  • Service Integration: Apache InLong provides unified system monitoring and alert services. It provides fine-grained metrics to facilitate data visualization. Users can view the running status of queues and topic-based data statistics in a unified data metric platform. Users can also configure the alert service based on their business requirements so that users can be alerted when errors occur.

  • Scalability: Apache InLong adopts a pluggable architecture that allows you to plug modules into the system based on specific protocols. Users can replace components and add features based on their business requirements.


In addition, the project recently released v1.2.0-incubating, the twelfth release of Apache InLong. Improvements include:

  • Refactoring the InLong Sort module to support Transform, such as String Split, Data Filter, Regular Join, etc.

  • Adding 8+ data nodes including MongoDB, SqlServer, Greenplum, Oracle DB, etc.

  • Optimizing the initialization process of data nodes such as Iceberg, ClickHouse, and Hive.

  • Supporting MySQL to collect Binlog from the specified offset.

  • Supporting the plug-in expansion of different types of message queues.

  • Supporting the management of multiple clusters and data nodes.


"We are very pleased to see InLong graduate from the Apache Incubator as ASF TLP," said Jiang Jie, vice president of Tencent. " InLong has been used widely across Tencent and in other critical industries, like finance and government. We are committed to contributing to the community and helping to expand the big data ecosystem."


“I was very impressed by the InLong community during the Apache incubation,” said JB Onofre, Apache InLong mentor. “The community grew in a healthy manner in both developers and users. Apache InLong is a great example of how to build a healthy and active community and successful project at The Apache Software Foundation.”


"As a one-stop data integration service, InLong's complete surrounding ecology greatly reduces the threshold for use," said Zhongbo Wu, PMC of Apache InLong. "High-performance, low-cost message queue services have also become the choice for enterprises to reduce costs and increase efficiency. And I wish the InLong community an even better and faster development in the future."


"Apache InLong integrates the capabilities of Apache Pulsar and increases data processing throughput with the help of Pulsar's queue buffering feature," said Zhai Jia, member of Apache Pulsar PMC. "I am thrilled to see the graduation of InLong. Different from previous open source big data projects, InLong integrates the capabilities of multiple big data projects and has rich usage scenarios. Congratulations to the InLong community."


Catch Apache InLong in action at ApacheCon Asia 2022 (29-31 July; https://apachecon.com/acasia2022) and Pulsar Summit Asia (15-16 January; https://pulsar-summit.org/event/asia-2021/schedule).


Availability and Oversight

Apache InLong software is released under the Apache License v2.0 and is overseen by a self-selected team of active contributors to the project. A Project Management Committee (PMC) guides the Project's day-to-day operations, including community development and product releases. For downloads, documentation, and ways to become involved with Apache InLong, visit https://inlong.apache.org and https://twitter.com/ApacheInlong.


About the Apache Incubator

The Apache Incubator is the primary entry path for projects and codebases wishing to become part of the efforts at The Apache Software Foundation. All code donations from external organizations and existing external projects enter the ASF through the Incubator to: 1) ensure all donations are in accordance with the ASF legal standards; and 2) develop new communities that adhere to our guiding principles. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF. For more information, visit https://incubator.apache.org/. 


About The Apache Software Foundation (ASF)

Established in 1999, The Apache Software Foundation is the world's largest open source foundation, stewarding 227M+ lines of code and providing more than $22B+ worth of software to the public at 100% no cost. The ASF’s all-volunteer community grew from 21 original founders overseeing the Apache HTTP Server to 820+ individual Members and 200 Project Management Committees who successfully lead 350+ Apache projects and initiatives in collaboration with 8,400+ Committers through the ASF's meritocratic process known as "The Apache Way." Apache software is integral to nearly every end user computing device, from laptops to tablets to mobile devices across enterprises and mission-critical applications. Apache projects power most of the Internet, manage exabytes of data, execute teraflops of operations, and store billions of objects in virtually every industry. The commercially-friendly and permissive Apache License v2 is an open source industry standard, helping launch billion dollar corporations and benefiting countless users worldwide. The ASF is a US 501(c)(3) not-for-profit charitable organization funded by individual donations and corporate sponsors that include Aetna, Alibaba Cloud Computing, Amazon Web Services, Anonymous, Baidu, Bloomberg, Capital One, Cloudera, Comcast, Confluent, Didi Chuxing, Facebook, Google, Huawei, IBM, Indeed, LINE Corporation, Microsoft, Namebase, Pineapple Fund, Red Hat, Replicated, Salesforce, Talend, Target, Tencent, Union Investment, VMware, Workday, and Yahoo. For more information, visit https://apache.org and https://twitter.com/TheASF. 


© The Apache Software Foundation. "Apache", "InLong", "Apache InLong", and "ApacheCon" are registered trademarks or trademarks of the Apache Software Foundation in the United States and/or other countries. All other brands and trademarks are the property of their respective owners.