by Mark Shan, Chairman of Tencent Open Source Alliance and Tencent Cloud Open Source Ecosystem General Manager


The Apache Software Foundation (ASF) manages more than 227 million lines of code, has 206 project management committees, leads more than 350 Apache projects and operates through a merit system, with more than 850 members, 8,100+ committers, and tens of thousands of contributors.


Previously the Apache Group, the ASF has grown to one of the largest open source foundations in the world today. It has built the well-known "Apache Way" through its leadership, sound community, and merit thinking, resulting in a set of schemes that promote the sustainable development of open source communities and guide the practice of open source projects.projects.


Since Tencent Open Source was created 11 years ago, a large number of Tencent engineers have formed a deep connection with the Apache community by participating and contributing to Apache projects. Furthermore, by learning from the Apache Way, Tencent is going through its open source journey.


At ApacheCon@Home 2021, I shared how Tencent thinks, explores, and practices open source according to the Apache Way. Below is a synopsis of this presentation:


The Apache Way's Importance in Community Building


The Apache Way is difficult to define. Although the Apache Way has evolved somewhat over the years, the original intention of "high transparency" has remained unchanged. In Mark Shan’s view, Tencent's learning experience from the Apache Way can be summarized into five main points:

  1. Everyone has the opportunity to participate and can become a contributor. Contributors can increase their impact and personal growth through their contributions to projects.
  2. The ASF has a structure that encourages contributions from everyone, regardless of employer. This means that, for example, committer and PMC roles are open to anyone who earns the title. Tencent encourages its engineers to participate in the Apache community actively.
  3. Understanding and practicing open communication is extremely important. Because open source is the collaboration of a global community, Tencent engineers are able to participate in the Apache open source project through asynchronous collaboration using the mailing list. Code and decision-related communication are open and transparent.
  4. Reaching consensus when making decisions is strongly encouraged. Consensus can maintain project momentum and productivity. But when a complete consensus is impossible, voting or other coordination is available to arrive at binding decisions.
  5. The most important point is that the Apache community's motto, "community over code", is often emphasized. Because a healthy community is more important than simply good code. A strong and healthy community can always correct code problems, while an unhealthy community may struggle to maintain the code repository in a sustainable manner. In addition, flexibility is also an integral part of ASF's sustainable open source success.


Tencent's Open Source Way —Inspired by the Apache Way

Apache projects and their communities are unique and diverse. In the community-led development process, Apache members formed the Apache Way based on their experience.

Many of Tencent's open source practices and results are executed following the model of the Apache Way. After many years of incorporating open source into its culture, Tencent has formed an approach, "open collaboration, open source for good," which reflects Tencent's value and vision, and is honed based on open source best practices.

Practicing the Apache Way: Contributing and Donating to open source projects

Tencent engineers have been contributing to and helping to lead many ASF projects, including:

1) Big Data

Over the past four years, several Tencent engineers helped lead releases for Apache Hadoop 2.8.4 and 2.8.5, Apache Ozone 1.0.0 (Ozone was a sub-project of Apache Hadoop, and became a Top-Level Project in 2020), and Apache Spark 2.3.2. Tencent engineers contributed more than 20 features and optimizations to several versions of Hadoop, and are core contributors in multiple Apache computing and AI frameworks that include Flink, HBase, Hive, MXNet, and Parquet.

2) Middleware

In 2019, Tencent donated TubeMQ, its self-developed trillion-level big data component, to the ASF through the Apache Incubator. In 2021 the project officially changed its name to Apache InLong, as part of its incubation process.

Tencent Applications based on Apache Projects

In addition to self-developed tools, Tencent widely uses ASF open source projects in its various business systems, with particular focus in big data, API gateways, and observability. As one of the largest daily real-time computing companies, Tencent’s overall big data platform exceeds 5 million cores, and must support single-day access message volume exceeding 55 trillion, real-time computing volume exceeding 65 trillion, and daily analysis tasks reaching 15 million.

More than two dozen Apache projects are used in applications like WeChat, QQ, and Tencent Cloud. These big data projects are used for data transmission, storage, computing, and analysis demand scenarios, supported by other technical fields of service governance that include API gateways and observability.

For big data, Apache Ozone is one of the key projects that provides support for Tencent's business scenarios that require large amounts of data and traffic. As early adopters, Tencent's big data platform deployed an Ozone cluster with more than 1,000 nodes as the back-end storage for big data applications and also uses Ozone as the main storage solution for some private data warehouse projects. 

Today, Tencent is connecting more and more businesses to Ozone, including data warehouses, machine learning platforms, Kubernetes cluster mounting volume, and more. Ozone helps Tencent operate stably, steadily, and at scale: of more than a thousand units without manual operation and maintenance intervention. In the process of verification and improvement, Tencent has done a lot of optimization work to improve performance and stability.

In addition, Apache Pulsar integrates messaging, storage and functional computing, and it adopts an architecture that separates storage and computing. Pulsar has successfully supported a large number of data and traffic business scenarios within Tencent Cloud and has some practical experience in cloud native environments, such as solving rapid and dynamic expansion and contraction, improving the utilization of cluster resources, and cluster forms.

For API Gateways, Apache APISIX offers high performance, a friendly developer experience, and rich plug-in capabilities, which is why Tencent's internal business chose it. Using APISIX, Tencent internally shares its self-developed cloud system components to solve business pain points and provide efficient API gateway services; externally, Tencent engineers contribute to the Apache APISIX open source community, expanding its influence, and leads the development of the open source community.

For Observability, Apache SkyWalking's application in Tencent's internal observable platform provides great convenience for selecting client tracing reporting of the microservices system, as well as the mechanism of computing storage separation and multi-layer query to provide very excellent performance output.

Building Tencent Open Source with 3 Major Projects

In addition to sponsoring the ASF at the Platinum level, Tencent is currently active in more than 10 foundations worldwide as a top member, including the Linux Foundation and the CNCF Foundation. Tencent is actively involved with open source communities such as Kubernetes, Spring Cloud, MariaDB, and it is also promoting the implementation of open source projects, products and solutions.

Foundations provide intellectual property management framework, code repositories, issue tracking, technical guidance, project governance, financial and public relations, and other services. By participating in these foundations, Tencent has learned many more mature open source organizational governance models and used them to guide the process to open source its internal projects to the world.

Tencent has open sourced more than 130 independent cloud native, big data, artificial intelligence, database, and other projects to the world. These projects obtained more than 370,000 GitHub Stars, and have exceeded 2,000 contributors.

By open sourcing projects across internal departments and to the world, actively collaborating with developers and communities around the world, and training open source talents, Tencent's open source ecosystem continuously improves and grows. 


Tencent Cloud’s future will focus on further strengthening the construction of Tencent's open source ecosystem through three major projects:

1) WeOpen: Tencent Open Source community

Tencent Cloud aims to build WeOpen, a platform for open source communication, promotion, and project incubation.

WeOpen is committed to connecting enthusiasts, practitioners, and leaders with global open source foundations to initiate new projects, co-create communities, and hold activities that extend open source culture and inspire the global open source ecosystem to flourish. 

2) Establish an industry open source joint laboratory

The open source laboratory is the landing place for the actual projects. Tencent Cloud plans to successively establish industry joint open source laboratories with well-known Chinese universities and open source organizations to provide a platform for students, researchers, and developers in the enterprise to contribute code and scenarios for open source projects to realize in the industry.

Starting with the "Rhino Bird Open Source Talent Cultivation Plan" held by Tencent this year, open source courses and practice training programs for college students help popularize open source culture, encourage contributions, and further the open source talent ecosystem.

3) Release "Cloud Native Open Source White Paper"

At the Cloud Native Industry Conference last May, Tencent Cloud and China's Institute of Information and Communications Technology announced the official preparation of the Cloud Native Open Source White Paper, which is expected to be released by the end of the year.

Tencent welcomes open source practitioners and companies to join their efforts in the above projects.

Conclusion

For more than 22 years, the ASF has proven that the Apache Way is one of the best practices for building an open organization that balances organizational structure and flexibility. Tencent continues to expand its own open source concepts, methodology, and ecosystem building, with plans to participate in more universities, companies, and organizations, while embracing the Apache Way into the future.


Mark Shan has a long career and practical experience in cloud-native, microservices, big data, edge computing, and open-source ecosystem. As the chairperson of Tencent Open Source Alliance, he works full of passion to build the ecosystem for Tencent Open Source and makes great efforts to accelerate innovation in technology and product with the open-source way. At Tencent Cloud, Mark leads the open-source team and works with organizations and communities including Apache Software Foundation,Linux Foundation, Open Atom Foundation, CAICT, COPU and others to build open-source ecosystem. He is also the observer of Linux Foundation Board, chairperson of TARS Foundation, TOC member of Open Atom Foundation and Magnolia Open Source Community, TSC member of Akraino Edge Stack, fellow of China Cloud Native Industry Alliance, advisor of Open Source Community.

= = =

"Success at Apache" is a monthly blog series that focuses on the processes behind why the ASF "just works". "Sponsor Success at Apache" features insights and experiences by select ASF Sponsors https://apache.org/foundation/thanks.html

For more Success at Apache posts, visit https://blogs.apache.org/foundation/category/SuccessAtApache