Middleware to Ecosystem Evolution: Understanding New Features of Apache ShardingSphere 5.0.0

Having undergone nearly two years of development and polishing with versions 5.0.0-Alpha and 5.0.0-Beta, Apache ShardingSphere 5.0.0 GA version will finally be available in November.

Coincidentally November 10th marks the 3rd anniversary of ShardingSphere’s entry into the Apache Software Foundation. The ShardingSphere core team has been working hard together with the community to meet this special deadline and release the 5.0.0 GA version to celebrate the third anniversary with the broader community, and the entire distributed database and security ecosystem.

Since the early development stages of version 5.0.0, ShardingSphere started to move away from industry positioning as a simple distributed database middleware solution. The development North-Star has been the Database Plus concept, which will allow a new positioning for 5.0.0 and a development outlook for future versions.

ShardingSphere recreates the distributed pluggable system, connecting actual user implementation scenarios and contributing valuable solutions to the community and the database industry.

Database Plus sets out to build a standard layer and an ecosystem layer above the fragmented database’s basic services. A unified and standardized database usage specification is provided for upper-level applications, and the challenges faced by businesses due to underlying databases fragmentation are minimized as much as possible. To link databases and applications, we use traffic and data rendering and parsing and provide users with enhanced core features such as distributed database, data security, database gateway, and stress testing. With the support of this concept, version 5.0.0 GA brings the following upgrades and optimizations:

Architecture Layer

Complete the pluggable kernel architecture of Database Plus

Foundation Layer: Provide a variety of access terminals and access forms to flexibly meet the needs of users in different scenarios.

Plugin Layer: As a pluggable core architecture, it provides infrastructure support. Function Layer: Provide a variety of functional plug-ins that meet the needs of users, allowing users a high degree of flexibility in plug-in choice and combination.

Product Layer: end-users are provided with industry-oriented and specific scenario-oriented standard product solutions.

Multi-Access terminal mixed-mode production available

ShardingSphere JDBC and ShardingSphere Proxy have been polished and tested for two years and are now available in production. Many community users provided relevant production community cases, and production feasibility has been verified.

Additionally, SphereEx leverages the ShardingSphere ecosystem and will provide the ShardingSphere Sidecar access terminal POC Version for Cloud Native based on the ShardingSphere ecosystem at the beginning of 2022 (to do).

With the shared core architecture and different ShardingSphere adapters, users can also choose hybrid adapter deployment if their production environment requires them to do so (shown in the figure below).

Standardized cluster management with DistSQL

Apache ShardingSphere first proposed the unique SQL dialect of DistSQL (distributed SQL) to connect all elements of the ShardingSphere ecosystem together. As the standard interaction language of the ShardingSphere distributed database ecosystem, DistSQL allows users to use one SQL command to create, modify or delete a distributed database table, or to encrypt or decrypt it. DistSQL also supports distributed scheduling management.

Significantly improved distributed governance capabilities

In the ShardingSphere ecosystem where computing and storage are separated, features are greatly enhanced in the new version including the distributed governance of databases (i.e., storage nodes) and Proxy/JDBC (i.e. computing nodes), online user metadata DDL change, on/off running storage nodes and computing nodes, circuit breaker and disabling, and high availability. Additionally, the new feature distributed lock is scheduled to be released soon.

Fully upgraded APM and monitoring display capabilities

In the new version, we optimize and add more runtime status indicators to monitor ShardingSphere. Through the agent dynamic loading mechanism, we provide users with various indicators for metrics and tracking, making it convenient to integrate the APM system with the Grafana dashboard.

Function Layer

In the future, the community will continue to optimize ShardingSphere and integrate new ideas and industry scenarios, while always listening to users’ feedback. Meanwhile, the core team members of the ShardingSphere community are willing to mentor anyone interested in open source and to provide more practice “ISSUES” to students interested in open source. We also hope that new friends or contributors will join the community, to promote ideas exchange and create a truly global developer community.

Author:

Sphere Ex Co-Founder & CTO; Apache Member; Apache ShardingSphere PMC, Apache brpc (Incubating) & Apache AGE (Incubating) Mentor; China Mulan Open Source Community Mentor.

She used to be responsible for the design and development of the intelligent database platform of JD Digital Science and Technology. She now focuses on the distributed database & middleware ecosystem, and open-source community. Recipient of the “2020 China Open-Source Pioneer” award & the “2021 OSCAR 2021 Top Open Source Pioneer”.

Links: Bio: https://tristazero.github.io LinkedIn: https://www.linkedin.com/in/panjuan GitHub: https://github.com/tristaZero Twitter: https://twitter.com/tristaZero

Open Source Project Links: ShardingSphere Github: https://github.com/apache/shardingsphere ShardingSphere Twitter: https://twitter.com/ShardingSphere ShardingSphere Slack Channel: https://join.slack.com/t/apacheshardingsphere/shared_invite/zt-sbdde7ie-SjDqo9\~I4rYcR18bq0SYTg