Recently, Apache DolphinScheduler released version 3.3.0 alpha. The 3.3.0 alpha version introduces breakthrough improvements in task management, fault tolerance, and scalability, offering a more stable and flexible solution for large-scale workflow scheduling. However, please note that this is an alpha version—upgrade with caution. Try it now: Try it now: Source code download: https://github.com/apache/dolphinscheduler/archive/refs/tags/3.3.0-alpha.zip
Upgrade guide: Please refer to the community documentation Source code download: https://github.com/apache/dolphinscheduler/archive/refs/tags/3.3.0-alpha.zip https://github.com/apache/dolphinscheduler/archive/refs/tags/3.3.0-alpha.zip Upgrade guide: Please refer to the community documentation This update includes numerous DSIPs (DolphinScheduler Improvement Proposals), Features (new functionalities), and Improvements. Below is a summary of the release highlights. For full update details, check the Release Note: https://github.com/apache/dolphinscheduler/releases/tag/3.3.0-alpha For full update details, check the Release Note: https://github.com/apache/dolphinscheduler/releases/tag/3.3.0-alpha https://github.com/apache/dolphinscheduler/releases/tag/3.3.0-alpha DSIP (DolphinScheduler Improvement Proposals) The following are structural improvement proposals included in the DolphinScheduler 3.3.0 alpha version, bringing systematic changes to core functionalities, architecture, and performance: ✅ Enhanced Connection Center Support Enhanced Connection Center Support DSIP-19 series: Added support for Zeppelin, SageMaker, K8s connections (by @xdu-chenrj)
DSIP-27: Java task plugin improvements (by @ailiujiarui) DSIP-19 series: Added support for Zeppelin, SageMaker, K8s connections (by @xdu-chenrj) DSIP-19 series DSIP-27: Java task plugin improvements (by @ailiujiarui) DSIP-27 🧾 Audit Logs and Remote Logs Optimization Audit Logs and Remote Logs Optimization DSIP-24 / 25 / 26 / 57: Upgraded architecture for remote and audit logs with rollback support (by @pegasas, @qingwli, @SbloodyS) DSIP-24 / 25 / 26 / 57: Upgraded architecture for remote and audit logs with rollback support (by @pegasas, @qingwli, @SbloodyS) DSIP-24 / 25 / 26 / 57 🔨 Master/Task Execution Logic Optimization Master/Task Execution Logic Optimization DSIP-44 / 54 / 56 / 61 / 67 / 65 / 73 / 87: Optimizations from thread pool scheduling, task execution abstraction, to scheduling command mechanisms (by @ruanwenjun) DSIP-44 / 54 / 56 / 61 / 67 / 65 / 73 / 87: Optimizations from thread pool scheduling, task execution abstraction, to scheduling command mechanisms (by @ruanwenjun) DSIP-44 / 54 / 56 / 61 / 67 / 65 / 73 / 87 💡 User Experience Optimization User Experience Optimization DSIP-50 / 51 / 59 / 60 / 62 / 68 / 72 / 78: Removed redundant plugins/modules and unified UI structure (by @SbloodyS) DSIP-50 / 51 / 59 / 60 / 62 / 68 / 72 / 78: Removed redundant plugins/modules and unified UI structure (by @SbloodyS) DSIP-50 / 51 / 59 / 60 / 62 / 68 / 72 / 78 🧰 Other Improvements Other Improvements DSIP-75: Automatically add license headers (by @ruanwenjun)
DSIP-70: Removed upgrade code for versions 1.x/2.x (by @SbloodyS)
DSIP-38: Optimized plugin upload/download and dependencies (by @SbloodyS) DSIP-75: Automatically add license headers (by @ruanwenjun) DSIP-75 DSIP-70: Removed upgrade code for versions 1.x/2.x (by @SbloodyS) DSIP-70 DSIP-38: Optimized plugin upload/download and dependencies (by @SbloodyS) DSIP-38 Feature (New Features) 🎧 Monitoring and Alerts Added Listener mechanism (#14981)
Supported Prometheus AlertManager, Aliyun Voice, HTTP test sending (#15079, #15248, #15163) Added Listener mechanism (#14981) Listener mechanism Supported Prometheus AlertManager, Aliyun Voice, HTTP test sending (#15079, #15248, #15163) 🧪 New Plugins and Integrations Supported OceanBase DataX tasks (#16281)
Supported EMR Serverless Spark (#16126)
Supported param passing for Dinky task Prepare Params (#15947) Supported OceanBase DataX tasks (#16281) Supported EMR Serverless Spark (#16126) Supported param passing for Dinky task Prepare Params (#15947) 🧱 Resource and Permission Management UI now displays resource paths; added Worker Group permission management (#16590, #15600) UI now displays resource paths; added Worker Group permission management (#16590, #15600) 🗂 Storage Extensions Added Tencent Cloud COS storage plugin (#16565) Added Tencent Cloud COS storage plugin (#16565) 📄 Documentation Enhancements Security model documentation and standardized security naming (#16609, #16587) Security model documentation and standardized security naming (#16609, #16587) Improvement (Optimizations) ⚙ Core Modules and Performance Master/Worker thread pools, Zookeeper timeout optimization, RPC retry mechanisms, and dependency task handling improvements (multiple PRs by @ruanwenjun) Master/Worker thread pools, Zookeeper timeout optimization, RPC retry mechanisms, and dependency task handling improvements (multiple PRs by @ruanwenjun) 🧪 Test Coverage Improvements Enhanced UT and E2E test coverage and filled in missing test cases (numerous PRs by @abzymeinsjtu) Enhanced UT and E2E test coverage and filled in missing test cases (numerous PRs by @abzymeinsjtu) 🧹 Cleanup and Refactoring Removed redundant code, optimized logging, and standardized variable naming (multiple PRs by @SbloodyS, @caicancai, @Gallardot) Removed redundant code, optimized logging, and standardized variable naming (multiple PRs by @SbloodyS, @caicancai, @Gallardot) 📦 Deployment/Documentation Enhancements Fixed documentation typos, added Helm config details, optimized Docker Hub usage and installation flow (multiple PRs by @SbloodyS, @Gallardot) Fixed documentation typos, added Helm config details, optimized Docker Hub usage and installation flow (multiple PRs by @SbloodyS, @Gallardot) Bug Fix The following modules were the focus of bug fixes during this release cycle: Core Scheduling Module:Fixed issues like sub-workflows continuing to run while in PAUSE state (#16564), dependency task status parsing errors (#16599), incorrect global parameter passing (#16376), and scheduling anomalies for conditional dependency tasks in WAITING_DEPEND state (#16232), improving stability and correctness.


Plugin-Related Issues:Fixed a NullPointerException in the Elasticsearch plugin (#16107) and potential failures in multi-threaded file downloads in the file plugin (#16063), enhancing plugin robustness and compatibility.


Data Source Configuration Issues:Addressed incorrect parsing of data source parameters (#15874) and resolved MySQL connection failures in some JDBC versions (#16376), improving connection adaptability.


Frontend Display Issues:Fixed a naming error in the DATASOURCE_CATALOG permission module that caused UI display issues (#16590), and resolved discrepancies between task state display and backend data (#16594), improving frontend usability.


Test Case Fixes:Fixed E2E test failures caused by faulty plugin dependencies (#15892) and addressed instability in some unit tests under specific environments (#16120), enhancing test system reliability. Core Scheduling Module:Fixed issues like sub-workflows continuing to run while in PAUSE state (#16564), dependency task status parsing errors (#16599), incorrect global parameter passing (#16376), and scheduling anomalies for conditional dependency tasks in WAITING_DEPEND state (#16232), improving stability and correctness. Core Scheduling Module:Fixed issues like sub-workflows continuing to run while in PAUSE state (#16564), dependency task status parsing errors (#16599), incorrect global parameter passing (#16376), and scheduling anomalies for conditional dependency tasks in WAITING_DEPEND state (#16232), improving stability and correctness. Core Scheduling Module PAUSE WAITING_DEPEND Plugin-Related Issues:Fixed a NullPointerException in the Elasticsearch plugin (#16107) and potential failures in multi-threaded file downloads in the file plugin (#16063), enhancing plugin robustness and compatibility. Plugin-Related Issues:Fixed a NullPointerException in the Elasticsearch plugin (#16107) and potential failures in multi-threaded file downloads in the file plugin (#16063), enhancing plugin robustness and compatibility. Plugin-Related Issues Data Source Configuration Issues:Addressed incorrect parsing of data source parameters (#15874) and resolved MySQL connection failures in some JDBC versions (#16376), improving connection adaptability. Data Source Configuration Issues:Addressed incorrect parsing of data source parameters (#15874) and resolved MySQL connection failures in some JDBC versions (#16376), improving connection adaptability. Data Source Configuration Issues Frontend Display Issues:Fixed a naming error in the DATASOURCE_CATALOG permission module that caused UI display issues (#16590), and resolved discrepancies between task state display and backend data (#16594), improving frontend usability. Frontend Display Issues:Fixed a naming error in the DATASOURCE_CATALOG permission module that caused UI display issues (#16590), and resolved discrepancies between task state display and backend data (#16594), improving frontend usability. Frontend Display Issues DATASOURCE_CATALOG Test Case Fixes:Fixed E2E test failures caused by faulty plugin dependencies (#15892) and addressed instability in some unit tests under specific environments (#16120), enhancing test system reliability. Test Case Fixes:Fixed E2E test failures caused by faulty plugin dependencies (#15892) and addressed instability in some unit tests under specific environments (#16120), enhancing test system reliability. Test Case Fixes Document (Documentation Updates) This release includes updates across installation guides, contributor documentation, and configuration references: Installation and Deployment Docs:Updated Helm installation instructions, clarified deployment steps, and documented Chart parameter configs for easier setup (#16604).


Contributor Documentation:Improved contribution guidelines with new PR title conventions and submission flowcharts to help new contributors participate more effectively (#16294, #16125).


Configuration and Usage Docs:Added details for task dependencies, data source configurations, and alert strategies (#15972, #16097, #16407), improving readability and accuracy. Installation and Deployment Docs:Updated Helm installation instructions, clarified deployment steps, and documented Chart parameter configs for easier setup (#16604). Installation and Deployment Docs:Updated Helm installation instructions, clarified deployment steps, and documented Chart parameter configs for easier setup (#16604). Installation and Deployment Docs Contributor Documentation:Improved contribution guidelines with new PR title conventions and submission flowcharts to help new contributors participate more effectively (#16294, #16125). Contributor Documentation:Improved contribution guidelines with new PR title conventions and submission flowcharts to help new contributors participate more effectively (#16294, #16125). Contributor Documentation Configuration and Usage Docs:Added details for task dependencies, data source configurations, and alert strategies (#15972, #16097, #16407), improving readability and accuracy. Configuration and Usage Docs:Added details for task dependencies, data source configurations, and alert strategies (#15972, #16097, #16407), improving readability and accuracy. Configuration and Usage Docs Chore (Miscellaneous Optimizations) Chore updates focus on CI processes, module cleanup, and partial refactoring: CI/CD Process Optimization: Introduced a Mock Server to simulate scheduling execution, accelerating E2E test speed and improving stability (#16030); added compatibility for scheduling failures under multi-task concurrency (#16139); simplified CI logic and E2E execution commands (#16643).
Module Cleanup and Refactoring :Removed unused old modules, plugins, and invalid test code, simplifying the codebase and improving clarity and maintainability (#15973, #16588). CI/CD Process Optimization: Introduced a Mock Server to simulate scheduling execution, accelerating E2E test speed and improving stability (#16030); added compatibility for scheduling failures under multi-task concurrency (#16139); simplified CI logic and E2E execution commands (#16643). CI/CD Process Optimization Module Cleanup and Refactoring :Removed unused old modules, plugins, and invalid test code, simplifying the codebase and improving clarity and maintainability (#15973, #16588). Module Cleanup and Refactoring Contributors This release was made possible by the collaboration of community developers. Special thanks to the following core contributors (partial list): Ruan Wenjun (PMC Member): Led architectural design and core feature development.
Thanks to Gallardot, weideguo, and a total of 71 contributors (GitHub IDs below): Ruan Wenjun (PMC Member): Led architectural design and core feature development. Ruan Wenjun Thanks to Gallardot, weideguo, and a total of 71 contributors (GitHub IDs below): @Blue-Fox, @ZarakiKenpachi, @zhangyongqin, @ironnight, @liuchuo, @yehfan, @CaIShunfeng, @yuxiaoyang-cn, @kongchongming, @MasahiroZhang, @wuxuqiang, @gaoqiang173, @ckeeper97, @twocolds, @hvhye, @liujx0429, @lingkong-jiankong, @jiangm2019, @izukayy, @derryyy, @serpywa, @2rinn, @ilyaszq, @weidaokun, @codenavie, @peijie-liu, @mario27547, @yassirlr, @mingxiaojun, @lixunxd, @wmm2019, @Shawn1997, @nobida, @Chen-lina, @hzytong, @jiangang-huawei, @zhangchaofan, @dongchunlin, @HXT-DAO, @zhaozhichao, @YucongZhu, @onakobayashi, @sdh2110, @laitinglin, @wumomo-npm, @tang-xi, @Maxwell-nz, @bigfrog828, @flarv, @buqiong, @peicongl, @xiayuheng, @kokaokami, @ztust-hsy, @ChengYu-Cloud, @sxt5699, @0d7000, @suxianglong, @one-man-team, @zhouyi0410, @tianxuyuan, @heguanghu, @LotharHuber, @chongxiao-qw, @mcz1988, @windzhe, @linhong-ua, @sunkiming, @nothedream, @kenbliu, @klovejun, @ywq1024, @jianghaitao

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Apache

GitHub

The $0 Scheduler That Almost Cost a Company Everything

Master DolphinScheduler Time Parameter With These Dynamic Configuration Secrets

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Apache DolphinScheduler Brings Major Enhancements and Performance Upgrades in Latest Release

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

(I) Principles of Data Model Architecture: Four Layers and Seven Stages

How to Seamlessly Transfer From Airflow to Apache DolphinScheduler With Air2phin

A Comprehensive Guide to Building DolphinScheduler 3.2.0 Production-Grade Cluster Deployment

How to Regularly Delete Log Instances in DolphinScheduler

How to Enable Auto-Start for Apache DolphinScheduler

(I) Principles of Data Model Architecture: Four Layers and Seven Stages

How to Seamlessly Transfer From Airflow to Apache DolphinScheduler With Air2phin

A Comprehensive Guide to Building DolphinScheduler 3.2.0 Production-Grade Cluster Deployment

How to Regularly Delete Log Instances in DolphinScheduler

How to Enable Auto-Start for Apache DolphinScheduler

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps