In any large organization, managing user identities is a constant challenge of balancing security with user convenience. For an enterprise-grade workflow orchestration platform like Apache DolphinScheduler, robust and flexible authentication is not just a feature-it's a necessity. Previously, DolphinScheduler offered several login options, including Password, LDAP, and Casdoor SSO. However, these methods had limitations, such as a high dependency on the Casdoor project or an inflexible OAuth implementation, making it challenging to integrate with diverse enterprise identity systems. 🤔 What is OIDC? A Simple Analogy Think of OIDC as a universal digital passport. Instead of creating a new, separate account for every service you use (like DolphinScheduler), you present a trusted passport from an identity provider (like Google, Keycloak, or your company's internal login system) to prove who you are. universal digital passport Technically, OIDC is a modern security standard that works as an identity layer on top of the OAuth 2.0 protocol. While OAuth 2.0 is about authorization (what you're allowed to do), OIDC is about authentication (confirming who you are) . OAuth 2.0 This approach is ideal for DolphinScheduler because of its: Universality 🌐: It works with any compliant identity provider, including Keycloak, Okta, Azure AD, Dexidp, and even social/enterprise logins like Feishu and WeChat Work.Security 🛡️: It uses digitally signed ID Tokens (JWTs) to ensure the user's identity is authentic and has not been tampered with.Flexibility 🔌: It offers a plug-and-play experience, allowing organizations to easily connect DolphinScheduler to their existing identity infrastructure. Universality 🌐: It works with any compliant identity provider, including Keycloak, Okta, Azure AD, Dexidp, and even social/enterprise logins like Feishu and WeChat Work. Universality Security 🛡️: It uses digitally signed ID Tokens (JWTs) to ensure the user's identity is authentic and has not been tampered with. Security Flexibility 🔌: It offers a plug-and-play experience, allowing organizations to easily connect DolphinScheduler to their existing identity infrastructure. Flexibility ✨ Key OIDC Features Now in DolphinScheduler This GSoC project brings several powerful, enterprise-focused features to DolphinScheduler. True Single Sign-On (SSO): Users can now log in once with their primary identity provider and gain immediate access to DolphinScheduler without needing to remember another password. This creates a frictionless and efficient user experience.Centralized Identity Management: In an enterprise setting, administrators can manage all user access from a central Identity Provider (IdP) like Keycloak or Okta. When an employee joins or leaves the company, their access to DolphinScheduler can be granted or revoked from a single location, dramatically improving security and simplifying user lifecycle management.Automated User Provisioning & Dynamic Role Sync: When a user logs in via OIDC for the first time, DolphinScheduler automatically creates their account on-the-fly. By mapping group claims from your Identity Provider (like a 'dolphinscheduler-admins' group in Keycloak), you can automatically assign admin privileges, streamlining user onboarding. This role synchronization is re-evaluated on every login, ensuring that the Identity Provider remains the single source of truth for permissions. This means any role updates like promotions, demotions, or revocations; made in your central IdP are applied automatically the next time a user signs in.Enhanced Security with ID Tokens: OIDC provides a secure JSON Web Token (JWT) called an ID Token. This token is digitally signed by the IdP, which cryptographically guarantees that the user's identity is authentic. DolphinScheduler validates this token on every login, ensuring that only verified users gain access.Broad Compatibility with Any OIDC Provider: The core strength of this project is its generalized nature. By adhering to the OIDC standard, DolphinScheduler is no longer tied to a specific provider. It is now compatible with a vast ecosystem of identity solutions, including but not limited to:Open Source IdPs: Keycloak, DexidpCommercial IdPs: Okta, Auth0, Microsoft Entra ID (Azure AD)Cloud-Native Proxies: OAuthProxyEnterprise & Social Logins: Feishu, WeChat Work Login True Single Sign-On (SSO): Users can now log in once with their primary identity provider and gain immediate access to DolphinScheduler without needing to remember another password. This creates a frictionless and efficient user experience. True Single Sign-On (SSO) Centralized Identity Management: In an enterprise setting, administrators can manage all user access from a central Identity Provider (IdP) like Keycloak or Okta. When an employee joins or leaves the company, their access to DolphinScheduler can be granted or revoked from a single location, dramatically improving security and simplifying user lifecycle management. Centralized Identity Management Automated User Provisioning & Dynamic Role Sync: When a user logs in via OIDC for the first time, DolphinScheduler automatically creates their account on-the-fly. By mapping group claims from your Identity Provider (like a 'dolphinscheduler-admins' group in Keycloak), you can automatically assign admin privileges, streamlining user onboarding. This role synchronization is re-evaluated on every login, ensuring that the Identity Provider remains the single source of truth for permissions. This means any role updates like promotions, demotions, or revocations; made in your central IdP are applied automatically the next time a user signs in. Automated User Provisioning & Dynamic Role Sync Enhanced Security with ID Tokens: OIDC provides a secure JSON Web Token (JWT) called an ID Token. This token is digitally signed by the IdP, which cryptographically guarantees that the user's identity is authentic. DolphinScheduler validates this token on every login, ensuring that only verified users gain access. Enhanced Security with ID Tokens Broad Compatibility with Any OIDC Provider: The core strength of this project is its generalized nature. By adhering to the OIDC standard, DolphinScheduler is no longer tied to a specific provider. It is now compatible with a vast ecosystem of identity solutions, including but not limited to: Broad Compatibility with Any OIDC Provider Open Source IdPs: Keycloak, Dexidp Open Source IdPs Commercial IdPs: Okta, Auth0, Microsoft Entra ID (Azure AD) Cloud-Native Proxies: OAuthProxy Cloud-Native Proxies Enterprise & Social Logins: Feishu, WeChat Work Login Enterprise & Social Logins How It Works: A Look Under the Hood The OIDC integration follows the standard Authorization Code Flow, ensuring a secure and reliable authentication process. 🛠️ Step-by-Step Guide: How to Configure OIDC with Keycloak Setting up OIDC is straightforward. Here’s a quick guide using Keycloak and Dex Idp. Example 1: Using Keycloak Here’s a quick guide using Keycloak, a popular open-source identity provider. Step 1: Configure Your Identity Provider (Keycloak) Step 1: Configure Your Identity Provider (Keycloak) Step 1: Configure Your Identity Provider (Keycloak) In your Keycloak instance, you'll need to register DolphinScheduler as a new client. Create a Client: Give it a Client ID (e.g., dolphinscheduler-client). Create a Client: Give it a Client ID (e.g., dolphinscheduler-client). Create a Client: Give it a Client ID (e.g., dolphinscheduler-client) Set Redirect URIs: This is crucial. Add the callback URL for your DolphinScheduler API server, which follows the format: http://{api-host:port}/dolphinscheduler/login/oauth2/code/{provider-id}. For a local setup, this would be http://localhost:12345/dolphinscheduler/login/oauth2/code/keycloak Set Redirect URIs: This is crucial. Add the callback URL for your DolphinScheduler API server, which follows the format: http://{api-host:port}/dolphinscheduler/login/oauth2/code/{provider-id}. For a local setup, this would be http://localhost:12345/dolphinscheduler/login/oauth2/code/keycloak Set Redirect URIs http://{api-host:port}/dolphinscheduler/login/oauth2/code/{provider-id} http://{api-host:port}/dolphinscheduler/login/oauth2/code/{provider-id} http://localhost:12345/dolphinscheduler/login/oauth2/code/keycloak Get Credentials: Go to the "Credentials" tab and copy the Client Secret. Get Credentials: Go to the "Credentials" tab and copy the Client Secret. Get Credentials 💡 Quick Start: For testing, you can use the pre-configured realm-export.json file located at dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/resources/docker/oidc-login/realm-export.json to spin up a Keycloak instance with all the necessary settings already in place! realm-export.json dolphinscheduler-api-test/dolphinscheduler-api-test-case/src/test/resources/docker/oidc-login/realm-export.json Step 2: Configure DolphinScheduler Step 2: Configure DolphinScheduler Step 2: Configure DolphinScheduler Next, update the dolphinscheduler-api/src/main/resources/application.yamlfile with the credentials from Keycloak. dolphinscheduler-api/src/main/resources/application.yaml Example 2: Using Dexidp (to show generality) Here is how you would configure Dex, another popular OIDC identity provider, demonstrating the system's flexibility. Step 1: Configure Dex In your Dex configuration file, add DolphinScheduler to the staticClients list. Step 1: Configure Dex In your Dex configuration file, add DolphinScheduler to the staticClients list. Step 1 staticClients Example Dex config.yaml : Step 2: Configure DolphinScheduler Add a new provider entry in your application.yaml.Step 3: Log In! Step 2: Configure DolphinScheduler Add a new provider entry in your application.yaml. Step 2 application.yaml Step 3: Log In! Step 3 After restarting the DolphinScheduler API server, the login page will now feature buttons for both "Login with Keycloak" and "Login with Dex". The user flow is seamless for both! The user clicks the new login button.They are redirected to the Keycloak/Dex login page to enter their credentials.Upon success, they are sent back to the DolphinScheduler UI, fully authenticated. The user clicks the new login button. They are redirected to the Keycloak/Dex login page to enter their credentials. Upon success, they are sent back to the DolphinScheduler UI, fully authenticated. 👨‍💻 My GSoC Journey & Acknowledgements This project was an incredible learning experience, focused on a single core principle: creating a truly generalized authentication solution for the enterprise. My journey began with a deep dive into DolphinScheduler's architecture while tackling the local environment setup. With invaluable guidance from my mentors, I adopted a backend-first approach, building a solid foundation with the robust Nimbus SDK to ensure the core OIDC logic was both reliable and standard-compliant. truly generalized A key design decision was to make the Identity Provider the single source of truth. This was essential for enabling the dynamic role-sync feature that re-evaluates a user's permissions on every login, not just the first one, ensuring that access control is always up-to-date. This required updating the core user service and ensures that a user's access is always in sync with your central identity system. single source of truth Proving the flexibility and reliability of this feature was the next major challenge. A top priority was quality assurance, which I addressed with a multi-layered strategy: quality assurance Unit Tests: I developed a comprehensive suite of unit tests, navigating tricky mocking scenarios to validate every component and achieving 93.2% test coverage on all new code. I also took the opportunity to improve the test coverage of some existing user management components.End-to-End API Tests: I built a full-fledged API test suite using Testcontainers, which spins up a complete Docker environment (including a Keycloak instance) to validate the entire authentication flow automatically.Cross-Provider Validation: To ensure the "generalized" promise was met, I successfully tested the implementation against both Keycloak and Dex, proving its vendor-neutral capabilities. Unit Tests: I developed a comprehensive suite of unit tests, navigating tricky mocking scenarios to validate every component and achieving 93.2% test coverage on all new code. I also took the opportunity to improve the test coverage of some existing user management components. Unit Tests End-to-End API Tests: I built a full-fledged API test suite using Testcontainers, which spins up a complete Docker environment (including a Keycloak instance) to validate the entire authentication flow automatically. End-to-End API Tests Cross-Provider Validation: To ensure the "generalized" promise was met, I successfully tested the implementation against both Keycloak and Dex, proving its vendor-neutral capabilities. Cross-Provider Validation Keycloak and Dex The journey didn't end with the backend. I also developed the complete frontend experience, implementing the API calls and creating the responsive UI components that allow users to log in with the configured OIDC providers. complete frontend Finally, to support future development and community adoption, I created extensive documentation. This includes a step-by-step developer guide for local testing with Keycloak and detailed configuration examples for enterprise integration, which are now part of the official DolphinScheduler documentation. extensive documentation This project wouldn't have been possible without my fantastic mentors, Gallardot and ZiHao Xiang (SbloodyS), who provided invaluable guidance, feedback, and support throughout the summer. I'm also immensely grateful to the Apache DolphinScheduler community and the Google Summer of Code program for this amazing opportunity. Gallardot ZiHao Xiang (SbloodyS) 🎯 Conclusion: The Impact The introduction of generalized OIDC authentication is a significant step forward for Apache DolphinScheduler, delivering tangible benefits for everyone: For Administrators: Simplified user management, centralized access control, and an improved security posture.For Users: A seamless and secure login experience with fewer passwords to manage.For Organizations: Effortless integration of DolphinScheduler into existing and future enterprise identity ecosystems. For Administrators: Simplified user management, centralized access control, and an improved security posture. For Administrators For Users: A seamless and secure login experience with fewer passwords to manage. For Users For Organizations: Effortless integration of DolphinScheduler into existing and future enterprise identity ecosystems. For Organizations This feature makes DolphinScheduler more secure, user-friendly, and better prepared for the complex demands of modern data platforms. 🤝 Get Involved & Try It Out! I encourage you to explore the new OIDC authentication feature! All the work was done in the open, and you can review the complete implementation in the official pull request. 🔗 Main Pull Request: #17340 on GitHubGSoC Project Archive: Dolphinscheudler GSoC 2025 Project📚 Official Documentation: Authentication Guide👨‍💻 Connect with me:GitHub: @tusaryanLinkedIn: Aryan Kumar 🔗 Main Pull Request: #17340 on GitHub Main Pull Request #17340 on GitHub GSoC Project Archive: Dolphinscheudler GSoC 2025 Project GSoC Project Archive Dolphinscheudler GSoC 2025 Project 📚 Official Documentation: Authentication Guide Official Documentation Authentication Guide 👨‍💻 Connect with me: Connect with me GitHub: @tusaryan @tusaryan LinkedIn: Aryan Kumar Aryan Kumar Your feedback and contributions are always welcome! Thank you for being a part of this journey.

Apache

Microsoft

Apache DolphinScheduler Adopts OpenID Connect for Seamless Enterprise Authentication

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A 10-Minute Deep Dive Into the Core Architecture of Apache SeaTunnel and DataX

How to Seamlessly Transfer From Airflow to Apache DolphinScheduler With Air2phin

A Comprehensive Guide to Building DolphinScheduler 3.2.0 Production-Grade Cluster Deployment

How to Regularly Delete Log Instances in DolphinScheduler

How to Enable Auto-Start for Apache DolphinScheduler

A 10-Minute Deep Dive Into the Core Architecture of Apache SeaTunnel and DataX

How to Seamlessly Transfer From Airflow to Apache DolphinScheduler With Air2phin

A Comprehensive Guide to Building DolphinScheduler 3.2.0 Production-Grade Cluster Deployment

How to Regularly Delete Log Instances in DolphinScheduler

How to Enable Auto-Start for Apache DolphinScheduler

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps