Struggling With DolphinScheduler Setup? This FAQ Can Help

Apache DolphinScheduler, as a scalable workflow scheduling system, is widely used for big data task orchestration. However, users may encounter issues such as environment setup, service startup, and workflow execution actual use. This article, combined with community documents and user practice experience, organizes the following frequently asked questions and detailed solutions to help users quickly locate and solve. I. Installation and Deployment Issues Environment Dependency Configuration Error Issue: Deployment fails due to missing JDK Maven, or database configuration. Solution: Install JDK 1.8 and configure the JAVA_HOME environment variable. Install 3.6 , set MAVEN_HOME and add it to the system PATH. The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). PseudoCluster Deployment Failure Issue: Services cannot start in pseudo-cluster mode due to SSH passwordless configuration or permission issues. Solution: Ensure that the deployment user (such as dolphinscheduler) has sudo passwordless permissions and modify the /etc/sudoers file to comment Defaults requiretty. Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict Issue: After version upgrade, it depends on a specific version of ZooKeeper (such as CDH'sK3.4.5). Solution: Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4. II. Service Startup and Configuration Issues Service Startup Failure -Issue**: Port occupation, dependent services not started, or log errors. Solution: Check port occupation: use netstat -ano |str :12345 (default UI port). View log files: such as logs/dolphin-scheduler-start.log, to locate the of the error. Ensure that dependent services (such as the database, ZooKeeper) have been started. Database Connection Failure Issue: The service cannot connect due to incorrect metadata database configuration. Solution: Modify bin/env/dolphinscheduler_env.sh, confirm the URL, username, and password. For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. III. Version Upgrade and Compatibility Issues Cross-version Upgrade Failure Issue: Compatibility issues when upgrading directly from 1.3 to 3.x. Solution: Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Python SDK Compatibility Issue: Python gateway unable to execute workflows or version incompatibility. Solution: - Upgrade the Python SDK to a version that supports the 3.2.x scheduler and adapt to the latest Java trigger code. Python SDK Compatibility - Issue: What stable version is currently recommended for use in production environments? Solution: It is recommended to use version 31.9. IV. Resource Management and Service Operation File Storage Configuration Issues Issue: Tasks cannot read resource files when using Minio or S3 storage. Solution: Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties. Service Crash and Data Compensation Failure Issue: Data compensation is not automatically triggered after service restart. Solution: Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). V. Community Resources and Advanced Support Official Documentation: Refer to the DolphinScheduler official website for deployment guides and parameter configurations. Support: Join the WeChat group or get real-time answers through the Weekly FAQ. Contribution Guide: Beginners can start with document fixes or simple code issues to participate in community contributions. By the above solutions, users can systematically address common issues in DolphinScheduler. If the problem still cannot be resolved, it is recommended to further investigate by combining log analysis and support. Apache DolphinScheduler, as a scalable workflow scheduling system, is widely used for big data task orchestration. However, users may encounter issues such as environment setup, service startup, and workflow execution actual use. This article, combined with community documents and user practice experience, organizes the following frequently asked questions and detailed solutions to help users quickly locate and solve. I. Installation and Deployment Issues I. Installation and Deployment Issues Environment Dependency Configuration Error Issue: Deployment fails due to missing JDK Maven, or database configuration. Solution: Install JDK 1.8 and configure the JAVA_HOME environment variable. Install 3.6 , set MAVEN_HOME and add it to the system PATH. The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). PseudoCluster Deployment Failure Issue: Services cannot start in pseudo-cluster mode due to SSH passwordless configuration or permission issues. Solution: Ensure that the deployment user (such as dolphinscheduler) has sudo passwordless permissions and modify the /etc/sudoers file to comment Defaults requiretty. Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict Issue: After version upgrade, it depends on a specific version of ZooKeeper (such as CDH'sK3.4.5). Solution: Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4. Environment Dependency Configuration Error Issue: Deployment fails due to missing JDK Maven, or database configuration. Solution: Install JDK 1.8 and configure the JAVA_HOME environment variable. Install 3.6 , set MAVEN_HOME and add it to the system PATH. The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). Environment Dependency Configuration Error Issue: Deployment fails due to missing JDK Maven, or database configuration. Solution: Install JDK 1.8 and configure the JAVA_HOME environment variable. Install 3.6 , set MAVEN_HOME and add it to the system PATH. The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). Issue : Deployment fails due to missing JDK Maven, or database configuration. Issue Solution : Install JDK 1.8 and configure the JAVA_HOME environment variable. Install 3.6 , set MAVEN_HOME and add it to the system PATH. The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). Solution Install JDK 1.8 and configure the JAVA_HOME environment variable. Install 3.6 , set MAVEN_HOME and add it to the system PATH. The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). Install JDK 1.8 and configure the JAVA_HOME environment variable. Install JDK 1.8 and configure the JAVA_HOME environment variable. JAVA_HOME Install 3.6 , set MAVEN_HOME and add it to the system PATH. Install 3.6 , set MAVEN_HOME and add it to the system PATH. MAVEN_HOME The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). The database must support MySQL 5.7 or PostgreSQL8.2.15 , and the metadata database should be correctly initialized (e.g., execute CREATE DATABASE and user authorization). CREATE DATABASE PseudoCluster Deployment Failure Issue: Services cannot start in pseudo-cluster mode due to SSH passwordless configuration or permission issues. Solution: Ensure that the deployment user (such as dolphinscheduler) has sudo passwordless permissions and modify the /etc/sudoers file to comment Defaults requiretty. Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict Issue: After version upgrade, it depends on a specific version of ZooKeeper (such as CDH'sK3.4.5). Solution: Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4. PseudoCluster Deployment Failure Issue: Services cannot start in pseudo-cluster mode due to SSH passwordless configuration or permission issues. Solution: Ensure that the deployment user (such as dolphinscheduler) has sudo passwordless permissions and modify the /etc/sudoers file to comment Defaults requiretty. Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict Issue: After version upgrade, it depends on a specific version of ZooKeeper (such as CDH'sK3.4.5). Solution: Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4. Issue : Services cannot start in pseudo-cluster mode due to SSH passwordless configuration or permission issues. Issue Solution : Ensure that the deployment user (such as dolphinscheduler ) has sudo passwordless permissions and modify the /etc/sudoers file to comment Defaults requiretty . Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict Solution dolphinscheduler sudo /etc/sudoers Defaults requiretty Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict Configure SSH passwordless login: after generating the key, use ssh-copy-id to distribute the public key to the cluster nodes3. ZooKeeper Dependency Conflict ssh-copy-id ZooKeeper Dependency Conflict Issue : After version upgrade, it depends on a specific version of ZooKeeper (such as CDH'sK3.4.5). Issue Solution : Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4. Solution Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4. Specify the parameter when recompiling DolphinScheduler, for example, mvn clean installPrelease -Dzk-3.4 . mvn clean installPrelease -Dzk-3.4 II. Service Startup and Configuration Issues II. Service Startup and Configuration Issues Service Startup Failure -Issue**: Port occupation, dependent services not started, or log errors. Solution: Check port occupation: use netstat -ano |str :12345 (default UI port). View log files: such as logs/dolphin-scheduler-start.log, to locate the of the error. Ensure that dependent services (such as the database, ZooKeeper) have been started. Database Connection Failure Issue: The service cannot connect due to incorrect metadata database configuration. Solution: Modify bin/env/dolphinscheduler_env.sh, confirm the URL, username, and password. For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. Service Startup Failure -Issue**: Port occupation, dependent services not started, or log errors. Solution: Check port occupation: use netstat -ano |str :12345 (default UI port). View log files: such as logs/dolphin-scheduler-start.log, to locate the of the error. Ensure that dependent services (such as the database, ZooKeeper) have been started. Service Startup Failure Solution: Check port occupation: use netstat -ano |str :12345 (default UI port). View log files: such as logs/dolphin-scheduler-start.log, to locate the of the error. Ensure that dependent services (such as the database, ZooKeeper) have been started. Solution : Check port occupation: use netstat -ano |str :12345 (default UI port). View log files: such as logs/dolphin-scheduler-start.log, to locate the of the error. Ensure that dependent services (such as the database, ZooKeeper) have been started. Solution Check port occupation: use netstat -ano |str :12345 (default UI port). View log files: such as logs/dolphin-scheduler-start.log, to locate the of the error. Ensure that dependent services (such as the database, ZooKeeper) have been started. Check port occupation: use netstat -ano |str :12345 (default UI port). netstat -ano |str :12345 View log files: such as logs/dolphin-scheduler-start.log , to locate the of the error. logs/dolphin-scheduler-start.log Ensure that dependent services (such as the database, ZooKeeper) have been started. Database Connection Failure Issue: The service cannot connect due to incorrect metadata database configuration. Solution: Modify bin/env/dolphinscheduler_env.sh, confirm the URL, username, and password. For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. Database Connection Failure Issue: The service cannot connect due to incorrect metadata database configuration. Solution: Modify bin/env/dolphinscheduler_env.sh, confirm the URL, username, and password. For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. Issue : The service cannot connect due to incorrect metadata database configuration. Issue Solution : Modify bin/env/dolphinscheduler_env.sh, confirm the URL, username, and password. For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. Solution Modify bin/env/dolphinscheduler_env.sh, confirm the URL, username, and password. For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. Modify bin/env/dolphinscheduler_env.sh , confirm the URL, username, and password. bin/env/dolphinscheduler_env.sh For MySQL 8 , manually download the mysql-connector-java driver (8.0.16 ) and it in the libs directory of each module. mysql-connector-java libs III. Version Upgrade and Compatibility Issues III. Version Upgrade and Compatibility Issues Cross-version Upgrade Failure Cross-version Upgrade Failure Cross-version Upgrade Failure Issue: Compatibility issues when upgrading directly from 1.3 to 3.x. Solution: Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Issue : Compatibility issues when upgrading directly from 1.3 to 3.x. Solution: Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Issue Solution: Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Solution : Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Solution Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Use a step-by-step: 1.3 → 2.0.x → 3.0.x→3.2.x, refer to the official upgrade documentation and test data. Python SDK Compatibility Issue: Python gateway unable to execute workflows or version incompatibility. Solution: - Upgrade the Python SDK to a version that supports the 3.2.x scheduler and adapt to the latest Java trigger code. Python SDK Compatibility - Issue: What stable version is currently recommended for use in production environments? Solution: It is recommended to use version 31.9. Python SDK Compatibility Issue: Python gateway unable to execute workflows or version incompatibility. Solution: - Upgrade the Python SDK to a version that supports the 3.2.x scheduler and adapt to the latest Java trigger code. Python SDK Compatibility Issue: Python gateway unable to execute workflows or version incompatibility. Solution: - Upgrade the Python SDK to a version that supports the 3.2.x scheduler and adapt to the latest Java trigger code. Issue : Python gateway unable to execute workflows or version incompatibility. Issue Solution : - Upgrade the Python SDK to a version that supports the 3.2.x scheduler and adapt to the latest Java trigger code. Solution Python SDK Compatibility - Issue : What stable version is currently recommended for use in production environments? Solution: It is recommended to use version 31.9. Issue Solution: It is recommended to use version 31.9. Solution : It is recommended to use version 31.9. Solution It is recommended to use version 31.9. It is recommended to use version 31.9. IV. Resource Management and Service Operation IV. Resource Management and Service Operation File Storage Configuration Issues Issue: Tasks cannot read resource files when using Minio or S3 storage. Solution: Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties. Service Crash and Data Compensation Failure Issue: Data compensation is not automatically triggered after service restart. Solution: Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). File Storage Configuration Issues Issue: Tasks cannot read resource files when using Minio or S3 storage. Solution: Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties. File Storage Configuration Issues Issue: Tasks cannot read resource files when using Minio or S3 storage. Solution: Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties. Issue : Tasks cannot read resource files when using Minio or S3 storage. Issue Solution : Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties. Solution Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties. Check if the resource reference path is correct and ensure that storage service (such as HDFS/S3/Minio) is configured in common.properties . common.properties Service Crash and Data Compensation Failure Issue: Data compensation is not automatically triggered after service restart. Solution: Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). Service Crash and Data Compensation Failure Issue: Data compensation is not automatically triggered after service restart. Solution: Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). Issue: Data compensation is not automatically triggered after service restart. Issue : Data compensation is not automatically triggered after service restart. Issue Solution: Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). Solution : Solution Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). Confirm that the workflow timing configuration is correct (start/end) and check the status of the service registration center (such as ZooKeeper). V. Community Resources and Advanced Support V. Community Resources and Advanced Support Official Documentation: Refer to the DolphinScheduler official website for deployment guides and parameter configurations. Support: Join the WeChat group or get real-time answers through the Weekly FAQ. Contribution Guide: Beginners can start with document fixes or simple code issues to participate in community contributions. Official Documentation: Refer to the DolphinScheduler official website for deployment guides and parameter configurations. Official Documentation: DolphinScheduler official website Support : Join the WeChat group or get real-time answers through the Weekly FAQ . Support Weekly FAQ Contribution Guide : Beginners can start with document fixes or simple code issues to participate in community contributions. Contribution Guide By the above solutions, users can systematically address common issues in DolphinScheduler. If the problem still cannot be resolved, it is recommended to further investigate by combining log analysis and support.