Apache DolphinScheduler is a distributed, decentralized, and scalable visual DAG workflow task scheduling systeåm. It is committed to solving the complex dependencies in data processing workflows, making the scheduling system ready-to-use out of the box. Ambari is a tool for managing Hadoop clusters. Although Ambari does not directly support the installation of DolphinScheduler, we can integrate it into Ambari through a custom service. This article will describe how to use Ambari to install DolphinScheduler. 1. Ambari Service Installation Package Place the service installation package that needs to be integrated into Ambari in the following directory: /var/lib/ambari-server/resources/stacks/HDP/3.1/services 2. Restart the Ambari-server Service sudo ambari-server restart After the integration, you can proceed with the service installation on the Ambari web page. Add Service and Select DolphinScheduler 
DolphinScheduler Service Configuration Choose to use MySQL as the metadata database for the DolphinScheduler service.  
Service Selection for Installation Select two DS-master nodes, at least two DS-ALERTS, DS-APIs, and deploy the remaining nodes as DS-worker nodes.  
Error Handling for hue-desktop-ini If you have configured namenode-ha before installing DolphinScheduler, you may encounter errors related to 'hue-desktop-ini'. You need to modify the params.py script. Script location: /var/lib/ambari-server/resources/stacks/HDP/3.3/services/DOLPHINSCHEDULER/package/scripts/params.py

Modify the content:
# hdfs_httpfs_host = config['configurations']['hue-desktop-ini']['hdfs_httpfs_host']
hdfs_httpfs_host = "dfs_ha_enabled"  # Comment out the original line and assign any value to hdfs_httpfs_host After making the changes, save and restart the ambari-server service to reinstall DolphinScheduler. Copy MySQL Driver to All Nodes' libs Directory cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/alert-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/api-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/master-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/worker-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/3.3.1.0-004/dolphinscheduler/tools/libs/ Change Ownership of All Services to hdfs User chown -R hdfs:hdfs /var/log/dolphinscheduler
chown -R hdfs:hdfs /usr/hdp/3.3.1.0-004/dolphinscheduler
chown -R hdfs:hdfs /etc/dolphinscheduler/
ln -s /usr/hdp/3.1.4.0-315/dolphinscheduler dolphinscheduler After making these changes, start DolphinScheduler from the Ambari management platform. The metadata will be automatically initialized and tables will be created during the startup process. Apache DolphinScheduler is a distributed, decentralized, and scalable visual DAG workflow task scheduling systeåm. It is committed to solving the complex dependencies in data processing workflows, making the scheduling system ready-to-use out of the box. Ambari is a tool for managing Hadoop clusters. Although Ambari does not directly support the installation of DolphinScheduler, we can integrate it into Ambari through a custom service. This article will describe how to use Ambari to install DolphinScheduler. 1. Ambari Service Installation Package Place the service installation package that needs to be integrated into Ambari in the following directory: /var/lib/ambari-server/resources/stacks/HDP/3.1/services /var/lib/ambari-server/resources/stacks/HDP/3.1/services 2. Restart the Ambari-server Service sudo ambari-server restart sudo ambari-server restart After the integration, you can proceed with the service installation on the Ambari web page. Add Service and Select DolphinScheduler DolphinScheduler Service Configuration Choose to use MySQL as the metadata database for the DolphinScheduler service. Service Selection for Installation Select two DS-master nodes, at least two DS-ALERTS, DS-APIs, and deploy the remaining nodes as DS-worker nodes. Error Handling for hue-desktop-ini If you have configured namenode-ha before installing DolphinScheduler, you may encounter errors related to 'hue-desktop-ini'. You need to modify the params.py script. Add Service and Select DolphinScheduler DolphinScheduler Service Configuration Choose to use MySQL as the metadata database for the DolphinScheduler service. Service Selection for Installation Select two DS-master nodes, at least two DS-ALERTS, DS-APIs, and deploy the remaining nodes as DS-worker nodes. Error Handling for hue-desktop-ini If you have configured namenode-ha before installing DolphinScheduler, you may encounter errors related to 'hue-desktop-ini'. You need to modify the params.py script. params.py Script location: /var/lib/ambari-server/resources/stacks/HDP/3.3/services/DOLPHINSCHEDULER/package/scripts/params.py

Modify the content:
# hdfs_httpfs_host = config['configurations']['hue-desktop-ini']['hdfs_httpfs_host']
hdfs_httpfs_host = "dfs_ha_enabled"  # Comment out the original line and assign any value to hdfs_httpfs_host Script location: /var/lib/ambari-server/resources/stacks/HDP/3.3/services/DOLPHINSCHEDULER/package/scripts/params.py

Modify the content:
# hdfs_httpfs_host = config['configurations']['hue-desktop-ini']['hdfs_httpfs_host']
hdfs_httpfs_host = "dfs_ha_enabled"  # Comment out the original line and assign any value to hdfs_httpfs_host After making the changes, save and restart the ambari-server service to reinstall DolphinScheduler. Copy MySQL Driver to All Nodes' libs Directory Copy MySQL Driver to All Nodes' libs Directory cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/alert-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/api-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/master-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/worker-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/3.3.1.0-004/dolphinscheduler/tools/libs/ cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/alert-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/api-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/master-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/current/dolphinscheduler/worker-server/libs/
cp /tmp/mysql-connector-j-8.0.33.jar /usr/hdp/3.3.1.0-004/dolphinscheduler/tools/libs/ Change Ownership of All Services to hdfs User Change Ownership of All Services to hdfs User chown -R hdfs:hdfs /var/log/dolphinscheduler
chown -R hdfs:hdfs /usr/hdp/3.3.1.0-004/dolphinscheduler
chown -R hdfs:hdfs /etc/dolphinscheduler/
ln -s /usr/hdp/3.1.4.0-315/dolphinscheduler dolphinscheduler chown -R hdfs:hdfs /var/log/dolphinscheduler
chown -R hdfs:hdfs /usr/hdp/3.3.1.0-004/dolphinscheduler
chown -R hdfs:hdfs /etc/dolphinscheduler/
ln -s /usr/hdp/3.1.4.0-315/dolphinscheduler dolphinscheduler After making these changes, start DolphinScheduler from the Ambari management platform. The metadata will be automatically initialized and tables will be created during the startup process.

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

Walkthroughs, tutorials, guides, and tips. This story will teach you how to do something new or how to do something better.

Take the Guesswork Out of Installing DolphinScheduler—Here’s How to Do It

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

A 10-Minute Deep Dive Into the Core Architecture of Apache SeaTunnel and DataX

8 Essential Maven Plugins Beyond The Core

A Comprehensive Guide to Apache Cassandra Architecture

A Detailed Guide To Using Apache Storm

ACID Transactions Are Coming To Apache Cassandra: Here's Why We're Excited

Apache Airflow And Its Contribution to Enterprise Data Integration

A 10-Minute Deep Dive Into the Core Architecture of Apache SeaTunnel and DataX

8 Essential Maven Plugins Beyond The Core

A Comprehensive Guide to Apache Cassandra Architecture

A Detailed Guide To Using Apache Storm

ACID Transactions Are Coming To Apache Cassandra: Here's Why We're Excited

Apache Airflow And Its Contribution to Enterprise Data Integration

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps