When it comes to tools for syncing MySQL to Oracle, besides the traditional OGG, SeaTunnel (ST) should definitely be on the list—it’s simply too convenient to use ~ Our production system has been running steadily for 4 months now, with 3 source databases, perfectly supported! Highly recommend everyone to give it a try~ 1. How to use SeaTunnel? Let’s check the help info ./bin/seatunnel.sh -h
Usage: seatunnel.sh [options]
  Options:
    --async                         Run the job asynchronously, when the job 
                                    is submitted, the client will exit
                                    (default: false)
    -can, --cancel-job              Cancel job by JobId
    --check                         Whether check config (default: false)
    -cj, --close-job                Close client the task will also be closed 
                                    (default: true)
    -cn, --cluster                  The name of cluster
    -c, --config                    Config file
    --decrypt                       Decrypt config file, When both --decrypt 
                                    and --encrypt are specified, only 
                                    --encrypt will take effect (default: false) 
    -m, --master, -e, --deploy-mode SeaTunnel job submit master, support 
                                    [local, cluster] (default: cluster)
    --encrypt                       Encrypt config file, when both --decrypt 
                                    and --encrypt are specified, only 
                                    --encrypt will take effect (default: false) 
    --get_running_job_metrics       Gets metrics for running jobs (default: false) 
    -h, --help                      Show the usage message
    -j, --job-id                    Get job status by JobId
    -l, --list                      list job status (default: false)
    --metrics                       Get job metrics by JobId
    -n, --name                      SeaTunnel job name (default: SeaTunnel)
    -r, --restore                   restore with savepoint by jobId
    -s, --savepoint                 savepoint job by jobId
    --set-job-id                    Set custom job id for job
    -i, --variable                  Variable substitution, such as -i 
                                    city=beijing, or -i date=20190318. We use 
',' as separator, when inside "", ',' are 
                                    treated as normal characters instead of 
                                    delimiters. For example, -i 
                                    city="beijing,shanghai". If you want to 
                                    use dynamic parameters, you can use the 
                                    following format: -i date=$(date +%Y%m%d). (default: []) ./bin/seatunnel.sh -h
Usage: seatunnel.sh [options]
  Options:
    --async                         Run the job asynchronously, when the job 
                                    is submitted, the client will exit
                                    (default: false)
    -can, --cancel-job              Cancel job by JobId
    --check                         Whether check config (default: false)
    -cj, --close-job                Close client the task will also be closed 
                                    (default: true)
    -cn, --cluster                  The name of cluster
    -c, --config                    Config file
    --decrypt                       Decrypt config file, When both --decrypt 
                                    and --encrypt are specified, only 
                                    --encrypt will take effect (default: false) 
    -m, --master, -e, --deploy-mode SeaTunnel job submit master, support 
                                    [local, cluster] (default: cluster)
    --encrypt                       Encrypt config file, when both --decrypt 
                                    and --encrypt are specified, only 
                                    --encrypt will take effect (default: false) 
    --get_running_job_metrics       Gets metrics for running jobs (default: false) 
    -h, --help                      Show the usage message
    -j, --job-id                    Get job status by JobId
    -l, --list                      list job status (default: false)
    --metrics                       Get job metrics by JobId
    -n, --name                      SeaTunnel job name (default: SeaTunnel)
    -r, --restore                   restore with savepoint by jobId
    -s, --savepoint                 savepoint job by jobId
    --set-job-id                    Set custom job id for job
    -i, --variable                  Variable substitution, such as -i 
                                    city=beijing, or -i date=20190318. We use 
',' as separator, when inside "", ',' are 
                                    treated as normal characters instead of 
                                    delimiters. For example, -i 
                                    city="beijing,shanghai". If you want to 
                                    use dynamic parameters, you can use the 
                                    following format: -i date=$(date +%Y%m%d). (default: []) Reference explanation: Reference explanation: Parameters related to job submission Parameters related to job submission Parameters related to job management Parameters related to job management 2. How to install? https://seatunnel.apache.org/download https://seatunnel.apache.org/download 3. Daily Ops: Job Management 3.1 View jobs: running means it’s in progress, of course, you’ll also see other statuses ./bin/seatunnel.sh -l ./bin/seatunnel.sh -l 3.2 Pause a job ./bin/seatunnel.sh -s 967714059992432641 ./bin/seatunnel.sh -s 967714059992432641 3.3 Restart a paused job ./bin/seatunnel.sh -r 967714059992432641 -c $SEATUNNEL_HOME/config/mysql_virdb_config ./bin/seatunnel.sh -r 967714059992432641 -c $SEATUNNEL_HOME/config/mysql_virdb_config Since no background run was added during restore, it stayed in the foreground. After exiting with ctrl+c, the status became: PS: When restoring, be sure to add the relevant parameters, job name, whether to run in the background, etc. ./bin/seatunnel.sh -r 967714059992432641 -c $SEATUNNEL_HOME/config/mysql_virdb_config --async -n job_mysql_virdb ./bin/seatunnel.sh -r 967714059992432641 -c $SEATUNNEL_HOME/config/mysql_virdb_config --async -n job_mysql_virdb Check the job status again—this is the result we want! Check the job status again—this is the result we want! 3.4 Cancel a job This command cancels the specified job. Once canceled, the job will stop, and its status will become CANCELED. Batch canceling is supported, allowing you to cancel multiple jobs at once. All checkpoint info for canceled jobs will be deleted and cannot be restored with seatunnel.sh -r. seatunnel.sh -r ./bin/seatunnel.sh -can 967714059992432641 ./bin/seatunnel.sh -can 967714059992432641 PS: The status after canceling looks very similar to directly exiting with Ctrl-C. Restarting may result in partial data loss. 4. SeaTunnel Logging Configuration Config file: $SEATUNNEL_HOME/config/log4j2.properties $SEATUNNEL_HOME/config/log4j2.properties 4.1 Configure a separate log file for each job (takes effect after restarting a job) Current config: rootLogger.appenderRef.file.ref = fileAppender
appender.file.layout.pattern = [%X{ST-JID}] %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n rootLogger.appenderRef.file.ref = fileAppender
appender.file.layout.pattern = [%X{ST-JID}] %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n Change to: rootLogger.appenderRef.file.ref = routingAppender
appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n rootLogger.appenderRef.file.ref = routingAppender
appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n 4.2 SeaTunnel supports the scheduled deletion of old log files to avoid a disk space shortage In $SEATUNNEL_HOME/config/seatunnel.yaml, add the following config: $SEATUNNEL_HOME/config/seatunnel.yaml Default values below (time is in minutes, e.g., 1440 = 1440 minutes): seatunnel:
engine:
history-job-expire-minutes:1440
telemetry:
logs:
scheduled-deletion-enable:true seatunnel:
engine:
history-job-expire-minutes:1440
telemetry:
logs:
scheduled-deletion-enable:true Explanation: Explanation: Explanation: history-job-expire-minutes: Sets retention time (in minutes) for historical jobs and logs. After this time, expired job info and logs are automatically deleted.scheduled-deletion-enable: Enables scheduled cleanup (default true). Logs are deleted once they reach the expiration time. If disabled, logs will remain forever and must be managed manually—otherwise, disk usage may be impacted. It’s recommended to configure this reasonably as needed. history-job-expire-minutes: Sets retention time (in minutes) for historical jobs and logs. After this time, expired job info and logs are automatically deleted. scheduled-deletion-enable: Enables scheduled cleanup (default true). Logs are deleted once they reach the expiration time. If disabled, logs will remain forever and must be managed manually—otherwise, disk usage may be impacted. It’s recommended to configure this reasonably as needed. 5. Web UI for Job Monitoring PS: Finally, it’s here—a Web UI for job monitoring! Awesome! Let’s see how to configure and access it. 5.1 Configure Web UI Config file: $SEATUNNEL_HOME/config/seatunnel.yaml, default config: $SEATUNNEL_HOME/config/seatunnel.yaml seatunnel:
engine:
http:
enable-http:true
port:8080 seatunnel:
engine:
http:
enable-http:true
port:8080 5.2 Access Web UI Open a browser, go to http://ip:8080 http://ip:8080 5.3 Data sync validation Compare using latest create/update time and row counts. Compare using latest create/update time and row counts. 6. Common Errors 6.1 Must configure database and table names; otherwise, an error occurs ... (log error omitted, showing database option required) ... (log error omitted, showing database option required) 6.2 Error 2: ID issue ... (log error omitted, showing can't find field [ID]) ... (log error omitted, showing can't find field [ID]) 6.3 If it already exists, an error will be prompted Error Msg = ORA-00955: name is already used by an existing object Error Msg = ORA-00955: name is already used by an existing object Web UI and server queries may show inconsistency Web UI and server queries may show inconsistency jobid: 967961857958608897 was paused before; after restart, the Web UI showed 2 records. 6.4 Could be due to data delay—after a few days, it disappeared 7. References Download: https://seatunnel.apache.org/download/Source MySQL-CDC config: https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/source/MySQL-CDCSink Oracle config: https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/sink/OracleConfigure individual log settings per job: https://seatunnel.apache.org/zh-CN/docs/2.3.10/seatunnel-engine/logging Download: https://seatunnel.apache.org/download/ https://seatunnel.apache.org/download/ Source MySQL-CDC config: https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/source/MySQL-CDC https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/source/MySQL-CDC Sink Oracle config: https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/sink/Oracle https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/sink/Oracle Configure individual log settings per job: https://seatunnel.apache.org/zh-CN/docs/2.3.10/seatunnel-engine/logging https://seatunnel.apache.org/zh-CN/docs/2.3.10/seatunnel-engine/logging

The code in this story is for educational purposes. The readers are solely responsible for whatever they build with it.

(I) Principles of Data Model Architecture: Four Layers and Seven Stages 

Read My Stories

Too Long; Didn't Read

Make resilience your competitive advantage

Step-by-Step Guide: Free Real-Time MySQL to Oracle Sync with SeaTunnel

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

(I) Principles of Data Model Architecture: Four Layers and Seven Stages

A 10-Minute Deep Dive Into the Core Architecture of Apache SeaTunnel and DataX

Source Code Analysis of Apache SeaTunnel Zeta Engine (Part 2): Task Submission Process on the Client Side

Source Code Analysis of Apache SeaTunnel Zeta Engine (Part 3): Server-Side Task Submission

Sync MySQL Data to S3 in Just 3 Steps Using Apache SeaTunnel

(I) Principles of Data Model Architecture: Four Layers and Seven Stages

A 10-Minute Deep Dive Into the Core Architecture of Apache SeaTunnel and DataX

Source Code Analysis of Apache SeaTunnel Zeta Engine (Part 2): Task Submission Process on the Client Side

Source Code Analysis of Apache SeaTunnel Zeta Engine (Part 3): Server-Side Task Submission

Sync MySQL Data to S3 in Just 3 Steps Using Apache SeaTunnel

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps