When it comes to tools for syncing MySQL to Oracle, besides the traditional OGG, SeaTunnel (ST) should definitely be on the list—it’s simply too convenient to use ~
Our production system has been running steadily for 4 months now, with 3 source databases, perfectly supported! Highly recommend everyone to give it a try~
1. How to use SeaTunnel? Let’s check the help info
./bin/seatunnel.sh -h
Usage: seatunnel.sh [options]
Options:
--async Run the job asynchronously, when the job
is submitted, the client will exit
(default: false)
-can, --cancel-job Cancel job by JobId
--check Whether check config (default: false)
-cj, --close-job Close client the task will also be closed
(default: true)
-cn, --cluster The name of cluster
-c, --config Config file
--decrypt Decrypt config file, When both --decrypt
and --encrypt are specified, only
--encrypt will take effect (default: false)
-m, --master, -e, --deploy-mode SeaTunnel job submit master, support
[local, cluster] (default: cluster)
--encrypt Encrypt config file, when both --decrypt
and --encrypt are specified, only
--encrypt will take effect (default: false)
--get_running_job_metrics Gets metrics for running jobs (default: false)
-h, --help Show the usage message
-j, --job-id Get job status by JobId
-l, --list list job status (default: false)
--metrics Get job metrics by JobId
-n, --name SeaTunnel job name (default: SeaTunnel)
-r, --restore restore with savepoint by jobId
-s, --savepoint savepoint job by jobId
--set-job-id Set custom job id for job
-i, --variable Variable substitution, such as -i
city=beijing, or -i date=20190318. We use
',' as separator, when inside "", ',' are
treated as normal characters instead of
delimiters. For example, -i
city="beijing,shanghai". If you want to
use dynamic parameters, you can use the
following format: -i date=$(date +%Y%m%d). (default: [])
Reference explanation:
- Parameters related to job submission
- Parameters related to job management
2. How to install?
https://seatunnel.apache.org/download
3. Daily Ops: Job Management
3.1 View jobs: running means it’s in progress, of course, you’ll also see other statuses
./bin/seatunnel.sh -l
3.2 Pause a job
./bin/seatunnel.sh -s 967714059992432641
3.3 Restart a paused job
./bin/seatunnel.sh -r 967714059992432641 -c $SEATUNNEL_HOME/config/mysql_virdb_config
Since no background run was added during restore, it stayed in the foreground. After exiting with ctrl+c, the status became:
PS: When restoring, be sure to add the relevant parameters, job name, whether to run in the background, etc.
./bin/seatunnel.sh -r 967714059992432641 -c $SEATUNNEL_HOME/config/mysql_virdb_config --async -n job_mysql_virdb
Check the job status again—this is the result we want!
3.4 Cancel a job
This command cancels the specified job. Once canceled, the job will stop, and its status will become CANCELED.
Batch canceling is supported, allowing you to cancel multiple jobs at once.
All checkpoint info for canceled jobs will be deleted and cannot be restored with seatunnel.sh -r.
./bin/seatunnel.sh -can 967714059992432641
PS: The status after canceling looks very similar to directly exiting with Ctrl-C. Restarting may result in partial data loss.
4. SeaTunnel Logging Configuration
Config file: $SEATUNNEL_HOME/config/log4j2.properties
4.1 Configure a separate log file for each job (takes effect after restarting a job)
Current config:
rootLogger.appenderRef.file.ref = fileAppender
appender.file.layout.pattern = [%X{ST-JID}] %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n
Change to:
rootLogger.appenderRef.file.ref = routingAppender
appender.file.layout.pattern = %d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%-30.30c{1.}] [%t] - %m%n
4.2 SeaTunnel supports the scheduled deletion of old log files to avoid a disk space shortage
In $SEATUNNEL_HOME/config/seatunnel.yaml, add the following config:
Default values below (time is in minutes, e.g., 1440 = 1440 minutes):
seatunnel:
engine:
history-job-expire-minutes:1440
telemetry:
logs:
scheduled-deletion-enable:true
Explanation:
- history-job-expire-minutes: Sets retention time (in minutes) for historical jobs and logs. After this time, expired job info and logs are automatically deleted.
- scheduled-deletion-enable: Enables scheduled cleanup (default true). Logs are deleted once they reach the expiration time. If disabled, logs will remain forever and must be managed manually—otherwise, disk usage may be impacted. It’s recommended to configure this reasonably as needed.
5. Web UI for Job Monitoring
PS: Finally, it’s here—a Web UI for job monitoring! Awesome! Let’s see how to configure and access it.
5.1 Configure Web UI
Config file: $SEATUNNEL_HOME/config/seatunnel.yaml, default config:
seatunnel:
engine:
http:
enable-http:true
port:8080
5.2 Access Web UI
Open a browser, go to http://ip:8080
5.3 Data sync validation
Compare using latest create/update time and row counts.
6. Common Errors
6.1 Must configure database and table names; otherwise, an error occurs
... (log error omitted, showing database option required)
6.2 Error 2: ID issue
... (log error omitted, showing can't find field [ID])
6.3 If it already exists, an error will be prompted
Error Msg = ORA-00955: name is already used by an existing object
Web UI and server queries may show inconsistency
jobid: 967961857958608897 was paused before; after restart, the Web UI showed 2 records.
6.4 Could be due to data delay—after a few days, it disappeared
7. References
- Download: https://seatunnel.apache.org/download/
- Source MySQL-CDC config: https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/source/MySQL-CDC
- Sink Oracle config: https://seatunnel.apache.org/zh-CN/docs/2.3.10/connector-v2/sink/Oracle
- Configure individual log settings per job: https://seatunnel.apache.org/zh-CN/docs/2.3.10/seatunnel-engine/logging