Oracle

任务类型

Oracle实时采集支持两种类型：通过LogMiner实时采集、通过间隔轮询进行采集

数据源

选择对应数据类型的已有数据源

Schema

选择数据源下指定Schema

是否分表

勾选，可以设置多对多分表写入；

表

选择数据源表，支持多表合并写入一张表；
多表合并写入时，无法保证完全根据业务顺序写入；

采集起点

用户可选择从任务运行时开始、按文件选择两种采集起点
从任务运行时开始：任务提交处于"运行中"时，开始实时采集；
按文件选择：根据选择用户的Binlog、LogMiner作为起始文件进行采集；

数据操作

支持Insert、Update、Delete三种数据操作类型选择，当发生insert、update等操作时，实时采集组件可根据LogMiner等信息立即获取其操作，并实时将变更信息同步至目标数据源

格式转换

勾选后，系统将会多层嵌套格式的JSON分解为单层结构，例如：{"a":1, "b": {"c":3}}，将会被分解为：{"a":1,"b_c":3}

高级配置

以JSON格式添加高级参数，例如对关系型数据库可配置fetchSize，每类数据源支持不同的参数

任务类型

Oracle实时采集支持两种类型：通过LogMiner实时采集、通过间隔轮询进行采集

数据源

选择对应数据类型的已有数据源

PDB

Schema

选择数据源下指定Schema

表

选择数据源表，仅支持单表间隔轮询

增量标识字段

用户需选择增量标示字段，每次同步时，系统自动记录增量标识的最大值，下次运行时，会从上一次的最大值继续同步数据，实现增量同步；支持将数值类型、Timestamp类型作为增量标识字段

采集字段

用户可指定要进行采集的目标字段，支持多选

采集起点

用户根据选择的增量标示字段设定相应的采集起点，若不填则默认从头开始拉取数据，输入格式请在"数据预览"中参考所选增量标示字段内容。采集时不包含采集起点，例如采集起点为40 则采集开始时不会包含id=40这一条数据。

轮询时间间隔

手动设定轮询时间间隔，单位为秒

高级配置

以JSON格式添加高级参数，例如对关系型数据库可配置fetchSize，每类数据源支持不同的参数，具体可参考

问题代码

原因

处理方法

ORA-00308 / ORA-27037

This simply means that if your Oracle version is 11.1, Oracle will scan or dump the redo records within 12 hours. If it is found that the required redo records have been deleted in the scan or dump, ora-00308 and ora-27037 errors will be reported.

you can restart the logMiner task or The recommended solution for Oracle is to patch 8825048 and upgrade to 11.1.0.7.3 (patch set update) or 11.2.0.1 (base release)

ORA-00310

The archived log was out of sequence, probably because it was corrupt or the wrong redo log file name was specified during recovery

it may be useful to increase the number and size of redo log groups, and the restart the logMiner task

ORA-00604

too many Oracle archive logs queried

increase the scn number of the query to reduce the number of archive logs queried OR set [readPosition] to [current]

ORA-01013

sql is timeout when query data from logMiner

the default value of logMiner parameter: [queryTimeout] is 300S, it may be useful to adjust [queryTimeout], for example: [\"queryTimeout\": 360]

ORA-01284

The file or directory may not exist or may be inaccessible or pathname exceeds 256 characters

Please ensure that the file and the directory exist and are accessible

ORA-01289

This simply means that the logfile specified has already been added to the list of logfiles and it may be we find same logfile

you can restart the logMiner task

Connection reset by peer

This simply means that something in the backend ( DBMS ) decided to stop working due to unavailability of resources etc

This is not a problem with the FlinkX program, so there is nothing to do but restarting the logMiner task. For more detail, look at this: https://stackoverflow.com/questions/6110395/sqlrecoverableexception-i-o-exception-connection-reset