本示例主要介绍通过实现OGG的接口函数，实现自定义处理增量数据，将数据实时写入到mariadb （OGG官方不支持此数据库，所以只能采用自定义方式实现）。以下是本次示例的4个类：

ConnectionFactory

package sample.handler.jdbc;

import java.sql.Connection;

public interface ConnectionFactory {

public Connection getConnection() throws Exception;

}

DriverClassConnectionFactory

package sample.handler.jdbc;

import java.sql.Connection;

import java.sql.Driver;

import java.sql.SQLException;

import java.util.Properties;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

public class DriverClassConnectionFactory implements ConnectionFactory {

private static final Logger logger = LoggerFactory.getLogger(DriverClassConnectionFactory.class);

protected String driver;

protected String url;

protected Properties arguments;

@Override

public Connection getConnection() throws InstantiationException, IllegalAccessException, ClassNotFoundException, SQLException {

logger.info(new StringBuilder("Connect to [").append(url).append("] via [").append(driver).append("] with arguments: ").append(arguments).toString());

Driver driv = (Driver)Class.forName(driver).newInstance();

return driv.connect(url, arguments);

}

/**

* @return the driver

*/

public String getDriver() {

return driver;

}

/**

* @param driver the driver to set

*/

public void setDriver(String driver) {

this.driver = driver;

}

/**

* @return the url

*/

public String getUrl() {

return url;

}

/**

* @param url the url to set

*/

public void setUrl(String url) {

this.url = url;

}

/**

* @return the arguments

*/

public Properties getArguments() {

return arguments;

}

/**

* @param arguments the arguments to set

*/

public void setArguments(Properties arguments) {

this.arguments = arguments;

}

/**

* @param user the user to set

*/

public void setUser(String user) {

if (arguments==null) {

arguments = new Properties();

}

arguments.setProperty("user", user);

}

/**

* @param password the password to set

*/

public void setPassword(String password) {

if (arguments==null) {

arguments = new Properties();

}

arguments.setProperty("password", password);

}

}

OGG接口实现类SimpleJDBCHandler

package sample.handler.jdbc;

import java.math.BigDecimal;

import java.sql.Connection;

import java.sql.Driver;

import java.sql.PreparedStatement;

import java.sql.SQLException;

import java.sql.Timestamp;

import java.sql.Types;

import java.text.ParseException;

import java.text.SimpleDateFormat;

import java.util.Arrays;

import java.util.Properties;

import org.slf4j.Logger;

import org.slf4j.LoggerFactory;

import oracle.goldengate.datasource.*;

import oracle.goldengate.datasource.GGDataSource.Status;

import oracle.goldengate.datasource.meta.*;

public class SimpleJDBCHandler extends AbstractHandler {

private static final Logger logger = LoggerFactory.getLogger(SimpleJDBCHandler.class);

protected ConnectionFactory connFactory;

protected Connection conn;

protected String driver;

protected String url;

protected Properties arguments;

protected PreparedStatement pstmt;

protected String lastOp;

protected int keyIndex;

protected int batchSize = 0;

protected int maxBatchSize = 10000;

protected long reportCount = 0;

protected long lastReportTime;

protected long opCount;

protected SimpleDateFormat dateFormat;

/**

* @param connectionFactory the Class name of connection factory to set

* @throws ClassNotFoundException

* @throws IllegalAccessException

* @throws InstantiationException

*/

public void setConnectionFactory(String connectionFactory) throws InstantiationException, IllegalAccessException, ClassNotFoundException {

connFactory = (ConnectionFactory)Class.forName(connectionFactory).newInstance();

}

// 获取目标端的DB连接

public Connection getConnection() throws Exception {

if (conn==null) {

if (connFactory==null) {

logger.info(new StringBuilder("Connect to [").append(url).append("] via [").append(driver).append("] with arguments: ").append(arguments).toString());

Driver driv = (Driver)Class.forName(driver).newInstance();

conn = driv.connect(url, arguments);

} else {

conn = connFactory.getConnection();

}

conn.setAutoCommit(false);

}

return conn;

}

/* (non-Javadoc)

* 一个事务的开始

*/

@Override

public Status transactionBegin(DsEvent e, DsTransaction tx) {

if (logger.isDebugEnabled())

logger.debug("Method transactionBegin invoked. TxID="+tx.getTranID());

//batchSize = 0;

return super.transactionBegin(e, tx);

}

/* (non-Javadoc)

*一个新的记录操作

*/

@Override

public Status operationAdded(DsEvent e, DsTransaction tx, DsOperation op) {

if (logger.isDebugEnabled())

logger.debug(new StringBuilder("Method operationAdded invoked. TxID=").append(tx.getTranID()).append(" OpNum=").append(tx.getTotalOps()).toString());

Status status = GGDataSource.Status.OK;

if (op.getOperationType().isInsert()) {

status = insertAdded(e, tx, op);

} else if (op.getOperationType().isUpdate()) {

status = updateAdded(e, tx, op);

} else if (op.getOperationType().isDelete()) {

status = deleteAdded(e, tx, op);

}

if (reportCount>=100) {

if (++opCount == reportCount) {

long duration = System.currentTimeMillis() - lastReportTime;

logger.info(new StringBuffer().append(opCount).append(" operations have been processed in last ").append(duration/1000).append(" seconds. Rate=").append(opCount*1000/duration).toString());

opCount = 0;

lastReportTime = System.currentTimeMillis();

}

}

return status;

}

// insert操作

protected Status insertAdded(DsEvent e, DsTransaction tx, DsOperation op) {

try {

String currentOp = op.getTableName()+".INSERT";

if (!currentOp.equals(lastOp)) {

if (batchSize>0 && pstmt!=null) {

executeBatch();

}

String sql = prepareInsertSql(e, tx, op);

System.out.println("insert-sql:"+ sql);

if (logger.isDebugEnabled())

logger.debug("Prepare insert. SQL=["+sql+"]");

pstmt = getConnection().prepareStatement(sql);

lastOp = currentOp;

}

TableMetaData tmeta = e.getMetaData().getTableMetaData(op.getTableName());

for (int i=0; iColumnMetaData cmeta = tmeta.getColumnMetaData(i);

DsColumn col = op.getColumn(i);

bindValue(pstmt, i+1, cmeta, col.getAfter());

}

pstmt.addBatch();

if (++batchSize == maxBatchSize) {

executeBatch();

}

return GGDataSource.Status.OK;

} catch (Exception ex) {

logger.error("Method insertAdded failed.", ex);

return GGDataSource.Status.ABEND;

}

}

// update操作

protected Status updateAdded(DsEvent e, DsTransaction tx, DsOperation op) {

try {

String sql = prepareUpdateSql(e, tx, op);

if (!sql.equals(lastOp)) {

if (batchSize>0 && pstmt!=null) {

executeBatch();

}

if (logger.isDebugEnabled())

logger.debug("Prepare update. SQL=["+sql+"]");

pstmt = getConnection().prepareStatement(sql);

lastOp = sql;

}

System.out.println("update-sql:"+ sql);

TableMetaData tmeta = e.getMetaData().getTableMetaData(op.getTableName());

for (int i=0, j=0, k=keyIndex; iColumnMetaData cmeta = tmeta.getColumnMetaData(i);

DsColumn col = op.getColumn(i);

if (!col.isMissing()) {

bindValue(pstmt, ++j, cmeta, col.getAfter());

}

if (cmeta.isKeyCol()) {

if (col.hasBeforeValue()) {

if (logger.isDebugEnabled())

logger.debug(new StringBuilder(op.getTableName().toString()).append(": Key ").append(cmeta.getColumnName()).append(" is updated. before=[").append(col.getBefore().getValue()).append("], after=[").append(col.getAfter().getValue()).append("]").toString());

bindValue(pstmt, ++k, cmeta, col.getBefore());

} else {

bindValue(pstmt, ++k, cmeta, col.getAfter());

}

}

}

pstmt.addBatch();

if (++batchSize == maxBatchSize) {

executeBatch();

}

return GGDataSource.Status.OK;

} catch (Exception ex) {

logger.error("Method updateAdded failed.", ex);

return GGDataSource.Status.ABEND;

}

}

// delete操作

protected Status deleteAdded(DsEvent e, DsTransaction tx, DsOperation op) {

try {

String currentOp = op.getTableName()+".DELETE";

if (!currentOp.equals(lastOp)) {

if (batchSize>0 && pstmt!=null) {

executeBatch();

}

String sql = prepareDeleteSql(e, tx, op);

System.out.println("delete-sql:"+ sql);

if (logger.isDebugEnabled())

logger.debug("Prepare delete. SQL=["+sql+"]");

pstmt = getConnection().prepareStatement(sql);

lastOp = currentOp;

}

TableMetaData tmeta = e.getMetaData().getTableMetaData(op.getTableName());

for (int i=0, j=0; iColumnMetaData cmeta = tmeta.getColumnMetaData(i);

if (cmeta.isKeyCol()) {

DsColumn col = op.getColumn(i);

bindValue(pstmt, ++j, cmeta, col.getBefore());

}

}

pstmt.addBatch();

if (++batchSize == maxBatchSize) {

executeBatch();

}

return GGDataSource.Status.OK;

} catch (Exception ex) {

logger.error("Method deleteAdded failed.", ex);

return GGDataSource.Status.ABEND;

}

}

// 多个操作批量执行，提升效率

protected int[] executeBatch() throws SQLException {

try {

int[] result = pstmt.executeBatch();

if (logger.isDebugEnabled())

logger.debug(new StringBuilder("Batch executed. Op=[").append(lastOp).append("] size=").append(batchSize).append(" result=").append(Arrays.toString(result)).toString());

batchSize = 0;

return result;

}

catch(Exception ex) {

System.out.println(ex.getMessage());

ex.printStackTrace();

return null;

}

}

protected String prepareInsertSql(DsEvent e, DsTransaction tx, DsOperation op) {

String tbName = op.getTableName().toString().toLowerCase().replace(" scetc.", " public.");

StringBuilder sql = new StringBuilder("insert into ").append(tbName).append("(");

StringBuilder valueClause = new StringBuilder(") values (");

for (ColumnMetaData cmeta : e.getMetaData().getTableMetaData(op.getTableName()).getColumnMetaData()) {

sql.append(cmeta.getColumnName()).append(",");

valueClause.append("?,");

}

sql.deleteCharAt(sql.length()-1).append(valueClause.deleteCharAt(valueClause.length()-1)).append(")");

return sql.toString();

}

protected String prepareUpdateSql(DsEvent e, DsTransaction tx, DsOperation op) {

String tbName = op.getTableName().toString().toLowerCase().replace(" scetc.", " public.");

StringBuilder sql = new StringBuilder("update ").append(tbName).append(" set ");

StringBuilder whereClause = new StringBuilder(" where 0=0");

TableMetaData tmeta = e.getMetaData().getTableMetaData(op.getTableName());

keyIndex = 0;

for (int i=0; iColumnMetaData cmeta = tmeta.getColumnMetaData(i);

DsColumn col = op.getColumn(i);

if (!col.isMissing()) {

sql.append(cmeta.getColumnName()).append("=?,");

keyIndex++;

}

if (cmeta.isKeyCol()) {

whereClause.append(" and ").append(cmeta.getColumnName()).append("=?");

}

}

sql.deleteCharAt(sql.length()-1).append(whereClause);

return sql.toString();

}

protected String prepareDeleteSql(DsEvent e, DsTransaction tx, DsOperation op) {

String tbName = op.getTableName().toString().toLowerCase().replace(" scetc.", " public.");

StringBuilder sql = new StringBuilder("delete from ").append(tbName).append(" where 0=0");

for (ColumnMetaData cmeta : e.getMetaData().getTableMetaData(op.getTableName()).getKeyColumns()) {

sql.append(" and ").append(cmeta.getColumnName()).append("=?");

}

return sql.toString();

}

protected void bindValue(PreparedStatement pstmt, int i,

ColumnMetaData cmeta, DsColumn col) throws SQLException, ParseException {

int jdbcType = cmeta.getDataType().getJDBCType();

if (logger.isDebugEnabled())

logger.debug(new StringBuilder("Bind value for column [").append(cmeta.getColumnName()).append("] Type=").append(cmeta.getDataType()).append(" Value=[").append(col.getValue()).append("]").toString());

if (col.isValueNull()) {

pstmt.setNull(i, jdbcType);

} else if (jdbcType==Types.CHAR || jdbcType==Types.VARCHAR) {

pstmt.setString(i, col.getValue());

} else if (jdbcType==Types.NUMERIC || jdbcType==Types.DECIMAL) {

pstmt.setBigDecimal(i, new BigDecimal(col.getValue()));

} else if (jdbcType==Types.DATE || jdbcType==Types.TIMESTAMP) {

//pstmt.setTimestamp(i, new Timestamp(dateFormat.parse(col.getValue()).getTime()));

pstmt.setString(i, col.getValue());

} else {

pstmt.setObject(i, col.getValue());

}

}

/* (non-Javadoc)

* @see com.goldengate.atg.datasource.AbstractHandler#transactionCommit(com.goldengate.atg.datasource.DsEvent, com.goldengate.atg.datasource.DsTransaction)

*/

@Override

public Status transactionCommit(DsEvent e, DsTransaction tx) {

if (logger.isDebugEnabled())

logger.debug("Method transactionCommit invoked. TxID="+tx.getTranID());

if (conn!=null)

try {

if (batchSize>0 && pstmt!=null) {

executeBatch();

}

conn.commit();

} catch (SQLException ex) {

logger.error("Transaction commit failed. TxID="+tx.getTranID(), ex);

}

return super.transactionCommit(e, tx);

}

/* (non-Javadoc)

* @see com.goldengate.atg.datasource.AbstractHandler#transactionRollback(com.goldengate.atg.datasource.DsEvent, com.goldengate.atg.datasource.DsTransaction)

*/

@Override

public Status transactionRollback(DsEvent e, DsTransaction tx) {

if (logger.isDebugEnabled())

logger.debug("Method transactionRollback invoked. TxID="+tx.getTranID());

if (conn!=null)

try {

conn.rollback();

} catch (SQLException ex) {

logger.error("Transaction rollback failed. TxID="+tx.getTranID(), ex);

}

return super.transactionRollback(e, tx);

}

/* (non-Javadoc)

* @see com.goldengate.atg.datasource.AbstractHandler#reportStatus()

*/

@Override

public String reportStatus() {

logger.debug("Method reportStatus invoked.");

return "SimpleJDBCHandler running...";

}

/* (non-Javadoc)

* @see com.goldengate.atg.datasource.AbstractHandler#destroy()

*/

@Override

public void destroy() {

logger.debug("Handler destroy...");

if (conn!=null)

try {

if (!conn.isClosed()) conn.close();

} catch (SQLException ex) {

logger.error("Connection close failed.", ex);

}

super.destroy();

}

/**

* @return the driver

*/

public String getDriver() {

return driver;

}

/**

* @param driver the driver to set

*/

public void setDriver(String driver) {

this.driver = driver;

}

/**

* @return the url

*/

public String getUrl() {

return url;

}

/**

* @param url the url to set

*/

public void setUrl(String url) {

this.url = url;

}

/**

* @return the arguments

*/

public Properties getArguments() {

return arguments;

}

/**

* @param arguments the arguments to set

*/

public void setArguments(Properties arguments) {

this.arguments = arguments;

}

/**

* @param user the user to set

*/

public void setUser(String user) {

if (arguments==null) {

arguments = new Properties();

}

arguments.setProperty("user", user);

}

/**

* @param password the password to set

*/

public void setPassword(String password) {

if (arguments==null) {

arguments = new Properties();

}

arguments.setProperty("password", password);

}

public void setDateFormat(String pattern) {

dateFormat = new SimpleDateFormat(pattern);

}

/**

* @return the maxBatchSize

*/

public int getMaxBatchSize() {

return maxBatchSize;

}

/**

* @param maxBatchSize the maxBatchSize to set

*/

public void setMaxBatchSize(int maxBatchSize) {

this.maxBatchSize = maxBatchSize;

}

/**

* @return the reportCount

*/

public long getReportCount() {

return reportCount;

}

/**

* @param reportCount the reportCount to set

*/

public void setReportCount(long reportCount) {

if (reportCountlogger.info("A reportCount less than 100 will turn off report.");

this.reportCount = 0;

} else if (this.reportCount != reportCount) {

this.reportCount = reportCount;

opCount = 0;

lastReportTime = System.currentTimeMillis();

}

}

}

从以上代码可以看出，可以通过重载不同的接口函数，获得事务、记录的相关信息，包括操作的表、时间、字段、操作类型及变化的数据等。

最后是基于此Handler创建一个测试类。

TestMariadb

package sample.handler.jdbc;

public class TestMariadb {

public static void main(String[] args) {

SimpleJDBCHandler mariadb = new SimpleJDBCHandler();

gbase.setUrl("jdbc:mariadb://192.168.26.44:5258/mydb? ");

gbase.setDateFormat("com.mariadb.jdbc.Driver");

gbase.setUser("root");

gbase.setPassword("admin123");

}

}

通过本示例提供的代码，可以实现将增量数据，从OGG的源端，以完全自定义的方式写入到任何需要的目标端。

GoldenGate for Java adapter介绍二（代码篇）的更多相关文章

  1. GoldenGate for Java Adapter介绍一(原理篇)

    前言 Oracle Goldengate在很早前就推出了一个for java的版本,主要目的是方便把关系型数据实时写入到不支持的目标端,如JMS或Redis等key value数据库.在Hadoop刚 ...

  2. [Android&Java]浅谈设计模式-代码篇：观察者模式Observer

    观察者,就如同一个人,对非常多东西都感兴趣,就好像音乐.电子产品.Game.股票等,这些东西的变化都能引起爱好者们的注意并时刻关注他们.在代码中.我们也有这种一种方式来设计一些好玩的思想来.今天就写个 ...

  3. 《手把手教你》系列基础篇（九十七）-java+ selenium自动化测试-框架设计篇-Selenium方法的二次封装和页面基类（详解教程）

    1.简介 上一篇宏哥介绍了如何设计支持不同浏览器测试,宏哥的方法就是通过来切换配置文件设置的浏览器名称的值,来确定启动什么浏览器进行脚本测试.宏哥将这个叫做浏览器引擎类.这个类负责获取浏览器类型和启动 ...

  4. JVM系列（二） — Java垃圾收集介绍

    这篇文章主要从以下几个方面介绍垃圾收集的相关知识 一.判断对象是否已死 二.主流垃圾收集算法 三.内存分配与回收策略 本章节主要从以下几个思考点着手介绍垃圾回收的相关知识:哪些内存需要回收?什么时候回 ...

  5. Python介绍RabbitMQ使用篇二WorkQueue

    1. RabbitMQ WorkQueue基本工作模式介绍 上一篇我们使用C#语言讲解了单个消费者从消息队列中处理消息的模型,这一篇我们使用Python语言来讲解多个消费者同时工作从一个Queue处理 ...

  6. Java面试知识点之线程篇（二）

    前言:接上篇,这里继续对java线程相关知识点进行总结. 1.notify和notifyall的区别 notify()方法能够唤醒一个正在等待该对象的monitor的线程,当有多个线程都在等待该对象的 ...

  7. 采样方法（二）MCMC相关算法介绍及代码实现

    采样方法(二)MCMC相关算法介绍及代码实现 2017-12-30 15:32:14 Dark_Scope 阅读数 10509更多 分类专栏: 机器学习   版权声明:本文为博主原创文章,遵循CC 4 ...

  8. Lucene.Net 2.3.1开发介绍 —— 二、分词（六）

    原文:Lucene.Net 2.3.1开发介绍 -- 二.分词(六) Lucene.Net的上一个版本是2.1,而在2.3.1版本中才引入了Next(Token)方法重载,而ReusableStrin ...

  9. JAVA代码覆盖率工具JaCoCo-原理篇

    JAVA代码覆盖率工具JaCoCo-原理篇 1.2 JAVA覆盖率工具介绍 1.3.3 Apache Maven方式 1.3.4 Eclipse EclDmma Plugin方式 JAVA代码覆盖率工 ...

随机推荐

  1. js库 - 浅拷贝 & 深拷贝

    学了堆栈内存空间,应该就理解了什么叫简单数据类型存在栈内存,复杂数据类型存在堆内存了. 然后面试中经常会问.业务中也经常会遇到的问题就是深浅拷贝的问题了. 栈内存中简单数据类型直接拷贝就能得到一个副本 ...

  2. Sublime Text 3 3143 注册码和遇到的PyV8无法下载问题

    packagecontrol官网:https://packagecontrol.io/installation sublime安装之后,安装Package Control管理插件. sublime3 ...

  3. 使用PsExec获取shell执行命令

    PsExec PsExec是pstools工具组套件的一部分,确成为了渗透利器,下载地址:点击这里下载 连接shell 我的Windows Server 2012默认打开域网络防火墙的时候,是不能连接 ...

  4. K - Heavy Cargo dijkstar

    来源poj2263 Big Johnsson Trucks Inc. is a company specialized in manufacturing big trucks. Their lates ...

  5. python sys.modules模块

    sys.modules是一个全局字典,该字典是python启动后就加载在内存中.每当程序员导入新的模块,sys.modules都将记录这些模块.字典sys.modules对于加载模块起到了缓冲的作用. ...

  6. mysql批量update的两种方法

    today a question let me happy(抓狂) 头儿分了一个小任务,让修改循环调用dao层的那些不啦不啦不啦,鉴于之前写过批量更新的玩意,so 很快代码就修改完了,but 测的时候 ...

  7. 剑指offer——python【第49题】把字符串转换成整数

    题目描述 将一个字符串转换成一个整数(实现Integer.valueOf(string)的功能,但是string不符合数字要求时返回0),要求不能使用字符串转换整数的库函数. 数值为0或者字符串不是一 ...

  8. 转 python测试框架最全资源汇总

    转自: http://www.testclass.net/list/python_list_1/ xUnit frameworks(单元测试框架) frameworks 框架 unittest - p ...

  9. PM2.5环境检测系统的设计与分析

    PM2.5环境检测系统的设计与分析   摘要: 大气颗粒物污染对人类健康和生态环境造成了很大的影响,这让人们逐渐重视起对细颗粒物PM2.5检测技术的研究.本文阐述了PM2.5浓度检测的五种方法,在对上 ...

  10. [教程]微信官方开源UI库-WeUI使用方法【申明：来源于网络】

    [教程]微信官方开源UI库-WeUI使用方法 [ 教程]微信官方开源UI库-WeUI使用方法 地址:http://www.weui.org.cn/?/article/1 微信公众号开发-WeUI使用说 ...