Executive Overview

This document is an introduction to Oracle GoldenGate (DIPC remote agent)’s best practices and guidelines for configuring OGG in an active-active environment using the built in functionality of DML Auto Conflict Detection and Resolution (Auto CDR), LOB data type column is also take into consideration.

Prerequisites

·       RDBMS is Oracle Database 12.2+

·       GoldenGate version 12.3/DIPC agent 18.2.3

·       OS: All Oracle GoldenGate supported platforms for Oracle

Introduction

Setting up Conflict Management is key to the OGG bi-direction data sync. Most Active-Active or Multi-Master Systems will have some form of conflict. Conflicts happen due to at least one row being modified on more than one system.

Prior to RDBMS 12.2 and OGG12.2

Tables should contain timestamp columns, or we have to add column onto the table likewise to achieve the same goal. The timestamp column i.e. CHANGE_TS is used for Conflict management and decide resolution of conflict when it happen.

CDR steps are as following:

Datatypes that can be compared are supported:

• NUMERIC

• DATE

• TIMESTAMP

• CHAR/NCHAR

• VARCHAR/NVARCHAR

Typically these data types are used with parameters like COMPARECOLS, GETBEFORECOLS, and in resolution parameters using RESOLVECONFLICT ([USEMIN] | [USEMAX]).

LOGALLSUPCOLS

Extract captures before images for UPDATE operations.

Extract captures before images of supplemental logged columns for both UPDATE and DELETE operations.

GETBEFORECOLS

Ensures certain columns are logged

TABLE option in extract

Example:

GETBEFORECOLS (ON UPDATE ALL, ON DELETE ALL)

TRANSLOGOPTION EXCLUDETAG

• Tag supplied to transaction as it is extracted to prevent receiving replicat from trying to send it back to source

• Classic/Integrated Extract (primary)

• Tag can be any number/letter [0-9 A-Z]

Example:

TRANSLOGOPTION EXCLUDETAG 0294

COMPARECOLS

Used by Replicat to detect and resolve update/delete conflicts

Example:

COMPARECOLS (ON UPDATE ALL, ON DELETE ALL)

Since RDBMS 12.2 and OGG12.3

Timestamp column can be added without effecting the application or code base- leverages invisible columns/RDBMS 12.2 kernel.

Auto CDR makes the conflict management much simpler than before, you also need to meet below prerequisites

·       Requires source and target to be Oracle Database 12.2

·       Requires Integrated Extract (Classic Extract does not support invisible columns)

·       Requires Integrated Replicat or Integrated Parallel Replicat

No Visible Application Changes:

•    Easy to enable

•    Invisible timestamp columns automatically created and maintained by database kernel

•    Delete tombstone log to support updates-delete conflicts (new capability)

•    Enables supplemental logging on required columns

•    Allows ability to detect conflicts for LOB column changes

•    No need to specify CDR/REPERROR parameters

•    Last record wins (highest timestamp)

Enable by ADD_AUTO_CDR for LOB columns

SQL> BEGIN

dbms_goldengate_adm.add_auto_cdr(

schema_name => 'HR',

table_name => 'EMPLOYEES');

END;

/

PL/SQL procedure successfully completed.

The procedure crates an invisible timestamp column to associate with the row, (If no LOB column exist in table). If we have multiple LOB columns, i.e. 2 LOB columns, it will create 2 separate invisible timestamp columns to associate with each LOB columns and one invisible timestamp column associate with the other columns by default.

If we have a test table with below structure:

After enable the Auto CDR by:

SQL> BEGIN

  DBMS_GOLDENGATE_ADM.ADD_AUTO_CDR(

    schema_name => 'hr',

    table_name  => 'test');

END;

/

PL/SQL procedure successfully completed.

We query the ALL_GG_AUTO_CDR_COLUMNS table

SQL>SELECT TABLE_OWNER,

       TABLE_NAME,

       COLUMN_GROUP_NAME,

       COLUMN_NAME,

       RESOLUTION_COLUMN

  FROM ALL_GG_AUTO_CDR_COLUMNS

  ORDER BY TABLE_OWNER, TABLE_NAME;

We can see 3 CDRTS timestamp columns were created, 2 for LOB columns and one for the other columns.

By enable the Auto CDR, we

•    Automatically creates an invisible timestamp column to associate with the row

•    Invisible timestamp column automatically maintained

•    Automatically enables Delete Tombstone Log Table

•    Tombstone table automatically maintained

•    Automatically enables all column supplemental logging

Monitoring

We can get the statics for CDR resolution by

GGCSI> stats replicat <group name>,reportcdr

From my test till now, with OGG Version 12.3.0.1.2 OGGCORE_12.3.0.1.0_PLATFORMS_171208.0005_FBO and Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production, LOB CDR resolution stats is NOT show here.

Steps for Implementing OGG in an active-active configuration

Enable Oracle GoldenGate on database

1. Open archive log mode

[Source] [Target]

SQL>shutdown immediate;

SQL>startup mount;

SQL>alter database archivelog;

SQL>alter database open;

SQL>alter system switch logfile;

SQL>archive log list;

SQL> alter PLUGGABLE database all open;

SQL>create user c##ogg identified by ogg;

SQL>GRANT DBA to c##ogg container=all;

SQL> exec dbms_goldengate_auth.grant_admin_privilege('c##ogg',container=>'ALL');

2. Open supplemental log data and force logging

[Source] [Target]

SQL>SELECT supplemental_log_data_min, force_logging FROM v$database;

SQL>alter database add supplemental log data;

SQL>alter database force logging;

SQL>SELECT supplemental_log_data_min, force_logging FROM v$database;

3. Enabling Oracle GoldenGate in the Database

[Source] [Target]

SQL> alter system set enable_goldengate_replication = true scope=both;

5. Create OGG Users

[Source] [Target]

SQL> create user ogg identified by ogg;

SQL>GRANT DBA to ogg;

SQL> BEGIN 
  dbms_goldengate_auth.grant_admin_privilege 
  ( 
    grantee => 'c##ogg', 
    privilege_type => 'CAPTURE', 
    grant_select_privileges => TRUE 
  ); 
END; 
/

[Source] [Target]

SQL>BEGIN

dbms_goldengate_auth.grant_admin_privilege

(

grantee => 'c##ogg',

privilege_type => 'APPLY',

grant_select_privileges => TRUE

);

END;

/

Install Oracle GoldenGate

We use the DIPC remote agent version 18.2.3 in this manual,
just download it and unzip, we can use it.

1. Update the tnsnames.ora

tnsnames.ora

SRC =

 
(DESCRIPTION =

   
(ADDRESS_LIST =

      (ADDRESS = (PROTOCOL = TCP)(HOST =
192.168.56.106)(PORT = 1521))

   
)

   
(CONNECT_DATA =

      (SERVICE_NAME = PDBSRC)

   
)

 
)

TGT =

 
(DESCRIPTION =

   
(ADDRESS_LIST =

      (ADDRESS = (PROTOCOL = TCP)(HOST =
192.168.56.106)(PORT = 1521))

   
)

   
(CONNECT_DATA =

      (SERVICE_NAME = PDBTGT)

   
)

 
)

ORCL =

 
(DESCRIPTION =

   
(ADDRESS_LIST =

      (ADDRESS = (PROTOCOL = TCP)(HOST =
192.168.56.106)(PORT = 1521))

   
)

   
(CONNECT_DATA =

      (SERVICE_NAME = ORCL)

   
)

 
)

2. Update the .bash_profile for OCI

.bash_profile

#
.bash_profile

# Get the
aliases and functions

# User
specific environment and startup programs

ORACLE_HOME=/u01/src/oci;  export ORACLE_HOME

export
JAVA_HOME=/usr/java/jdk1.8.0_92

export
JRE_HOME=/usr/java/jdk1.8.0_92/jre

PATH=$JAVA_HOME/bin:$ORACLE_HOME/bin:$PATH
; export PATH

CLASSPATH=$ORACLE_HOME/JRE:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$JAVA_HOME/lib:

LD_LIBRARY_PATH=$ORACLE_HOME:/lib:/usr/lib; export
LD_LIBRARY_PATH

export
TNS_ADMIN='/u01/src/oci/network/admin/'

3. Testing the Connection

Setting up OGG

OGG Parameters on the Source:

1. Manager:

PORT 8809

2. Extracts:

-- Extract :extsrc

extract
extsrc

userid
c##ogg@orcl,password ogg

exttrail
./dirdat/es

TranLogOptions
Excludetag 00

table
pdbsrc.hr.*;

--Pump: pumsrc

extract
pumsrc

passthru

rmthost
<ip>,mgrport 8919

rmttrail
./dirdat/rs

table
pdbsrc.hr.*;

3. Replicat:

--Replicat: reptgt

replicat
reptgt

userid
ogg@src, password ogg

discardfile
reptgt.dsc, append

NOMAPINVISIBLECOLUMNS

map
pdbtgt.hr.test, target pdbsrc.hr. test;

map
pdbtgt.hr.jobs, target pdbsrc.hr. jobs;

OGG Parameters on the Target:

1. Manager:

PORT
8919

2. Extracts:

Extract exttgt:

extract
exttgt

userid
c##ogg@orcl,password ogg

exttrail
./dirdat/et

TranLogOptions
Excludetag 00

table
pdbtgt.hr.*;

Pump: pumtgt

extract
pumtgt

passthru

rmthost
<ip>,mgrport 8809

rmttrail
./dirdat/rt

table
pdbtgt.hr.*;

3. Replicat:

replicat
repsrc

userid
ogg@tgt, password ogg

discardfile
repsrc.dsc, append

NOMAPINVISIBLECOLUMNS

map
pdbtgt.hr.test, target pdbsrc.hr. test;

map
pdbtgt.hr.jobs, target pdbsrc.hr. jobs;

Adding groups:

Source:

GGCSI> obey Src.oby

--Source OGG
Setup

add extract
extsrc, integrated tranlog, begin now

add exttrail
./dirdat/es, extract extsrc

dblogin
userid c##ogg@orcl password ***

register
extract extsrc database container(pdbsrc)

add
schematrandata PDBSRC.HR,allcols

info
trandata PDBSRC.HR.*

add extract
pumsrc, exttrailsource ./dirdat/es

add rmttrail
./dirdat/rs, extract pumsrc

add replicat
reptgt, integrated ,exttrail ./dirdat/rt

Target:

GGCSI>
obey Tgt.oby

----Target
OGG Setup

add extract exttgt,integrated
tranlog, begin now

add exttrail
./dirdat/et, extract exttgt

dblogin
userid c##ogg@orcl password ***

register
extract exttgt database container(pdbtgt)

add
schematrandata pdbtgt.hr,allcols

info
trandata pdbtgt.hr.*

add extract
pumtgt, exttrailsource ./dirdat/et

add rmttrail
./dirdat/rt,extract pumtgt

add replicat
repsrc, integrated ,exttrail ./dirdat/rs

Start the processes

Source and Target:

GGSCI>
start mgr !

GGSCI>
start er *

Source:

Target:

Testing Steps for Auto-CDR

In all our test cases below, we took the DML against the
source database firstly and then against target in very short time.

Insert Conflict

Source:

And then on Target:

Query the source again:

Review in GGSCI:

Update Conflict

This time we are going to test a table with LOB data type
column,

INSERT INTO hr.test
VALUES(3,'Pic No.1',bfilename('IMAGE_DIR','1.png')); commit;

What we get on the source and target:

Source:

Target:

Update the LOB value in table:

Source:

update  hr.test set
IMAGE =(bfilename('IMAGE_DIR','2.png')),MEMO='PIC #2' where id=3; commit;

Target:

update  hr.test set
IMAGE =(bfilename('IMAGE_DIR','3.png')),MEMO='PIC #3' where id=3; commit;

We also query the sample data for those 2 LOB column

For Pic#2:

For Pic#3:

Query source and target we get:

We did not get the CDR stats for LOB columns by running:

stats
<Group_name>,reportcdr

Delete Conflict

1. Run the SQL to delete the record on both source
and target:

delete from
hr.emp where empno=1; commit;

2. Query the result on both databases:

select
EMPNO,SAL,CDRTS$ROW from hr.emp  where
empno=1;

No record returns

3. Review in OGG

Source:

Target:

Conclusion

This paper covers the OGG Auto CDR feature which introduced
in version 12.3.

•    It
has No Visible Application Changes and more simple to handle.

•    It
can handle to detect conflicts for LOB column changes.

•    A
Last record wins (with highest timestamp).

If you want to customize the conflict detection resolution
more complex, not only highest timestamp win, you can still try classic CDR. Please
refer to the OGG Administrative Guide for a more extensive view of CDR.

Reference

1. Oracle GoldenGate Best Practices: Active-Active
Configuration with DDL and DML CDR --Tracy West

2. Oracle GoldenGate 12.3: Conflict, Detection and
Resolution –Bobby Curtis

https://www.slideshare.net/BobbyCurtisMBA/goldengate-cdr-from-ukoug-2017

3. CDR Info:
https://docs.oracle.com/goldengate/c1230/gg-winux/GGODB/automatic-conflict-detection-and-resolution2.htm#GGODB-GUID-29EDF7BF-6BD8-4E02-98AB-DDBB4719D038

Oracle GoldenGate Best Practices: Active-Active Configuration with DML Auto CDR的更多相关文章

  1. Oracle Grid control 11g及Active DataGuard 11g安装部署

    Oracle Grid control 11g及Active DataGuard 11g安装部署(一) 原贴 http://blog.csdn.net/lichangzai/article/detai ...

  2. Oracle GoldenGate for Oracle 11g to PostgreSQL 9.2.4 Configuration

    Installing and setting up Oracle GoldenGate connecting to an Oracle database Also please make sure t ...

  3. zabbix 启到不起来:active check configuration update from [127.0.0.1:10051] started to fail (cannot connect to [[127.0.0.1]:10051]: [111] Connection refused)

    cat /var/log/zabbix_agent_log 查看日记出现报错:active check configuration update from [127.0.0.1:10051] star ...

  4. Oracle GoldenGate实现数据库同步

    前言:最近刚好在弄数据库同步,网上查了些资料再加上自己整理了一些,做个分享! 一.GoldenGate的安装 1.安装包准备 数据库版本:Oracle Database 11g Release 2(1 ...

  5. oracle goldengate 远程捕获和投递

    很早之前,OGG只支持部署在数据库主机上,这叫本地化部署.而现在OGG支持远端部署,即OGG软件不安装在数据库主机上,而是安装在单独的机器上,负责数据抽取和投递. 这样做的好处: l 易于管理 - 在 ...

  6. Oracle GoldenGate 二、配置和使用

    Oracle GoldenGate 二.配置和使用 配置和使用GoldenGate的步骤 1 在源端和目标端配置数据库支持GoldenGate 2 在源端和目标端创建和配置GoldenGate实例 3 ...

  7. Oracle GoldenGate

    Oracle GoldenGate实现数据库同步 前言:最近刚好在弄数据库同步,网上查了些资料再加上自己整理了一些,做个分享! 一.GoldenGate的安装 官方文档: Oracle®GoldenG ...

  8. oracle ogg 单实例双向复制搭建(oracle-oracle)--Oracle GoldenGate

    oracle ogg 单实例双向复制搭建(oracle-oracle)--Oracle GoldenGate --继昨天的测试,这一篇实施单实例双向复制(完全重新搭建) --环境不变 db1,db2( ...

  9. Streaming data from Oracle using Oracle GoldenGate and Kafka Connect

    This is a guest blog from Robin Moffatt. Robin Moffatt is Head of R&D (Europe) at Rittman Mead, ...

随机推荐

  1. 备战省赛组队训练赛第十四场(UPC)

    codeforces:传送门 upc:传送门 外来题解: [1]:https://blog.csdn.net/ccsu_cat/article/details/86707446 [2]:https:/ ...

  2. Visio文本相关操作

    三种方式:双击形状输入文本,插入文本框, 文本工具 文本块工具 选择后可以对文本进行移动旋转 如果要给文本加入边框 直接显示线条就可以了 因为都是文本框 添加特殊文本: 插入符号 插入域 比如当前时间 ...

  3. CodeTypeDeclaration,CodeMemberProperty动态生成代码

    由于是CodeDom些列,所以先介绍几个CodeDom表达式: :CodeConditionStatement:判断语句即是if(condition){} else{},看最全的那个构造函数: pub ...

  4. 基于 HTML5 + WebGL 的无人机 3D 可视化系统

    前言 近年来,无人机的发展越发迅速,既可民用于航拍,又可军用于侦察,涉及行业广泛,也被称为“会飞的照相机”.但作为军事使用,无人机的各项性能要求更加严格.重要.本系统则是通过 Hightopo 的   ...

  5. 本地安装配置Gradle及IDEA使用本地Gradle

    一.下载Gradle   下载地址:http://services.gradle.org/distributions/   下载版本的bin.zip 二.配置环境变量 三.验证   在cmd模式下查看 ...

  6. MySQL基础之事务编程学习笔记

    MySQL基础之事务编程学习笔记 在学习<MySQL技术内幕:SQL编程>一书,并做了笔记.本博客内容是自己学了<MySQL技术内幕:SQL编程>事务编程一章之后,根据自己的理 ...

  7. 只用这 6 个字符,就可以写出任意 JavaScript 代码!

    你可能在网上见过有人用 几个不同的字符写的各种稀奇古怪的 JavaScript 代码,虽然看起来奇怪,但是能正常运行!比如这个: (!(~+[])+{})[--[~+""][+[] ...

  8. C# 对象与引用变量

    从宏观的角度来看,对象是类的实例.比如: //定义一个名为Someone的类,代表这么一些人(通过指定年龄,性别,性格等基本信息)class Someone {  public int age;  p ...

  9. wide&deep模型演化

    推荐系统模型演化 LR-->GBDT+LR FM-->FFM-->GBDT+FM|FFM FTRL-->GBDT+FTRL Wide&DeepModel (Deep l ...

  10. win服务器管理工具,服务器vps管理

    win系列服务器,vps桌面如何管理?用这个工具: IIS7远程桌面批量管理,同时管理上千台vps,服务器,3389远程端口.