OGG19.1 oracle12c到oracle12c经典模式配置实施

OGG19.1 oracle12c到oracle12c经典和集成模式配置实施

1. 目的说明
   
   本文提供Oracle GoldenGate在Oracle db到Oracle db的数据复制安装配置指导，适用于入门练习使用，生产环境配置，在关键步骤和重要参数设置需要小心对待。
   
   OGG19.1 oracle12c到oracle12c经典和集成模式配置实施 ---- 支持DDL复制配置。

2. 参考文档
   
   《叱咤风云GoldenGate企业级运维实战》
   
   Oracle GoldenGate Document 12.3.0/19.1.0

3. 准备系统
   
   准备好两套已经装好Oracle db和Oracle GoldenGate for Oracle的环境。
   
   -- 安装步骤略
   
   -- https://docs.oracle.com/en/middleware/goldengate/core/19.1/installing/installing-classic-architecture.html
   
   -- ./runInstaller -silent -responseFile /tmp/trace/fbo_ggs_Linux_x64_shiphome/Disk1/response/oggcore.rsp
   
   -- /orabin/oraInventory/logs/installActions2020-07-06_04-46-55PM.log
   
   --

source db：

ip：192.168.6.20

db type：Oracle db 12.2.0.1.0

target db：

ip：192.168.6.20

db type：Oracle db 12.2.0.1.0

GoldenGate:

ogg type：GoldenGate for Oracle 19.1.0.0.4

1. 配置source db
   
   4.1 配置source db参数
   
   source db的配置主要是归档模式修改、附加日志的添加、强制日志和--ddl语句的支持。

     --#源端DB开启归档模式和最小附加日志
   
    --4.1.1 源端DB开启归档模式
    archive log list
     --#启用主动归档
     -- 关闭并mount数据库
    shutdown immediate
    startup mount
     -- 开启归档日志
    alter database archivelog;
     -- 设置本地归档路径
    alter system set log_archive_dest_1='location=/oradata/oggsdb/archive' scope=spfile;
     -- 设置归档日志文件名格式
    show parameter  log_archive_format
    ----default    %t_%s_%r.dbf
    alter system set log_archive_format='oggsdb_%t_%s_%r.arc' scope=spfile;
     --关闭数据库，再启动数据库 ，使参数 log_archive_dest_1生效
    shutdown immediate
    startup
     --检查配置和参数生效
    archive log list
    show parameter log_archive_dest_1
    show parameter log_archive_format
   
    -- 4.1.2 强制日志变更
    alter database force logging;  --实时生效  强制日志
    --ALTER DATABASE NO FORCE LOGGING;
   
    -- 4.1.3 开启最小附加日志 和 查看最小附加日志
    alter database add supplemental log data;  --实时生效   最小附加日志
    --alter database drop supplemental log data;
   
     --查看source db归档、附加日志和强制日志：
    select supplemental_log_data_min from v$database;
    select db.LOG_MODE, db.SUPPLEMENTAL_LOG_DATA_MIN, db.FORCE_LOGGING from v$database db;
   
    select db.NAME,
           db.LOG_MODE,
           db.FLASHBACK_ON,
           db.FORCE_LOGGING,
           db.supplemental_log_data_min,
           db.supplemental_log_data_pk,
           db.supplemental_log_data_ui,
           db.supplemental_log_data_ui,
           db.supplemental_log_data_fk,
           db.supplemental_log_data_all,
           db.supplemental_log_data_pl
      from v$database db;
   
    -- 4.1.4  修改GoldenGate参数
    show parameter enable_goldengate_replication
   
    alter system set enable_goldengate_replication = true scope = both;
   
    show parameter enable_goldengate_replication
   
    -- 4.1.5 创建GoldenGate用户
    create tablespace ogg_data datafile '/oradata/oggsdb/ogg_data01.dbf' size 50M autoextend on next 8M;
    create user ogg identified by Password123 default tablespace ogg_data  quota unlimited on ogg_data;
   
    授权GoldeGate用户 ：
    grant connect, resource to ogg;
    grant alter any table, alter system,alter user, create session, alter session to ogg;
    grant select any dictionary, SELECT ANY TRANSACTION,select any table,flashback any table to ogg;
   
    --grant DBA to  ogg; -- DDL and sequence support
   
    --GRANT FLASHBACK ANY TABLE TO db_user
    --GRANT FLASHBACK ON schema.table TO db_user
   
    --11.2.0.4 or later需要执行以下package 权限 ：
    begin
        dbms_goldengate_auth.grant_admin_privilege(grantee             => 'OGG',
                                                   privilege_type         => 'CAPTURE',
                                                   grant_select_privileges => TRUE,
                                                   do_grants             => TRUE);
    END;
    /
      /*----此包dbms_goldengate_auth.grant_admin_privilege授予权限如下18个权限：
        -- Grant/Revoke object privileges
        grant execute on LOGMNR$COL_GG_TABF_PUBLIC to OGG;
        grant execute on LOGMNR$GSBA_GG_TABF_PUBLIC to OGG;
        grant execute on LOGMNR$KEY_GG_TABF_PUBLIC to OGG;
        grant execute on LOGMNR$SEQ_GG_TABF_PUBLIC to OGG;
        grant execute on LOGMNR$TAB_GG_TABF_PUBLIC to OGG;
        grant insert on LOGMNR_RESTART_CKPT$ to OGG;
        -- Grant/Revoke role privileges
        grant select_catalog_role to OGG;
        -- Grant/Revoke system privileges
        grant alter any table to OGG;
        grant alter session to OGG;
        grant create evaluation context to OGG with admin option;
        grant create job to OGG;
        grant create rule to OGG with admin option;
        grant create rule set to OGG with admin option;
        grant dequeue any queue to OGG with admin option;
        grant execute any rule set to OGG with admin option;
        grant flashback any table to OGG;
        grant logmining to OGG;
        grant select any table to OGG;
      */
   
    --4.1.6 配置DDL语句支持 ( Support for DDL Capture in Classic Capture Mode )
    --Classic capture mode requires the use of the Oracle GoldenGate DDL trigger to
    --capture DDL from an Oracle Database. Native DDL capture is not supported by classic capture mode.
   
    --授权GoldeGate用户 ：(除以上 4.1.5 授权部分外，还需要 DBA 权限 支持 DDL and sequence 捕获)
   
    --a.
    --创建存放ddl信息的GoldenGate用户： ( 如上 共用 ogg 用户 )
    --create tablespace ogg_data datafile '/oradata/oggsdb/ogg_data01.dbf' size 50M autoextend on next 8M;
    --create user ogg identified by Password123 default tablespace ogg_data  quota unlimited on ogg_data;
   
    --b. 此可暂时忽略
    --   (Optional)To cause user DDL activity to fail when the DDL tablespace fills up, edit the params.sql script
    --   and set the ddl_fire_error_in_trigger parameter to TRUE. As a best practice, make certain to size the tablespace appropriately in the first place.
    #params.sql内容
    define the ddl_fire_error_in_trigger = 'TRUE'
    define allow_invisible_index_keys = 'TRUE'
    #To enable trigger-based DDL replication to recognize Oracle invisible indexes as
    #unique identifiers, set the following parameter to TRUE in the params.sql script:
   
    --c. 此可暂时忽略
    --#at the root of the Oracle GoldenGate directory. Do not alterthe file name or location.
    EDIT PARAMS ./GLOBALS
    # 内容
    GGSCHEMA schema_name
    ##Save and close the GLOBALS file and the params.sql file.
   
    --d.
    grant dba to ogg ;
    GRANT EXECUTE ON utl_file TO ogg;
   
    #Change directories to the Oracle GoldenGate installation directory.
    # ----The specific location is: oggma_install_home/lib/sql/legacy.
    #Exit all Oracle sessions. Prevent the start of any new sessions.
    进入GoldenGate目录，调用SQL脚本：
    cd $OGG_HOME
    sqlplus / as sysdba
   
    SQL> @marker_setup.sql;
    #输入准备好的用户ogg
   
    SQL> @ddl_setup.sql;
    #输入准备好的用户ogg 
   
    SQL> @role_setup.sql;
    #输入准备好的用户ogg
    #Grant the role that was created (default name is GGS_GGSUSER_ROLE) to all Oracle GoldenGate Extract users.
    #根据上面执行结果的提示授权角色给ogg用户
    SQL> grant GGS_GGSUSER_ROLE to ogg;
   
    SQL> @ddl_enable.sql;
   
    ##To Install and Use the Optional Performance Tool
    SQL> @ddl_pin ogg
    至此，ddl语句的支持配置完成。
   
    #验证DDL安装 --运行marker_status.sql校验DDL的状态
    SQL> @marker_status.sql
   
    ------------------------------------
    -- Removing the DDL Objects from the System
    -- 卸载 DDL 复制 支持
    -- cd $OGG_HOME
    -- ggsci
    -- ggsci> STOP EXTRACT group1
    -- ggsci> STOP REPLICAT group1
    --
    -- sqlplus / as sysdba
    -- SQL> @ddl_disable.sql;
    -- SQL> @ddl_remove.sql
    -- SQL> @marker_remove.sql
   

4.2 配置source db manager进程

--EXTRACT exoggs

--SETENV (ORACLE_HOME="/orabin/product/12.2.0.1")

--SETENV (ORACLE_SID = "OGGSDB")

进入GoldenGatean安装目录，创建工作区目录：

cd $OGG_HOME

./ggsci

GGSCI (dbserver) 1> create subdirs

Creating subdirectories under current directory /oradata/ogg

Parameter file /oradata/ogg/dirprm: created.

Report file /oradata/ogg/dirrpt: created.

Checkpoint file /oradata/ogg/dirchk: created.

Process status files /oradata/ogg/dirpcs: created.

SQL script files /oradata/ogg/dirsql: created.

Database definitions files /oradata/ogg/dirdef: created.

Extract data files /oradata/ogg/dirdat: created.

Temporary files /oradata/ogg/dirtmp: created.

Credential store files /oradata/ogg/dircrd: created.

Masterkey wallet files /oradata/ogg/dirwlt: created.

Dump files /oradata/ogg/dirdmp: created.

GGSCI (dbserver) 2>

--创建数据目录(可在dirdat目录下为 oggsdb的抽取进程创建一个单独数据存放目录 )

--cd $OGG_HOME

--oracle[/oradata/ogg]$mkdir -p dirdat/oggsdb

---- /oradata/ogg/dirdat/oggsdb

编辑配置manager参数文件 ：

GGSCI > edit params mgr

添加以下内容

PORT 7809

DYNAMICPORTLIST 7800-7900

ACCESSRULE, PROG , IPADDR 192.168.6.20, ALLOW

ACCESSRULE, PROG SERVER, ALLOW

--PURGEOLDEXTRACTS ./dirdat/oggsdb/ex, usecheckpoints, minkeephours 12 #### 参数 minkeepdays 2

PURGEOLDEXTRACTS ./dirdat//, usecheckpoints, minkeepdays 2 #### 同一台主机，源和目标库共用一个MGR

AUTORESTART EXTRACT *, RETRIES 5, WAITMINUTES 2, RESETMINUTES 60

lagreporthours 1

laginfominutes 30

lagcriticalminutes 45

*/

对需要的表添加trandata

cd $OGG_HOME

./ggsci

GGSCI > dblogin userid ogg@oggsdb password Password123

Successfully logged into database.

GGSCI > add trandata scott.emp

--info trandata scott.emp

--select * from DBA_LOG_GROUPS

4.3 配置source db extract进程

--select * from nls_database_parameters

添加一个extract进程 ：

GGSCI > add extract exggsdb,tranlog,begin now ----, threads 2

编辑extract参数文件：

GGSCI (dbserver) 13> edit params exggsdb

添加以下内容

extract exggsdb

setenv (ORACLE_SID = "OGGSDB")

setenv (NLS_LANG = "AMERICAN_AMERICA.AL32UTF8")

userid ogg@oggsdb, password Password123

TRANLOGOPTIONS DBLOGREADER

reportcount every 30 minutes, rate

discardrollover at 3:00

warnlongtrans 2h, checkinterval 300

exttrail /oradata/ogg/dirdat/oggsdb/ex

discardfile /oradata/ogg/dirdat/exggsdb.dsc, append, megabytes 100

--gettruncates ---- DDL support parameter

getupdatebefores

nocompressdeletes

nocompressupdates

--LOGALLSUPCOLS

dboptions allowunusedcolumn

fetchoptions nousesnapshot

fetchoptions fetchpkupdatecols

table SCOTT.EMP;

创建本地trail文件 ：

GGSCI > add exttrail /oradata/ogg/dirdat/oggsdb/ex, extract exggsdb

EXTTRAIL added.

4.4 配置source db pump进程

添加一个pump进程（本质上也是extract进程），并指定远程trail文件：

GGSCI > add extract ppggsdb,exttrailsource /oradata/ogg/dirdat/oggsdb/ex

EXTRACT added.

编辑pump参数文件：

GGSCI (dbserver) 16> edit params ppggsdb

添加以下内容

extract ppggsdb

passthru

rmthost 192.168.6.20, mgrport 7809 ---- target db主机ip，管理进程端口号 ##共用同一台主机的mgr进程 测试

rmttrail /oradata/ogg/dirdat/oggtdb/pp

--ddl -- DDL support parameter

table scott.emp;

指定远程trail文件：

GGSCI > add rmttrail /oradata/ogg/dirdat/oggtdb/pp, extract ppggsdb

RMTTRAIL added.

4.5 启动source进程

查看extrac进程，启动进程后注意查看ggserr.log的日志信息，确认无警告错误，启动pump进程需先启动target db的mgr进程，否则会报错无法启动：

开一个命令行：

cd $OGG_HOME

tail -f 50 ggserr.log

再开一个命令行：

cd $OGG_HOME

./ggsci

GGSCI > info all

GGSCI > start mgr

GGSCI > start exggsdb

GGSCI > start ppggsdb

GGSCI > info all

---

##########同一台主机，两个oracle数据库测试(source --> target), 可以共用OGG MGR 进程?!

##########若 源数据库 和 目标数据库 在不同主机，目标端也需要安装OGG ，并配置OGG 数据目录。

1. 配置target db

5.1 配置target db参数

5.1.1 创建GoldenGate用户

创建GoldenGate用户

create tablespace ogg_data datafile '/oradata/oggtdb/ogg_data01.dbf' size 50M autoextend on next 8M;

create user ogg identified by Password123 default tablespace ogg_data quota unlimited on ogg_data;

	设置enable_goldengate_replication参数为true
	alter system set enable_goldengate_replication = true scope = both;

    授权GoldeGate用户 ：
    grant connect, resource to ogg;
    grant alter any table, alter system,alter user, create session, alter session to ogg;
    grant select any dictionary, SELECT ANY TRANSACTION,select any table,flashback any table to ogg;

	授权GoldeGate用户 ：
	grant dba to ogg;   ---- 支持DDL replicate配置

5.1.2 添加checkpoint表

GGSCI> edit params ./GLOBALS

添加以下内容

ggschema ogg

checkpointtable ogg.checkpoint

登陆target db添加checkpoint表

export ORACLE_SID=OGGTDB

./ggsci

GGSCI > dblogin userid ogg@OGGTDB, password Password123

Successfully logged into database.

GGSCI (dbserver as ogg@OGGTDB) > add checkpointtable ogg.checkpoint

Successfully created checkpoint table ogg.checkpoint.

5.1 配置target db manager进程

----编辑manager参数文件： ---- 同一台主机，两个oracle数据库，可以共用同一个 MGR

----GGSCI (dbserver as ogg@orcl) > edit params mgr

----#添加以下内容

----PORT 7809

----DYNAMICPORTLIST 7800-7900

----ACCESSRULE, PROG , IPADDR 192.168.6.20, ALLOW

----PURGEOLDEXTRACTS ./dirdat/oggtdb/, usecheckpoints, minkeepdays 2 #### 参数 minkeepdays 2 #minkeephours 12

----AUTORESTART ER *, RETRIES 5, WAITMINUTES 2, RESETMINUTES 60

----lagreporthours 1

----laginfominutes 30

----lagcriticalminutes 45

5.2 配置target db replicat进程

添加一个replicat进程 ：

GGSCI (dbserver) 2>

GGSCI (dbserver) 2>

GGSCI (dbserver as ogg@oggtdb) 4> add replicat repggtdb,exttrail /oradata/ogg/dirdat/oggtdb/pp

编辑replicat参数文件：

GGSCI (dbserver as ogg@oggtdb) 6> edit params repggtdb

添加以下内容

replicat repggtdb

setenv (ORACLE_SID = "OGGTDB")

setenv (NLS_LANG = "AMERICAN_AMERICA.AL32UTF8")

userid ogg@oggtdb, password Password123

dboptions deferrefconst

gettruncates

report at 06:00

reportcount every 30 minutes, rate

reportrollover at 02:00

--reperror default, abend

REPERROR DEFAULT,DISCARD

DBOPTIONS NOSUPPRESSTRIGGERS

--handlecollisions

allownoopupdates

--assumetargetdefs

discardfile /oradata/ogg/dirdat/oggtdb/repggtdb.dsc, append, megabytes 100

discardrollover at 02:00

--DDL

map scott.emp, target scott.emp;

----启动 MGR 和 REPLICATE 进程。

1. 故障排查trouble shooting

2. 测试case