Track 造成Goldengate abended的那条record

Email收到了这样的报错：

2016-12-07 02:52:22  WARNING OGG-01004  Aborted grouped transaction on 'MSP.USER_ACTIVITY_LIFETIME_AGG', Database error 1403 (OCI Error ORA-01403: no data found, SQL <UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" x SET x."USER_ACTIVITY_LIFETIME_ID" = :a0,x."NIKE_PLUS_USER_ID" = :a1,x."ACTIVITY_NUM" = :a3,x."FUEL_TOTAL" = :a4,x."STEP_TOTAL" = :a5,x."CALORIE_TOTAL" = :a6,x."DURATION_TOTAL" = :a7,x."DISTANCE_TOTAL" = :a8,x."WALK_TOTAL" = :a9,x."FUEL_ACTIVITY_NUM" = :a10,x."FUEL_DAILY_AVG" = :a11,x."BEST_ACTIVE_HOUR_START" = :a12,x."BEST_ACTIVE_MINUTE_START" = :a13,x."BEST_ACTIVE_HOUR_END" = :a14,x."BEST_ACTIVE_MINUTE_END" = :a15,x."FUEL_BEST_ACTIVE_HOUR" = :a16,x."FUEL_BEST_DAY" = :a17,x."FUEL_BEST_DAY_AMT" = :a18,x."FUEL_BEST_WEEK" = :a19,x."FUEL_BEST_WEEK_AMT" = :a20,x."FUEL_BEST_MONTH" = :a21,x."FUEL_BEST_MONTH_AMT" = :a22,x."CREATE_DATE" = :a23,x."UPDATE_DATE" = :a24,x."FUEL_BEST_HOUR_AMT" = :a25,x."CURRENT_STREAK" = :a27,x."CURRENT_STREAK_START_DATE" = :a28,x."MISS_STREAK" = :a29,x."MISS_STREAK_START_DATE" = :a30,x."LONGEST_STREAK" = :a31,x."LONGEST_STREAK_START_DATE" = :a32,x."LONGEST_MISS_STREAK" = :a33,x."LONGEST_MISS_STREAK_START_DATE" = :a34,x."DEVICE_ID" = :a35,x."FUEL_BEST_SUNDAY" = :a36,x."FUEL_BEST_SUNDAY_AMT" = :a37,x."FUEL_BEST_MONDAY" = :a38,x."FUEL_BEST_MONDAY_AMT" = :a39,x."FUEL_BEST_TUESDAY" = :a40,x."FUEL_BEST_TUESDAY_AMT" = :a41,x."FUEL_BEST_WEDNESDAY" = :a42,x."FUEL_BEST_WEDNESDAY_AMT" = :a43,x."FUEL_BEST_THURSDAY" = :a44,x."FUEL_BEST_THURSDAY_AMT" = :a45,x."FUEL_BEST_FRIDAY" = :a46,x."FUEL_BEST_FRIDAY_AMT" = :a47,x."FUEL_BEST_SATURDAY" = :a48,x."FUEL_BEST_SATURDAY_AMT" = :a49,x."ACTIVE_PERCENTAGE" = :a50,x."FUEL_BEST_YEAR" = :a51,x."FUEL_BEST_YEAR_AMT" = :a52,x."SUPER_GOAL_CNT" = :a53,x."TAG_IT_STREAK_CAREER" = :a54,x."TAG_IT_STREAK_CURRENT" = :a55,x."TAG_IT_TOTAL" = :a56,x."TAG_NOT_IT_TOTAL" = :a57,x."TAG_STREAK_CAREER" = :a58,x."TAG_STREAK_CURRENT" = :a59,x."ESP_SCORE_TOTAL" = :a60,x."ACTIVE_TIME_TOTAL" = :a61,x."STEP_CNT" = :a62,x."STEP_CNT_AVG" = :a63,x."STEP_WEEK_AVG" = :a64,x."STEP_CNT_MAX" = :a65,x."STEP_DATE_LONGEST" = :a66,x."FUEL_BEST_CURRENT_WEEK" = :a67,x."FUEL_BEST_CURRENT_WEEK_AMT" = :a68,x."START_DATE" = :a69,x."SMALL_HOURS_COUNT" = :a70,x."LATE_NIGHT_COUNT" = :a71,x."BRIGHT_AND_EARLY_COUNT" = :a72,x."MORNING_COUNT" = :a73,x."EARLY_MORNING_COUNT" = :a74,x."LATE_MORNING_COUNT" = :a75,x."DAYTIME_COUNT" = :a76,x."EARLY_AFTERNOON_COUNT" = :a77,x."LATE_AFTERNOON_COUNT" = :a78,x."EVENING_COUNT" = :a79,x."EARLY_EVENING_COUNT" = :a80,x."EARLY_NIGHT_COUNT" = :a81,x."MOST_CALORIES_BURNED" = :a82,x."LONGEST_ACTIVITY_DURATION" = :a83,x."FUEL_BEST_SESSION" = :a84,x."WORKOUTOFDAY_TOTAL" = :a85,x."SOCIAL_SHARE_TOTAL" = :a86,x."FUEL_BEST_SESSION_DATE" = :a87,x."LAST_ACTIVITY_TZ_OFFSET" = :a88,x."LAST_ACTIVITY_DST_OFFSET" = :a89,x."CALORIE_BEST_WEEK" = :a90,x."CALORIE_BEST_WEEK_AMT" = :a91,x."LONGEST_ACTIVITY_DURATION_DATE" = :a92,x."DURATION_BEST_WEEK" = :a93,x."DURATION_BEST_WEEK_AMT" = :a94,x."INTENSITY_BEST_AMT" = :a95,x."INTENSITY_BEST" = :a96,x."FUEL_SUNDAY_TOTAL" = :a97,x."FUEL_SUNDAY_TOTAL_COUNT" = :a98,x."FUEL_MONDAY_TOTAL" = :a99,x."FUEL_MONDAY_TOTAL_COUNT" = :a100,x."FUEL_TUESDAY_TOTAL" = :a101,x."FUEL_TUESDAY_TOTAL_COUNT" = :a102,x."FUEL_WEDNESDAY_TOTAL" = :a103,x."FUEL_WEDNESDAY_TOTAL_COUNT" = :a104,x."FUEL_THURSDAY_TOTAL" = :a105,x."FUEL_THURSDAY_TOTAL_COUNT" = :a106,x."FUEL_FRIDAY_TOTAL" = :a107,x."FUEL_FRIDAY_TOTAL_COUNT" = :a108,x."FUEL_SATURDAY_TOTAL" = :a109,x."FUEL_SATURDAY_TOTAL_COUNT" = :a110,x."STARS_BEST_DAY_AMT" = :a111,x."STARS_BEST_DAY_DATE" = :a112,x."MOST_FUEL_IN_SESSION" = :a113,x."MOST_FUEL_IN_SESSION_DATE" = :a114 WHERE x."UPM_USER_ID" = :b0 AND x."ACTIVITY_TYPE_ID" = :b1>).

2016-12-07 02:52:28  WARNING OGG-01154  SQL error 1403 mapping MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG OCI Error ORA-01403: no data found, SQL <UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" x SET x."USER_ACTIVITY_LIFETIME_ID" = :a0,x."NIKE_PLUS_USER_ID" = :a1,x."ACTIVITY_NUM" = :a3,x."FUEL_TOTAL" = :a4,x."STEP_TOTAL" = :a5,x."CALORIE_TOTAL" = :a6,x."DURATION_TOTAL" = :a7,x."DISTANCE_TOTAL" = :a8,x."WALK_TOTAL" = :a9,x."FUEL_ACTIVITY_NUM" = :a10,x."FUEL_DAILY_AVG" = :a11,x."BEST_ACTIVE_HOUR_START" = :a12,x."BEST_ACTIVE_MINUTE_START" = :a13,x."BEST_ACTIVE_HOUR_END" = :a14,x."BEST_ACTIVE_MINUTE_END" = :a15,x."FUEL_BEST_ACTIVE_HOUR" = :a16,x."FUEL_BEST_DAY" = :a17,x."FUEL_BEST_DAY_AMT" = :a18,x."FUEL_BEST_WEEK" = :a19,x."FUEL_BEST_WEEK_AMT" = :a20,x."FUEL_BEST_MONTH" = :a21,x."FUEL_BEST_MONTH_AMT" = :a22,x."CREATE_DATE" = :a23,x."UPDATE_DATE" = :a24,x."FUEL_BEST_HOUR_AMT" = :a25,x."CURRENT_STREAK" = :a27,x."CURRENT_STREAK_START_DATE" = :a28,x."MISS_STREAK" = :a29,x."MISS_STREAK_START_DATE" = :a30,x."LONGEST_STREAK" = :a31,x."LONGEST_STREAK_START_DATE" = :a32,x."LONGEST_MISS_STREAK" = :a33,x."LONGEST_MISS_STREAK_START_DATE" = :a34,x."DEVICE_ID" = :a35,x."FUEL_BEST_SUNDAY" = :a36,x."FUEL_BEST_SUNDAY_AMT" = :a37,x."FUEL_BEST_MONDAY" = :a38,x."FUEL_BEST_MONDAY_AMT" = :a39,x."FUEL_BEST_TUESDAY" = :a40,x."FUEL_BEST_TUESDAY_AMT" = :a41,x."FUEL_BEST_WEDNESDAY" = :a42,x."FUEL_BEST_WEDNESDAY_AMT" = :a43,x."FUEL_BEST_THURSDAY" = :a44,x."FUEL_BEST_THURSDAY_AMT" = :a45,x."FUEL_BEST_FRIDAY" = :a46,x."FUEL_BEST_FRIDAY_AMT" = :a47,x."FUEL_BEST_SATURDAY" = :a48,x."FUEL_BEST_SATURDAY_AMT" = :a49,x."ACTIVE_PERCENTAGE" = :a50,x."FUEL_BEST_YEAR" = :a51,x."FUEL_BEST_YEAR_AMT" = :a52,x."SUPER_GOAL_CNT" = :a53,x."TAG_IT_STREAK_CAREER" = :a54,x."TAG_IT_STREAK_CURRENT" = :a55,x."TAG_IT_TOTAL" = :a56,x."TAG_NOT_IT_TOTAL" = :a57,x."TAG_STREAK_CAREER" = :a58,x."TAG_STREAK_CURRENT" = :a59,x."ESP_SCORE_TOTAL" = :a60,x."ACTIVE_TIME_TOTAL" = :a61,x."STEP_CNT" = :a62,x."STEP_CNT_AVG" = :a63,x."STEP_WEEK_AVG" = :a64,x."STEP_CNT_MAX" = :a65,x."STEP_DATE_LONGEST" = :a66,x."FUEL_BEST_CURRENT_WEEK" = :a67,x."FUEL_BEST_CURRENT_WEEK_AMT" = :a68,x."START_DATE" = :a69,x."SMALL_HOURS_COUNT" = :a70,x."LATE_NIGHT_COUNT" = :a71,x."BRIGHT_AND_EARLY_COUNT" = :a72,x."MORNING_COUNT" = :a73,x."EARLY_MORNING_COUNT" = :a74,x."LATE_MORNING_COUNT" = :a75,x."DAYTIME_COUNT" = :a76,x."EARLY_AFTERNOON_COUNT" = :a77,x."LATE_AFTERNOON_COUNT" = :a78,x."EVENING_COUNT" = :a79,x."EARLY_EVENING_COUNT" = :a80,x."EARLY_NIGHT_COUNT" = :a81,x."MOST_CALORIES_BURNED" = :a82,x."LONGEST_ACTIVITY_DURATION" = :a83,x."FUEL_BEST_SESSION" = :a84,x."WORKOUTOFDAY_TOTAL" = :a85,x."SOCIAL_SHARE_TOTAL" = :a86,x."FUEL_BEST_SESSION_DATE" = :a87,x."LAST_ACTIVITY_TZ_OFFSET" = :a88,x."LAST_ACTIVITY_DST_OFFSET" = :a89,x."CALORIE_BEST_WEEK" = :a90,x."CALORIE_BEST_WEEK_AMT" = :a91,x."LONGEST_ACTIVITY_DURATION_DATE" = :a92,x."DURATION_BEST_WEEK" = :a93,x."DURATION_BEST_WEEK_AMT" = :a94,x."INTENSITY_BEST_AMT" = :a95,x."INTENSITY_BEST" = :a96,x."FUEL_SUNDAY_TOTAL" = :a97,x."FUEL_SUNDAY_TOTAL_COUNT" = :a98,x."FUEL_MONDAY_TOTAL" = :a99,x."FUEL_MONDAY_TOTAL_COUNT" = :a100,x."FUEL_TUESDAY_TOTAL" = :a101,x."FUEL_TUESDAY_TOTAL_COUNT" = :a102,x."FUEL_WEDNESDAY_TOTAL" = :a103,x."FUEL_WEDNESDAY_TOTAL_COUNT" = :a104,x."FUEL_THURSDAY_TOTAL" = :a105,x."FUEL_THURSDAY_TOTAL_COUNT" = :a106,x."FUEL_FRIDAY_TOTAL" = :a107,x."FUEL_FRIDAY_TOTAL_COUNT" = :a108,x."FUEL_SATURDAY_TOTAL" = :a109,x."FUEL_SATURDAY_TOTAL_COUNT" = :a110,x."STARS_BEST_DAY_AMT" = :a111,x."STARS_BEST_DAY_DATE" = :a112,x."MOST_FUEL_IN_SESSION" = :a113,x."MOST_FUEL_IN_SESSION_DATE" = :a114 WHERE x."UPM_USER_ID" = :b0 AND x."ACTIVITY_TYPE_ID" = :b1>.

2016-12-07 02:52:28  ERROR   OGG-01296  Error mapping from MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG.

2016-12-07 02:52:28  ERROR   OGG-01668  PROCESS ABENDING.

大致的看一下就是因为要update的数据没找到，然后abended了

平时这种报错看得太多了，加个HC参数(handlecollision) 然后等lag追上了或者等几分钟把HC参数注掉就是了

今天这个case也不例外

但是自己想去挖掘是哪条/哪几条记录造成update的no data found

先来看看HC参数的影响，

使用HANDLECOLLISIONS的几个场景：

1.    target丢失delete记录(missing delete)，忽略该问题并不记录到discardfile

2.    target丢失update记录(missing update)

•    更新的键值是主键=》 update转换成INSERT ，默认情况下插入记录不完整
•    更新的键值是非主键=》 忽略该问题并不记录到discardfile

3.    重复插入已存在的主键值到target表中，这将被replicat转换为UPDATE现有主键值的行的其他非主键列

另：该参数仅处理数据本身的Insert/Delete冲突，如果出现两端映射或其它结构性问题Replicat进程依然会abend，不能被忽略

 

此外对于主键的更新操作，若在target使用HANDLECOLLISIONS且该update丢失，在会转换为INSERT该主键的操作，注意默认情况下插入的记录不完整，FETCHOPTIONS FETCHPKUPDATECOLS将捕获完整的redo image镜像到trail中，这保证把primary key的更新通过HANDLECOLLISIONS转换为对target的一个完整记录的插入。

因为自己一开始遇到这报错，也就先加个HC参数把进程启起来再说了。

事后深入分析嘛---->如下：

看了HC关于update的说法，先看看表主键

SQL> select a.constraint_name,  a.column_name

2   from dba_cons_columns a, dba_constraints b

3   where a.constraint_name = b.constraint_name

and b.constraint_type = 'P'

and a.table_name = 'USER_ACTIVITY_LIFETIME_AGG'  4    5  ;

no rows selected

没有主键！！！

那么就是说加了HC之后就忽略这个报错也不记到discard file里面去了。

那么就是说有数据丢失了！那几条数据也就不管了！（关于数据丢失的话其实得看数据库的数据要求严格与否！因为这里的库是个报表库，定期也有数据的refresh，所以对数据的精确度要求没太高）

看看一开始因为什么数据不存在造成的abended

之前自己都是在logdump里面乱看的，完全找不到很明确的record

经过一番的分析，进程abended的时候会记ggserr.log 也会去记discard file或者report file呀

应该去report、discard file里面找找有价值的信息

先把abended以后的info r4f_sp信息贴出来

2016-12-07 02:52:28  WARNING OGG-01154  SQL error 1403 mapping MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG OCI Error ORA-01403: no data found,

2016-12-07 02:52:28  WARNING OGG-01003  Repositioning to rba 709137304 in seqno 6889.

===================================

GGSCI (ora-bi-p-5.va2.b2c.nike.com) 6> info R4F_SP

REPLICAT   R4F_SP    Last Started 2016-12-03 18:16   Status ABENDED

Checkpoint Lag       00:00:08 (updated 00:05:57 ago)

Log Read Checkpoint  File /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889

2016-12-07 02:52:20.002353  RBA 709137304

=========================================

GGSCI (ora-bi-p-5.va2.b2c.nike.com) 29> sh ls -lrt /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4*

.......（省略）

-rw-r-----. 1 ggadmin oinstall 1023999438 Dec  7 01:04 /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006888

-rw-r-----. 1 ggadmin oinstall  820886242 Dec  7 03:11 /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889

==========================================

去report file找找信息：

***********************************************************************

*                   ** Run Time Statistics **                         *

***********************************************************************

Last record for the last committed transaction is the following:

___________________________________________________________________

Trail name :  /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889

Hdr-Ind    :     E  (x45)     Partition  :     .  (x04)

UndoFlag   :     .  (x00)     BeforeAfter:     A  (x41)

RecLength  :  1619 (x0653)    IO Time    : 2016-12-07 02:52:20.002353

IOType     :    15  (x0f)     OrigNode   :   255  (xff)

TransInd   :     .  (x02)     FormatType :     R  (x52)

SyskeyLen  :     0  (x00)     Incomplete :     .  (x00)

AuditRBA   :      55760       AuditPos   : 2023822924

Continued  :     N  (x00)     RecCount   :     1  (x01)

2016-12-07 02:52:20.002353 FieldComp          Len  1619 RBA 709135576

Name: MSP.USER_ACTIVITY_LIFETIME_AGG

___________________________________________________________________

Reading /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889, current RBA 709153355, 11861808 records

Report at 2016-12-07 02:52:28 (activity since 2016-12-07 00:01:00)

From Table MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG:

#                   inserts:      6112

#                   updates:    450184

#                   deletes:         0

    #                  discards:         1       因为process abended所以会记到discard file了（在未启用HC的情况下）（所以要求我们平时正常运行的goldengate一定不要加上HC参数，实在紧急处理数据问题、abended问题再去用，用完也记得及时注释掉）

Last log location read:

FILE:      /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4006889

SEQNO:     6889

  RBA:       709153355（最后一次log读到这里，待会可以用logdump看看这个RBA的位置）

TIMESTAMP: 2016-12-07 02:52:20.002360

EOF:       NO

READERR:   0

2016-12-07 02:52:28  ERROR   OGG-01668  PROCESS ABENDING.

好的，report file就如上的有用的信息。那么接下来可以看看discard file看看是哪条记录！！！！

ggadmin@ora-bi-p-5:DCBIPRD5:/gg/app/ggadmin/product/12.1.2.1.0/dirrpt$less r4f_sp1.dsc

Oracle GoldenGate Delivery for Oracle process started, group R4F_SP discard file opened: 2016-12-03 18:16:22.461903

Current time: 2016-12-07 02:52:28

Discarded record from action ABEND on error 1403

OCI Error ORA-01403: no data found, SQL <UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG" x SET x."USER_ACTIVITY_LIFETIME_ID" = :a0,x."NIKE_PLUS_USER_ID" = :a1,x."ACTIVITY_NUM" = :a3,x."FUEL_TOTAL" = :a4,x."STEP_TOTAL" = :a5,x."CALORIE_TOTAL" = :a6,x."DURATION_TOTAL" = :a7,x."DISTANCE_TOTAL" = :a8,x."WALK_TOTAL" = :a9,x."FUEL_ACTIVITY_NUM" = :a10,x."FUEL_DAILY_AVG" = :a11,x."BEST_ACTIVE_HOUR_START" = :a12,x."BEST_ACTIVE_MINUTE_START" = :a13,x."BEST_ACTIVE_HOUR_END" = :a14,x."BEST_ACTIVE_MINUTE_END" = :a15,x."FUEL_BEST_ACTIVE_HOUR" = :a16,x."FUEL_BEST_DAY" = :a17,x."FUEL_BEST_DAY_AMT" = :a18,x."FUEL_BEST_WEEK" = :a19,x."FUEL_BEST_WEEK_AMT" = :a20,x."FUEL_BEST_MONTH" = :a21,x."FUEL_BEST_MONTH_AMT" = :a22,x."CREATE_DATE" = :a23,x."UPDATE_DATE" = :a24,x."FUEL_BEST_HOUR_AMT" = :a25,x."CURRENT_STREAK" = :a27,x."CURRENT_STREAK_START_DATE" = :a28,x."MISS_STREAK" = :a29,x."MISS_STREAK_START_DATE" = :a30,x."LONGEST_STREAK" = :a31,x."LONGEST_STREAK_START_DATE" = :a32,x."LONGEST_MISS_STREAK" = :a33,x."LONGEST_MISS_STREAK_STAR

Aborting transaction on /gg/app/ggadmin/product/12.1.2.1.0/dirdat/replicate/pdsp/r4 beginning at seqno 6889 rba 709137304

       error at seqno 6889 rba 709153355（这个RBA跟之前看report file时候log读到最后一个位置的RBA一样，已经很明显的说明了这个位置开始的记录必定是造成process abended的记录，不要暂时不要着急，慢慢来）

Problem replicating MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG

Record not found

Mapping problem with compressed update record (target format)...

*

USER_ACTIVITY_LIFETIME_ID = fa873844-39c5-4b9f-bee1-9dcbe76ece4e

000000: 66 61 38 37 33 38 34 34 2d 33 39 63 35 2d 34 62 |fa873844-39c5-4b|

000010: 39 66 2d 62 65 65 31 2d 39 64 63 62 65 37 36 65 |9f-bee1-9dcbe76e|

000020: 63 65 34 65                                     |ce4e            |

NIKE_PLUS_USER_ID = 430995F13A1119D6E05330690C0ADA3B

000000: 34 33 30 39 39 35 46 31 33 41 31 31 31 39 44 36 |430995F13A1119D6|

000010: 45 30 35 33 33 30 36 39 30 43 30 41 44 41 33 42 |E05330690C0ADA3B|

UPM_USER_ID = 430995F13A1019D6E05330690C0ADA3B

000000: 34 33 30 39 39 35 46 31 33 41 31 30 31 39 44 36 |430995F13A1019D6|

000010: 45 30 35 33 33 30 36 39 30 43 30 41 44 41 33 42 |E05330690C0ADA3B|

后面的列的信息就不贴了，因为这张表有一百多列

看完这里，我就去查查记录在源库PDSP和目标库BI的数据信息

BI：目标库无记录

SQL> select USER_ACTIVITY_LIFETIME_ID,NIKE_PLUS_USER_ID from msp.USER_ACTIVITY_LIFETIME_AGG where UPM_USER_ID ='430995F13A1019D6E05330690C0ADA3B';

no rows selected

PDSP：源库有三条纪录

SQL> select USER_ACTIVITY_LIFETIME_ID,NIKE_PLUS_USER_ID from msp.USER_ACTIVITY_LIFETIME_AGG where UPM_USER_ID ='430995F13A1019D6E05330690C0ADA3B';

USER_ACTIVITY_LIFETIME_ID               NIKE_PLUS_USER_ID

---------------------------------------------------------------------------------------------------------------------------

fa873844-39c5-4b9f-bee1-9dcbe76ece4e                              430995F13A1119D6E05330690C0ADA3B

d8174109-5a5c-4ef4-a752-d30370b67f7c                               430995F13A1119D6E05330690C0ADA3B

2cc196ae-4959-4d3e-a34d-713310aab95e                             430995F13A1119D6E05330690C0ADA3B

看到这里于是明白了，结合这条update语句来看，update语句如下格式

UPDATE "MSP"."USER_ACTIVITY_LIFETIME_AGG"  SET 表的各个列=xxxx    WHERE "UPM_USER_ID" = :b0 AND "ACTIVITY_TYPE_ID" = :b1>.

但是UPM_USER_ID ='430995F13A1019D6E05330690C0ADA3B';这个在BI里面没记录呀（当然也可以根据ACTIVITY_TYPE_ID来找，或者两个一起找，结果一样的）

可以再去logdump里面看看trail file是不是这样的：

直接pos 709153355到这个位置来看

可以看到从709153355这个位置开始后面三条纪录正好是update所丢失的记录

所以这次的问题分析的很透彻，就是BI里面没这三条记录从而导致update不到造成abended，那么该怎么做呢？

个人觉得可以跳过这三条记录，然后手工insert这三条记录进去。（这三条记录直接往源库里面查下，然后转换成insert语句执行在目标库即可完事）

跳过的方法无非就是HANDLECOLLISION或者REPORT ERROR，不过report error好就好在会把这三条跳过去的记录记到discard file！

关于HC和report error的区别在后面另开一篇可以详细的讲下。

分析到这里，也结束了。

自己当时只是加了HC参数去起起来，然后等lag没了再去把HC注掉。其实这样在数据严格要求的环境下这样是不行的。

但是这里要求不太高。定期也有数据重新refresh的操作。

这里再发点加完HC参数起起来之后的信息状态，也可以佐证之前的分析：

可以看到stats看到的        Total update collisions                            3.00

即说明了是三条记录，之前也说了加了HC参数有冲突的数据是不会记到discard file的，我去看了，是没有！

GGSCI (ora-bi-p-5.va2.b2c.nike.com) 34> stats R4F_SP

Sending STATS request to REPLICAT R4F_SP ...

Start of Statistics at 2016-12-07 03:18:16.

Replicating from MSP.USER_ACTIVITY_LIFETIME_AGG to MSP.USER_ACTIVITY_LIFETIME_AGG:

*** Total statistics since 2016-12-07 03:07:15 ***

Total inserts                                    719.00

Total updates                                  54583.00

Total deletes                                      0.00

Total discards                                     0.00

Total operations                               55302.00

    Total update collisions                            3.00

*** Daily statistics since 2016-12-07 03:07:15 ***

Total inserts                                    719.00

Total updates                                  54583.00

Total deletes                                      0.00

Total discards                                     0.00

Total operations                               55302.00

Total update collisions                            3.00

*** Hourly statistics since 2016-12-07 03:07:15 ***

Total inserts                                    719.00

Total updates                                  54583.00

Total deletes                                      0.00

Total discards                                     0.00

Total operations                               55302.00

Total update collisions                            3.00

*** Latest statistics since 2016-12-07 03:07:15 ***

Total inserts                                    719.00

Total updates                                  54583.00

Total deletes                                      0.00

Total discards                                     0.00

Total operations                               55302.00

Total update collisions                            3.00

End of Statistics.