又一例:ORA-600 kclchkblk_4和2662故障---惜分飞

又一例:ORA-600 kclchkblk_4和2662故障

发表于 2024 年 2 月 21 日 由 惜分飞

联系：手机/微信(+86 17813235971) QQ(107644445)

标题：又一例:ORA-600 kclchkblk_4和2662故障

作者：惜分飞版权所有[未经本人同意,不得以任何形式转载,否则有进一步追究法律责任的权利.]

有客户恢复请求:由于未知原因导致aix环境的rac两台主机同时重启之后数据库无法正常启动,初步判断是由于写丢失导致故障（ORA-00742 ORA-00353）

Wed Feb 21 09:23:06 2024

ALTER DATABASE OPEN

This instance was first to open

Abort recovery for domain 0

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_5767246.trc:

ORA-01113: file 32 needs media recovery

ORA-01110: data file 32: '+DATA/xff/datafile/x5sys_cs.dbf'

ORA-1113 signalled during: ALTER DATABASE OPEN...

Wed Feb 21 09:23:27 2024

ALTER DATABASE RECOVER  datafile '+DATA/xff/datafile/x5sys_cs.dbf' 

Media Recovery Start

Serial Media Recovery started

WARNING! Recovering data file 32 from a fuzzy backup. It might be an online

backup taken without entering the begin backup command.

Recovery of Online Redo Log: Thread 2 Group 14 Seq 48490 Reading mem 0

  Mem# 0: +DATA/xff/onlinelog/group_14.313.1060528521

Recovery of Online Redo Log: Thread 1 Group 7 Seq 64195 Reading mem 0

  Mem# 0: +DATA/xff/onlinelog/group_7.306.1060527979

Wed Feb 21 09:24:10 2024

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_5767246.trc:

ORA-00742: Log read detects lost write in thread %d sequence %d block %d

ORA-00334: archived log: '+DATA/xff/onlinelog/group_14.313.1060528521'

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_5767246.trc  (incident=336478):

ORA-00353: log corruption near block 139727 change 26346459680 time 02/20/2024 20:13:50

ORA-00312: online log 14 thread 2: '+DATA/xff/onlinelog/group_14.313.1060528521'

尝试屏蔽一致性强制拉库后数据库报ORA-600 kclchkblk_4
参考:ora-600 2662和ora-600 kclchkblk_4恢复、redo异常 ORA-600 kclchkblk_4 故障恢复

Wed Feb 21 09:55:26 2024

SMON: enabling cache recovery

Wed Feb 21 09:55:26 2024

Redo thread 2 internally disabled at seq 5 (CKPT)

Archived Log entry 112707 added for thread 2 sequence 4 ID 0xffffffffe144183b dest 1:

ARC0: Archiving disabled thread 2 sequence 5

Archived Log entry 112708 added for thread 2 sequence 5 ID 0xffffffffe144183b dest 1:

Wed Feb 21 09:55:28 2024

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_6423264.trc  (incident=360479):

ORA-00600: internal error code, arguments: [kclchkblk_4], [6], [576721660], [6], [576702892]

Incident details in: /oracle/db/diag/rdbms/xff/xff1/incident/incdir_360479/xff1_ora_6423264_i360479.trc

Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

Thread 1 advanced to log sequence 3 (LGWR switch)

  Current log# 7 seq# 3 mem# 0: +DATA/xff/onlinelog/group_7.306.1161510375

Archived Log entry 112709 added for thread 1 sequence 2 ID 0xffffffffe144183b dest 1:

Wed Feb 21 09:55:31 2024

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_6423264.trc:

ORA-00704: bootstrap process failure

ORA-00704: bootstrap process failure

ORA-00600: internal error code, arguments: [kclchkblk_4], [6], [576721660], [6], [576702892]

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_6423264.trc:

ORA-00704: bootstrap process failure

ORA-00704: bootstrap process failure

ORA-00600: internal error code, arguments: [kclchkblk_4], [6], [576721660], [6], [576702892]

后续处理中出现和这个错误类似的ORA-600 2662错误

Wed Feb 21 15:37:35 2024

SMON: enabling cache recovery

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_6357664.trc  (incident=432423):

ORA-00600: internal error code, arguments: [2662], [6], [576742938], [6], [576834283], [12583104]

Incident details in: /oracle/db/diag/rdbms/xff/xff1/incident/incdir_432423/xff1_ora_6357664_i432423.trc

Use ADRCI or Support Workbench to package the incident.

See Note 411.1 at My Oracle Support for error and packaging details.

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_6357664.trc:

ORA-00600: internal error code, arguments: [2662], [6], [576742938], [6], [576834283], [12583104], [], [], [], [], [], []

Errors in file /oracle/db/diag/rdbms/xff/xff1/trace/xff1_ora_6357664.trc:

ORA-00600: internal error code, arguments: [2662], [6], [576742938], [6], [576834283], [12583104], [], [], [], [], [], []

Error 600 happened during db open, shutting down database

USER (ospid: 6357664): terminating the instance due to error 600

Instance terminated by USER, pid = 6357664

通过对oracle scn进行修改,数据库open成功

SQL> recover database;

Media recovery complete.

SQL>

SQL>

SQL>

SQL> oradebug setmypid

Statement processed.

SQL> oradebug DUMPvar SGA kcsgscn_

kcslf kcsgscn_ [700000000019B70, 700000000019BA0) = 00000000 00000000 00000000 00000000

SQL> oradebug poke

BEFORE: [700000000019B70, 700000000019B78) = 00000000 00000000

AFTER:  [700000000019B70, 700000000019B78) = 00000006 22710D2B

SQL> oradebug DUMPvar SGA kcsgscn_

kcslf kcsgscn_ [700000000019B70, 700000000019BA0) = 00000006 22710D2B 00000000 00000000

SQL> alter database open;

 

Database altered.

后续检查发现obj$中的index异常（ORA-08102: index key not found, obj# 39）
类似文章:通过bbed修改obj$中dataobj$重现I_OBJ4索引报ORA-08102错误

SQL> truncate CLUSTER "SYS"."SMON_SCN_TO_TIME_AUX";

 

truncate CLUSTER "SYS"."SMON_SCN_TO_TIME_AUX"

                       *

ERROR at line 1:

ORA-00604: error occurred at recursive SQL level 1

ORA-08102: index key not found, obj# 39, file 1, block 967206 (2)

 

SQL> SQL> select object_name,object_type from dba_objects where object_id=39;

 

OBJECT_NAME                    OBJECT_TYPE

------------------------------ -------------------

I_OBJ4                         INDEX

对于此类问题使用非常规方法把obj$字典表进行重建（需要注意undo需要为自动管理方式,temp不能为空）,参考:
bootstrap$核心index(I_OBJ1,I_USER1,I_FILE#_BLOCK#,I_IND1,I_TS#,I_CDEF1等)异常恢复—ORA-00701错误解决

• ora-600 2662和ora-600 kclchkblk_4恢复
• ORA-600 kcrf_resilver_log_1故障处理
• tab$异常被处理之后报ORA-600 13304故障处理
• Oracle 启动后一会儿就挂掉故障处理—ORA-600 17182
• aix平台tab$被删除可能出现ORA-600 [16703], [1403], [28]错误
• redo异常强制拉库报ORA-600 kcbzib_kcrsds_1修复
• ORA-00600: internal error code, arguments: [4194], [], [], [], [], [], [], []
• 正常open库报ORA-600 2662,ORA-600 4097错误
• 存储断电导致ORA-00325故障恢复
• Patch SCN工具快速解决ORA-600 2662问题
• 分享一次ORA-01113 ORA-01110故障处理过程
• 记录一次200T的数据库恢复经历