环境:Solaris 10 + Oracle 11.2.0.1

现象:alert告警日志定期出现ORA-600 [13011]错误

1.故障现象

数据库alert 日志:

Fri Jul 13 02:00:00 2018
Clearing Resource Manager plan via parameter
Fri Jul 13 02:00:46 2018
Errors in file /u01/app/oracle/diag/rdbms/prod/prod/trace/prod_j000_1757.trc (incident=42249):
ORA-00600: internal error code, arguments: [13011], [67896], [8421830], [44], [8421830], [17], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/prod/prod/incident/incdir_42249/prod_j000_1757_i42249.trc
Fri Jul 13 02:00:49 2018
Trace dumping is performing id=[cdmp_20180713020049]
Fri Jul 13 02:00:49 2018
Sweep [inc][42249]: completed
Sweep [inc2][42249]: completed
Fri Jul 13 03:00:55 2018
Errors in file /u01/app/oracle/diag/rdbms/prod/prod/trace/prod_j000_2053.trc (incident=42250):
ORA-00600: internal error code, arguments: [13011], [67896], [8421830], [44], [8447731], [0], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/prod/prod/incident/incdir_42250/prod_j000_2053_i42250.trc
Fri Jul 13 03:00:56 2018
Trace dumping is performing id=[cdmp_20180713030056]
Fri Jul 13 03:01:48 2018
Sweep [inc][42250]: completed
Sweep [inc2][42250]: completed
Fri Jul 13 04:00:03 2018
Errors in file /u01/app/oracle/diag/rdbms/prod/prod/trace/prod_j000_2338.trc (incident=42251):
ORA-00600: internal error code, arguments: [13011], [67896], [8421830], [44], [8421830], [0], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/prod/prod/incident/incdir_42251/prod_j000_2338_i42251.trc
Fri Jul 13 04:00:05 2018
Trace dumping is performing id=[cdmp_20180713040005]
Fri Jul 13 04:00:05 2018
Sweep [inc][42251]: completed
Sweep [inc2][42251]: completed
省略部分相似输出..
Fri Jul 13 09:00:50 2018
Errors in file /u01/app/oracle/diag/rdbms/prod/prod/trace/prod_j000_3816.trc (incident=42273):
ORA-00600: internal error code, arguments: [13011], [67896], [8421830], [44], [8421827], [0], [], [], [], [], [], []
Incident details in: /u01/app/oracle/diag/rdbms/prod/prod/incident/incdir_42273/prod_j000_3816_i42273.trc
Fri Jul 13 09:00:53 2018
Trace dumping is performing id=[cdmp_20180713090053]
Fri Jul 13 09:01:39 2018
Sweep [inc][42273]: completed

2.初步分析

从/u01/app/oracle/diag/rdbms/prod/prod/trace/prod_j000_1757.trc文件中没有过多信息:

Incident 42249 created, dump file: /u01/app/oracle/diag/rdbms/prod/prod/incident/incdir_42249/prod_j000_1757_i42249.trc
ORA-00600: internal error code, arguments: [13011], [67896], [8421830], [44], [8421830], [17], [], [], [], [], [], []

从/u01/app/oracle/diag/rdbms/prod/prod/incident/incdir_42249/prod_j000_1757_i42249.trc文件中可以看到Current SQL:

Dump continued from file: /u01/app/oracle/diag/rdbms/prod/prod/trace/prod_j000_1757.trc
ORA-00600: internal error code, arguments: [13011], [67896], [8421830], [44], [8421830], [17], [], [], [], [], [], [] ========= Dump for incident 42249 (ORA 600 [13011]) ======== *** 2018-07-13 02:00:46.871
dbkedDefDump(): Starting incident default dumps (flags=0x2, level=3, mask=0x0)
----- Current SQL Statement for this session (sql_id=11p815z8hkfms) -----
DELETE MGMT_SYSTEM_PERFORMANCE_LOG WHERE TIME < :B2 AND ROWNUM <= :B1

3.匹配MOS

搜索MOS可以找到以下两篇文档:
- ORA-00600 [13011] (文档 ID 1547827.1)
- ORA-600 [13011] "Problem occurred when trying to delete a row" (文档 ID 28184.1)

从文档 ID 1547827.1中可以匹配到现象Current SQL一致:

DELETE MGMT_SYSTEM_PERFORMANCE_LOG WHERE TIME < :B2 AND ROWNUM <= :B1

MOS中给出的解决方案,

SOLUTION

1)As a possible solution, please apply patch set 11.2.0.3

  • OR -

2)Drop and recreate the index(es) on SYSMAN.MGMT_SYSTEM_PERFORMANCE_LOG.

To drop and recreate the indexes:

A. Get the DDL for recreating the indexes:

set heading off

set echo off

set flush off

set pagesize 9999

set linesize 9999

set long 9999

SQL> select dbms_metadata.get_ddl('INDEX', 'MGMT_SYSTEM_PERF_LOG_IDX_01', 'SYSMAN') from dual;

SQL> select dbms_metadata.get_ddl('INDEX', 'MGMT_SYSTEM_PERF_LOG_IDX_02', 'SYSMAN') from dual;

B. Drop the index:

drop index sysman.MGMT_SYSTEM_PERF_LOG_IDX_01;

drop index sysman.MGMT_SYSTEM_PERF_LOG_IDX_02;

C. Recreate the indexes using the output from step 2A.

3)Run validate against the table and its indexes once more:

SQL> analyze table sysman.MGMT_SYSTEM_PERFORMANCE_LOG validate structure cascade;

4.定位解决

这里选择MOS中第二种解决方案重建MGMT_SYSTEM_PERFORMANCE_LOG索引:

--4.1 查看MGMT_SYSTEM_PERFORMANCE_LOG的索引
SQL> select index_name from dba_indexes where table_name = 'MGMT_SYSTEM_PERFORMANCE_LOG'; MGMT_SYSTEM_PERF_LOG_IDX_01
MGMT_SYSTEM_PERF_LOG_IDX_02 --4.2 获取MGMT_SYSTEM_PERFORMANCE_LOG的2个索引的DDL语句
SQL> select dbms_metadata.get_ddl('INDEX', 'MGMT_SYSTEM_PERF_LOG_IDX_01', 'SYSMAN') from dual;
SQL> select dbms_metadata.get_ddl('INDEX', 'MGMT_SYSTEM_PERF_LOG_IDX_02', 'SYSMAN') from dual; --4.3 删除之前的2个索引
SQL> drop index sysman.MGMT_SYSTEM_PERF_LOG_IDX_01;
SQL> drop index sysman.MGMT_SYSTEM_PERF_LOG_IDX_02; --4.4 重新创建2个索引
SQL> CREATE INDEX "SYSMAN"."MGMT_SYSTEM_PERF_LOG_IDX_01" ON "SYSMAN"."MGMT_SYSTEM_PERFORMANCE_LOG" ("JOB_NAME", "TIME", "NAME")
PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS COMPRESS 3
STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
TABLESPACE "SYSAUX"; SQL> CREATE INDEX "SYSMAN"."MGMT_SYSTEM_PERF_LOG_IDX_02" ON "SYSMAN"."MGMT_SYSTEM_PERFORMANCE_LOG" ("TIME")
PCTFREE 10 INITRANS 2 MAXTRANS 255 COMPUTE STATISTICS
STORAGE(INITIAL 65536 NEXT 1048576 MINEXTENTS 1 MAXEXTENTS 2147483645
PCTINCREASE 0 FREELISTS 1 FREELIST GROUPS 1 BUFFER_POOL DEFAULT FLASH_CACHE DEFAULT CELL_FLASH_CACHE DEFAULT)
TABLESPACE "SYSAUX"; --4.5 重新分析表和索引
SQL> analyze table sysman.MGMT_SYSTEM_PERFORMANCE_LOG validate structure cascade; Table analyzed.

至此,完成索引重建,后续定期观察alert日志。

记录一则ORA-600 [13011]错误的更多相关文章

  1. ORA 各种oraclesql错误

    ORA-00001: 违反唯一约束条件 (.) ORA-00017: 请求会话以设置跟踪事件 ORA-00018: 超出最大会话数 ORA-00019: 超出最大会话许可数 ORA-00020: 超出 ...

  2. ASP.NET全局错误处理和异常日志记录以及IIS配置自定义错误页面

    应用场景和使用目的 很多时候,我们在访问页面的时候,由于程序异常.系统崩溃会导致出现黄页.在通常的情况下,黄页对于我们来说,帮助是极大的,因为它可以帮助我们知道问题根源,甚至是哪一行代码出现了错误.但 ...

  3. 记录一个bootstrap惨痛的错误

    记录一个bootstrap的错误,这个错误因为我删除了一个class就导致了页面上显示的错误,这是一个惨痛的教训,特此记录,提醒自己在做前端的修改时,一定要慎之又慎.如果真的要做改动,改完之后也要测一 ...

  4. 记录一则ORA

    应用服务器:Windows Server 2008 R2 Enterprise故障现象:项目侧同事反映应用服务器上的程序连接数据库报错:ORA-12560: TNS: 协议适配器错误 1.故障重现 2 ...

  5. 记录同事的一个bug-ajax-413错误-fullhead

    症状表现为在form下面的textarea里的字符数只有几十个的时候,请求可以成功,但是如果有几百字,则会出现413错误,提示fullhead,我第一反应是cookie体积太小,但是清了缓存还是一样的 ...

  6. 【Android 错误记录】installation error: INSTALL_PARSE_FAILED_NO_CERTIFICATES 错误

    在运行android应用的时候报出了如题的错误: installation error: INSTALL_PARSE_FAILED_NO_CERTIFICATES 原因之一:在src源文件目录下面,有 ...

  7. javaScript 消除错误,并将错误记录在控制台,阻止浏览器错误警告

    当我们使用jquery,和其他各种框架时,有的时候会出现各种错误,  例如jquery文件报错,但又不影响功能,  又不能对jquery做出更改,怎么办呢? window.onerror=functi ...

  8. 记录一个IIS的服务器错误问题的解决方案

    部署一个mvc项目到iis的时候提示有下面这样的错误, 看提示是Microsoft.CodeDom.Providers.DotNetCompilerPlatform,权限问题. 我是第一次遇到,所以只 ...

  9. 编程, 细心永远都不嫌多(记录java连接数据库的一个错误)

    最近在学习Java连接oracle数据库操作, 无意间一个小问题, 浪费了一个下午和半个晚上去找这个错误, 本来可以做更多的事情的, 现将这个错误贴出来, 每次看到, 定将勉励! .......... ...

随机推荐

  1. Node.js编程规范

    摘自:https://github.com/dead-horse/node-style-guide https://github.com/felixge/node-style-guide 2空格缩进 ...

  2. tensorflow的ckpt文件总结

    1.TensorFlow的模型文件 --checkpoint_dir | |--checkpoint | |--MyModel.meta | |--MyModel.data-00000-of-0000 ...

  3. 配置zsh

    .zshrc export ZSH="/root/.oh-my-zsh" ZSH_THEME="robbyrussell" plugins=(git zsh-s ...

  4. flume 学习总结

    flume 总结 flume 总结 下载配置安装 1 下载 2 配置安装 flume 架构 agent 配置 1 source 配置 11 监听网络端口 12 监控文件 2 channel 配置 3 ...

  5. LeetCode 961 N-Repeated Element in Size 2N Array 解题报告

    题目要求 In a array A of size 2N, there are N+1 unique elements, and exactly one of these elements is re ...

  6. sqlserver2014两台不同服务器上数据库同步

    sqlserver2014两台不同服务器上数据库同步   同步了快一个月了,哈哈,因为途中比较麻烦,第一次,遇到烦的地方就停下了,今天终于同步成功了,哈哈,下面我就来介绍一下我实现两台数据库同步的过程 ...

  7. Maven基本介绍及安装

    什么是Maven 是一个跨平台的项目管理工具. 跨平台是指它几乎可以在现有所有流行的操作系统中运行 maven不仅可以构建项目,还可以依赖管理和项目信息管理 Maven解决了什么问题 maven解决了 ...

  8. ELKstack5.6.5

    ELKstack 安装: 一:环境准备: 1.给添加块磁盘50G,相当于模拟raid 2.分别给它们做本机的主机解析 [root@node01 ~ ]#cat  /etc/hosts 127.0.0. ...

  9. 内核atom机制

    内核版本:linux2.6.22.6 硬件平台:JZ2440 驱动源码 atom_ipc_poll_key_int_drv.c : #include <linux/module.h> #i ...

  10. 自定义指令(v-check、v-focus)的方法有哪些?它有哪些钩子函数?还有哪些钩子函数参数?

    全局定义指令:在vue对象的directive方法里面有两个参数,一个是指令名称,另一个是函数.组件内定义指令:directives: 钩子函数:bind(绑定事件触发).inserted(节点插入的 ...