现象:
RAC环境,数据文件状态变为recover,查看alert日志有如下报错:

Wed Jun 26 02:31:03 2013

Thread 1 advanced to log sequence 33187

 Current log# 1 seq# 33187 mem# 0: +TJDISK/tj/onlinelog/group_1.257.757797483

Wed Jun 26 10:10:03 2013

Errors in file /opt/app/diag/rdbms/tj/tj1/trace/tj1_dbw0_6145.trc:

ORA-01148: cannot refresh file size for datafile 17

ORA-01110: data file 17: '+TJDISK/tj/datafile/ntj_index03.301.757894747'

ORA-01031: insufficient privileges

Automatic datafile offline due to media error on

file 17: +TJDISK/tj/datafile/ntj_index03.301.757894747

Unexpected communication failure with ASM instance:

 error 1031 (ORA-01031: insufficient privileges

)

Wed

分析:

1.查看所有节点的messages系统日志、asm日志均没有出现错误信息。

2.查看DG的raw权限,也没有异常。

/dev/raw/raw6

/dev/raw/raw7

3.该Datafile为autoextend模式。

SQL> select file_name,autoextensible from dba_data_files where file_name like '+TJDISK/tj/datafile/ntj_index03.301.757894747';

FILE_NAME                                                        

--------------------------------------------------------------------------------

AUT

---

+TJDISK/tj/datafile/ntj_index03.301.757894747

YES

最后查了下Metalink,怀疑是命中了Oracle的一个BUG:Bug 16734525或Bug 9357097(Bug 16734525 is the duplicate of Bug 9357097)。

Bug 16734525 : ORA-1148: CANNOT REFRESH FILE SIZE FOR DATAFILE

Hdr: 16734525 10.2.0.5 RDBMS 11.1.0.7 ASM PRODID-5 PORTID-23 ORA-1148 9357097

Abstract: ORA-1148: CANNOT REFRESH FILE SIZE FOR DATAFILE

*** 04/27/13 02:21 am ***

PROBLEM:--------

Fri Apr 26 11:31:28 EDT 2013

Redo Shipping Client Connected as PUBLIC--

Connected User is ValidRedo Shipping Client Connected as PUBLIC--

Connected User is Valid

Fri Apr 26 11:44:55 EDT 2013

Errors in file /home/oracle/admin/ctopprul/bdump/ctopprul1_dbw0_20315.trc:

ORA-1148: cannot refresh file size for datafile 340

ORA-1110: data file 340: '+DATA/ctopprul_rdc/datafile/wires_data.1968.789654733'

ORA-1031: insufficient privilegesFri

Apr 26 11:44:55 EDT 2013

Automatic datafile offline due to media error onfile 340: +DATA/ctopprul_rdc/datafile/wires_data.1968.789654733

Fri Apr 26 11:44:59 EDT 2013

Unexpected communication failure with ASM instance: error 1031

ORA-1031: insufficient privileges)

NOTE: ASMB process state dumped to trace file /home/oracle/admin/ctopprul/bdump/ctopprul1_dbw0_20315.trc

NOTE: force a map free for map id 345 DIAGNOSTIC

ANALYSIS:--------------------

1. Matches the bug 9357097: SMALL BEEHIVE: FAILURE TO REFRESH FILE SIZE DUE TO SPACE OFFLINES DATAFILE

Need to confirm from DEV as audit file space issues were not there

2. Not using role separation and oracle executable is with correct permissions

3. CT is not sure if dbv or rman validate was run on the problematic datafiledue to media error

ORA-1148: cannot refresh file size for datafile 340

ORA-1110: data file 340: '+DATA/ctopprul_rdc/datafile/wires_data.1968.789654733'

ORA-1031: insufficient privileges

Fri Apr 26 11:44:55 EDT 2013

Automatic datafile offline due to media error on  >>>>>>>>>>>> Media error

4. Ulimit was showing nofiles of low value         Customer Visible         

[Open Update screen]     

[Double Click on Activity Text to enable Save operation]         

[Audit]Hi team,   Oracle:-----------  -

Checked if there was any space issues on the server and nothing foundas the above bug is hit when audit files are not able to write-OS watcher logs shows normal

WORKAROUND:-----------

RELATED BUGS:-------------

REPRODUCIBILITY:----------------

TEST CASE:----------

STACK TRACE:------------

SUPPORTING INFORMATION:-----------------------

Uploaded all the relevant info to the bug 24 HOUR CONTACT INFORMATION FOR P1

BUGS:----------------------------------------

DIAL-IN INFORMATION:--------------------

IMPACT DATE:------------

Bug 9357097  ORA-1148 Failure to refresh file size offlines datafile producing ORA-372 ORA-376

Symptoms:

Related To:

1 Error May Occur

2 ORA-1148 / ORA-372 / ORA-376

Range of versions believed to be affected   <-- 12.1下的版本都有可能命中

Versions BELOW 12.1    

Versions confirmed as being affected

?11.2.0.1

?11.1.0.7

?10.2.0.5

?10.2.0.4

Platforms affected

 Generic (all / most platforms affected)

Fixed:

This issue is fixed in                  <-- 12.1.0.1 11.2.0.2中已修复

 ?12.1.0.1 (Base Release)

?11.2.0.2 (Server Patch Set)

DBWR can offline the datafile with message "Automatic datafile offline due to media error"

if file size refresh fails with error ORA-1148.

As the file is offline, subsequent attempts to read the affected file produce

error ORA-372 or ORA-376 requiring media recovery.

解决方法:

临时解决方法是将手动将文件online,

Oracle并没有提供专门的补丁,需要升级到对应版本才能彻底解决(11.2.0.2)。

诊断时在ASM实例中执行了以下脚本。

SPOOL ASM_FIRST<instance#>.HTML

SET MARKUP HTML ON

set echo on

set pagesize 200

alter session set nls_date_format='DD-MON-YYYY HH24:MI:SS';

select 'THIS ASM REPORT WAS GENERATED AT: ==)> ' , sysdate " " from dual;

select 'HOSTNAME ASSOCIATED WITH THIS ASM INSTANCE: ==)> ' , MACHINE " " from v$session where program like '%SMON%';

select * from v$asm_diskgroup;

SELECT * FROM V$ASM_DISK ORDER BY GROUP_NUMBER,DISK_NUMBER;

SELECT * FROM V$ASM_CLIENT;

select * from V$ASM_ATTRIBUTE;

select * from gv$asm_operation;

select * from v$version;

show parameter asm

show parameter cluster

show parameter instance_type

show parameter instance_name

show parameter spfile

show sga

spool off

exit

又遇BUG-ORA-01148:数据文件忽然变为recover状态的更多相关文章

  1. ORACLE ORA-01157: 无法标识/锁定数据文件

    create undo tablespace MOZI datafile 'E:\oracle\product\10.2.0\oradata\orcl\MOZI.DBF' size 2048M ext ...

  2. 在ASM中移动数据文件

    实验目的:在ASM存储环境下,要删除一个磁盘组,从而将磁盘组中的数据文件移动到另外一个磁盘组中. 查看数据文件存放的位置: SQL> select file#,name from v$dataf ...

  3. 【基础】Oracle 表空间和数据文件

    多个表空间的优势:1.能够将数据字典与用户数据分离出来,避免由于字典对象和用户对象保存在同一个数据文件中而产生的I/O冲突2.能够将回退数据与用户数据分离出来,避免由于硬盘损坏而导致永久性的数据丢失3 ...

  4. [20171225]没有备份数据文件的恢复.txt

    [20171225]没有备份数据文件的恢复.txt --//别人问的问题,增加了数据文件没有备份,如何恢复,实际上很简单,因为当前控制文件有记录建立时间只要从建立数据文件开始的--//归档日志都存在恢 ...

  5. Oracle数据文件迁移到裸设备

    本文主要描述如何将Oracle表空间的文件系统形式的数据文件迁移到LV裸设备上. 前提条件 1.oracle运行正常. 2.已使用LVM命令规划好LV文件.如/dev/vgoracle/lvdatat ...

  6. Oracle数据文件转移操作

    由于oracle表空间数据文件规划问题导致当前数据文件所在文件系统空间不足,当其他文件系统空间充足情况下,可将数据文件移动到空间充足的文件系统下.本文主要描述Oracle表空间数据文件移动的操作步骤. ...

  7. Oracle 11g Rac 用rman实现把本地数据文件迁移到ASM共享存储中

    在Oracle Rac环境中,数据文件都是要存放在ASM共享存储上的,这样两个节点才能同时访问.而当你在某一节点下把数据文件创建在本地磁盘的时候,那么在另一节点上要访问该数据文件的时候就会报错,因为找 ...

  8. Oracle 无备份情况下的恢复--控制文件/数据文件

    13.3无备份恢复控制文件 没有备份恢复控制文件其实就是在nomount状态,create control创建一个新的控制文件. dba必须知道4个信息才能正确的创建:数据库名.在线日志路径及其大小. ...

  9. oracle 普通数据文件备份与恢复

    普通数据文件指:非system表空间.undo_tablespace表空间.临时表空间和只读表空间的数据文件.它们损坏导致用户数据不能访问,不会导致db自身异常.实例崩溃.数据库不恢复就无法启动的情况 ...

随机推荐

  1. asp.net正则表达式过滤标签和数据提取

    无论什么语言,正则表达式的处理方法都是非常灵活.高效的,尤其是对某些字符串的抓取.过滤方面,更显其优势. 正则表达式的写法通常比较简单,几行短代码便能轻松完成看似很复杂的事情,更值得称赞的是,它的执行 ...

  2. iOS应用的crash日志的分析基础

        Outline如何获得crash日志如何解析crash日志如何分析crash日志     1. iOS策略相关     2. 常见错误标识     3. 代码bug 一.如何获得crash日志 ...

  3. django --------------------- [必要操作]

    基本models 命令: python manage.py validate (验证模型有效性, 记得配置 settings.py - INSTALLED_APPS) python manage.py ...

  4. h264检测是I帧还是P帧,解决录像第一帧有马赛克问题

    最近使用h264码流数据进行录像,但是录出来的第一帧有马赛克,究其原因是录像的第一帧不是关键帧,所以需要录像是需要判断第一帧是否是关键帧,方法有两种,第一种是原码流的基础上查找,第二种是将原码流传递给 ...

  5. metadata lock

    1 锁等待的例子 session 1: 执行查询但不提交 mysql> begin; mysql> select * from test where c2 = '1'; session 2 ...

  6. java web 学习八(HttpServletResponse对象2)

    一.HttpServletResponse常见应用——生成验证码 1.1.生成随机图片用作验证码 生成图片主要用到了一个BufferedImage类,

  7. Google Maps API显示地图的小示例

    来源:http://www.ido321.com/1089.html 效果(新版Firefox中测试): 代码: <!DOCTYPE> <html> <head> ...

  8. CentOS下编译安装hping3

    安装hping之前,先装上libpcap-dev和tcl-dev 1.获取源码包 wget http://www.hping.org/hping3-20051105.tar.gz 2.解压,得到 hp ...

  9. Hibernate逆向工程

    MySQL Administrator 创建表   MyEclipse Database Explorer视图: 1. New  2 .Driver template: MySQL Connector ...

  10. [HIve - LanguageManual] Join Optimization (不懂)

    Join Optimization Join Optimization Improvements to the Hive Optimizer Star Join Optimization Star S ...