中断ORACLE数据库关闭进程导致错误案例

昨晚下班的时候，我准备关闭本机的虚拟机上的ORACLE数据库后准备下班，但是由于我SecureCRT开了多个窗口，结果一不小心，疏忽之下在一个生产服务器上执行了shutdown immediate命令，大概过了6到7秒，发现该命令还没有响应，我才发现我这个命令执行错了服务器。一惊之下，想都没有想直接CTRL+C想中断这个操作。 如下所示：

SQL> shutdown immeidate;

SP2-0717: illegal SHUTDOWN option

SQL> shutdown immediate;

^C^C^C^C^C

ORA-01013: user requested cancel of current operation

于是我在另外一个窗口里面查看了一下告警日志文件，发现数据库已经关闭了一些进程。大体情况如下所示

tail -40f alert_SCM2.log

Wed Aug 6 17:54:37 2014

ARCH shutting down

ARC8: Archival stopped

Wed Aug 6 17:54:42 2014

ARCH shutting down

ARC7: Archival stopped

Wed Aug 6 17:54:47 2014

ARCH shutting down

ARC6: Archival stopped

Wed Aug 6 17:54:52 2014

ARCH shutting down

ARC5: Archival stopped

Wed Aug 6 17:54:57 2014

ARCH shutting down

ARC4: Archival stopped

Wed Aug 6 17:55:05 2014

CLOSE: Error 1013 during database close

Wed Aug 6 17:55:05 2014

SMON: enabling cache recovery

SMON: enabling tx recovery

Wed Aug 6 17:55:05 2014

ORA-1013 signalled during: ALTER DATABASE CLOSE NORMAL...

Wed Aug 6 17:55:07 2014

ARCH shutting down

ARC2: Archival stopped

Wed Aug 6 17:55:12 2014

ARCH shutting down

ARC1: Archival stopped

Wed Aug 6 17:55:17 2014

ARC3: Becoming the heartbeat ARCH

ARC3: Archiving disabled

ARCH shutting down

ARC3: Archival stopped

Wed Aug 6 17:55:17 2014

ARCH shutting down

Wed Aug 6 17:55:17 2014

ARC0: Archival stopped

Wed Aug 6 17:55:18 2014

Thread 1 closed at log sequence 97562

Successful close of redo thread 1

^C

于是立马查看数据库的状态，看看是否正常，结果如下所示，出现了ORA-00604、ORA-00376、ORA-01110等错误。

SQL> SQL> SQL> select status from v$instance;

select status from v$instance

*

ERROR at line 1:

ORA-00604: error occurred at recursive SQL level 2

ORA-00376: file 1 cannot be read at this time

ORA-01110: data file 1: '/u01/oradata/SCM2/system01.dbf'

一惊之下，立马退出了会话，重新登录后（当时不怎么冷静，慌忙之下已经不谈定了）

sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Wed Aug 6 17:57:11 2014

Copyright (c) 1982, 2007, Oracle. All Rights Reserved.

Connected.

SQL> select status from v$instacne;

select status from v$instacne

*

ERROR at line 1:

ORA-01012: not logged on

SQL> shutdown immdeiate;

SP2-0717: illegal SHUTDOWN option

SQL> exit

Disconnected

SQL> shutdwon immeidiate;

SP2-0734: unknown command beginning "shutdwon i..." - rest of line ignored.

SQL> shutdown immediate;

ORA-24324: service handle not initialized

ORA-24323: value not allowed

ORA-01089: immediate shutdown in progress - no operations are permitted

此时告警日志里面有大量的这类错误。

无奈之下，我只能使用shutdown abort命令了，这时候我反而冷静下来。但是居然报ORA-01031: insufficient privileges 错误，立马退出然后重新登录后，将数据库关闭然后重新启动。

SQL> shutdown abort

ORA-01031: insufficient privileges

sqlplus / as sysdba

SQL*Plus: Release 10.2.0.4.0 - Production on Wed Aug 6 18:15:00 2014

Copyright (c) 1982, 2007, Oracle. All Rights Reserved.

Connected.

SQL> shutdown abort

ORACLE instance shut down.

SQL> startup

ORACLE instance started.

Total System Global Area 7516192768 bytes

Fixed Size 2095640 bytes

Variable Size 5167384040 bytes

Database Buffers 2298478592 bytes

Redo Buffers 48234496 bytes

Database mounted.

Database opened.

SQL> exit

重新启动后，监控告警日志，发现没有异常出现，逐个检查后发现没有什么问题，一颗悬着的心才淡定下来。不过还是要总结一下：这是一次低级失误，也是印象比较深的一次失误，我之所以要记录下来，一来这也是一个案例，二来要自己谨记于心。整个过程中，发现自己一直不冷静、不谈定。其实本来已经shutdown了数据库，那应该先冷静分析一下，到底是等数据库关闭后重新启动，还是中断这个进程。 本身ORACLE数据库已经关闭了一些进程，如果此时中断shutdown 进程，明显是个不明智的决定。错误的决策导致后面一系列问题的出现，典型的修为不够！ 老大给我的邮件叫我下次应该 relax, calm down and be careful 。 谨记于心。