SLES 12: Database Startup Error with ORA-27300 ORA-27301 ORA-27303 While Starting using Srvctl (Doc ID 2340986.1)

APPLIES TO:

Oracle Database - Enterprise Edition - Version 12.1.0.2 and later
Linux x86-64

SYMPTOMS

12.1 RAC database error while starting via srvctl, alert log showing database startup and further goes down

通过 srvctl 启动12.1 RAC数据库时出现错误,alert log显示database startup and further goes down

alert_ORCL2.log inside <oracle base>/diag/rdbms/<db name>/<SID name>/trace:

----------------------------------------------------------------------------------------------------------------

Fri Nov 03 15:06:25 2017
Adjusting the default value of parameter parallel_max_servers  
from 960 to 486 due to the value of parameter processes (600)
Starting ORACLE instance (normal) (OS id: 19684)
.
.
Fri Nov 03 15:10:45 2017
Process startup failed, error stack:
Fri Nov 03 15:10:45 2017
Errors in file /u01/app/oracle/diag/rdbms/ORCL/ORCL2/trace/ORCL2_psp0_19706.trc:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3
Fri Nov 03 15:10:46 2017
Shutting down instance (abort)
License high water mark = 2
Fri Nov 03 15:10:46 2017
USER (ospid: 22067): terminating the instance
Fri Nov 03 15:10:47 2017
Instance terminated by USER, pid = 22067
Fri Nov 03 15:10:47 2017
Instance shutdown complete
Errors in file //u01/app/oracle/diag/rdbms/ORCL/ORCL2/trace/ORCL2_mmon_19853.trc (incident=14681):
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
Incident details in: /u01/app/oracle/diag/rdbms/ORCL/ORCL2/incident/incdir_14681/ORCL2_mmon_19853_i14681.trc
Use ADRCI or Support Workbench to package the incident.
See Note 411.1 at My Oracle Support for error and packaging details.

ORCL2_psp0_19706.trc: 

-------------------------------------------------------------------------------------------------------------------

*** 2017-11-03 15:10:44.989
Process startup failed, error stack:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3

*** 2017-11-03 15:10:45.993
Process startup failed, error stack:
ORA-27300: OS system dependent operation:fork failed with status: 11
ORA-27301: OS failure message: Resource temporarily unavailable
ORA-27302: failure occurred at: skgpspawn3

 ORCL2_mmon_19853.trc:

-------------------------------------------------------------------------------------------------------------
*** 2017-11-03 15:09:57.908
***KELR Apply Log: unable to schedule MMON Slave, error 3

*** 2017-11-03 15:10:47.847
Incident 14681 created, dump file: /u01/app/oracle/diag/rdbms/ORCL/ORCL2/incident/incdir_14681/ORCL2_mmon_19853_i14681.trc
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1

KEBM: MMON action policy violation. 'PQ: Adjust Slave Pool' viol=0; err=600
error 0 detected in background process
kgxgnsdr: clssgsshdereg: warning: return status 26 (-558242808 )
OPIRIP: Uncaught error 447. Error stack:
ORA-00447: fatal error in background process
ORA-00600: internal error code, arguments: [KSLGES_3], [], [], [], [], [], [], [], [], [], [], []
ORA-27300: OS system dependent operation:semop failed with status: 43
ORA-27301: OS failure message: Identifier removed
ORA-27302: failure occurred at: sskgpwwait1
kgxgnsdr: clssgsshdereg: warning: return status 26 (-813323584 )

Following error reported in OS logs

messages:

2017-11-03T15:10:11.690393+01:00 rachost1 Oracle Audit[7888]: LENGTH : '200' ACTION :[52] 'ALTER DATABASE MOUNT /* db agent *//* {2:39656:2} */' DATABASE USER:[1] '/' PRIVILEGE :[6] 'SYSDBA' CLIENT USER:[6] 'oracle' CLIENT TERMINAL:[0] '' STATUS:[1] '0' DBID:[10] '2949004148'
2017-11-03T15:10:47.700094+01:00 rachost1 kernel: [ 173.788859] cgroup: fork rejected by pids controller in /system.slice/ohasd.service <<<<<<<<<<<<<<<<<<<<<

Database able to startup with sqlplus.   能够使用sqlplus启动的数据库

CAUSE

This is due the PIDs cgroup controller introduced with SUSE12.  这是由于 SUSE12 附带的 PIDs cgroup 控制器

From the "Support for PIDs cgroup Controller" section on https://www.suse.com/releasenotes/x86_64/SUSE-SLES/12-SP2/#fate-320358

"To control the default TasksMax= setting for services and scopes running on the system, use the system.conf setting DefaultTasksMax=. This setting defaults to 512, which means services that are not explicitly configured otherwise will only be able to create 512 processes or threads at maximum.

要控制TasksMax的默认值,请使用system.conf设置DefaultTasksMax。此设置默认为512,这意味着未明确配置的服务最多只能创建512个进程或线程。

For thread- or process-heavy services, you may need to set a higher TasksMax value. In such cases, set TasksMax directly in the specific unit files. Either choose a numeric value or even infinity."

对于大量使用线程或进程的服务,您可能需要设置更高的TasksMax值。 在这种情况下,请直接在特定 unit files 中设置 TasksMax。 选择一个数字值,甚至选择infinity。”

From SLES12 onwards, systemd is used instead of initd and the OHASD server is only allowed to open a maximum of 512 tasks.

从SLES12开始,使用systemd代替initd,并且只允许OHASD服务器打开最多512个任务。

SOLUTION

Configure the value of DefaultTasksMax to 65535 in the file /etc/systemd/system.conf or or set the TasksMax value properly for the ohasd systemd service.

在文件 /etc/systemd/system.conf 中将 DefaultTasksMax 的值配置为65535,或者为 ohasd systemd 服务正确设置TasksMax值。

For example:

# cat /etc/systemd/system/ohasd.service.d/lunar.conf
[Service]
TasksMax=16384 <<<<<<<<<<< # systemctl status ohasd
● ohasd.service - LSB: Start and Stop Oracle High Availability Service
Loaded: loaded (/etc/init.d/ohasd; bad; vendor preset: disabled)
Drop-In: /etc/systemd/system/ohasd.service.d
└─lunar.conf
Active: active (exited) since Mon 2017-11-13 14:29:23 CET; 3h 5min ago
Docs: man:systemd-sysv-generator(8)
Process: 4876 ExecStart=/etc/init.d/ohasd start (code=exited, status=0/SUCCESS)
Tasks: 612 (limit: 16384) <<<<<<=================== here was the limit with 512

SLES 12: Database Startup Error with ORA-27300 ORA-27301 ORA-27303 While Starting using Srvctl (Doc ID 2340986.1)的更多相关文章

  1. Orace 12.2 ORA-12012: error on auto execute of job "SYS"."ORA$AT_OS_OPT_SY_21"

    一个测试环境的12.2.0.1数据库后台alert不断报出以下错误信息: Errors in file /d12/app/oracle/diag/rdbms/test/test/trace/test_ ...

  2. linux安装phpstorm出现 Startup Error: Unable to detect graphics environment

    在centos6.2下在安装phpstorm下遇到 Startup Error: Unable to detect graphics environment 其中jdk安装的版本是 1.7.0_79  ...

  3. android.database.sqlite.SQLiteCantOpenDatabaseException: unknown error(Sqlite code 14): Could not open database,(OS error - 13:Permission denied)

    07-24 15:03:14.490 6291-6291/com.tongyan.nanjing.subway E/SQLiteDatabase: Failed to open database '/ ...

  4. wsse:InvalidSecurity Error When Testing FND_PROFILE Web Service in Oracle Applications R 12.1.2 from SOAP UI (Doc ID 1314946.1)

    wsse:InvalidSecurity Error When Testing FND_PROFILE Web Service in Oracle Applications R 12.1.2 from ...

  5. [Done]com.aerospike.client.AerospikeException: Error Code 12: Bin type error

    今天遇到了一个问题:com.aerospike.client.AerospikeException: Error Code 12: Bin type error 异常栈: 网上找了一些资料:https ...

  6. Start Failed, Internal error: recovering IDE to the working state after the critical startup error

    Start Failed, Internal error: recovering IDE to the working state after the critical startup error F ...

  7. 利用mysql数据库中的TMD表修复“is marked as crashed and last (automatic?) repair failed”的错误 Database query error

    ===========================测试成功============================================= 1.页面出现错误:Database query ...

  8. ORA-4031 During Startup Nomount using RMAN without parameter file (PFILE) (Doc ID 1176443.1)

    ORA-4031 During Startup Nomount using RMAN without parameter file (PFILE) (Doc ID 1176443.1) APPLIES ...

  9. TNS-12535/12606 and ORA-3136 on Connection to Database (Doc ID 2313573.1)

    今天遇到一问题 telnet 都是通的,但是两台数据库服务器还是无法 sqlplus 连接 ,最后发现 两台服务器的 mtu 值不同,其中一台为 1500 一台为9000, 以前只是认为 telnet ...

随机推荐

  1. BZOJ2809 dispatching(左偏树)

    在一个忍者的帮派里,一些忍者们被选中派遣给顾客,然后依据自己的工作获取报偿.在这个帮派里,有一名忍者被称之为 Master.除了 Master以外,每名忍者都有且仅有一个上级.为保密,同时增强忍者们的 ...

  2. linux mysql,tomcat与java的安装

    先将服务器的安全组设置为 把所有端口或者所需要的端口开放 然后测试远程访问 ssh 用户@ip                       登录 输入密码 service iptables stop  ...

  3. spring源码学习之设计模式(1)单例模式

    众所周知,单例模式分为饿汉式和懒汉式,昨天在看了<spring5核心原理与30个类手写实战>之后才知道饿汉式有很多种写法,分别适用于不同场景,避免反射,线程不安全问题.下面就各种场景.采用 ...

  4. Docker私有仓库搭建与界面化管理

    一.关于Registry 官方的Docker hub是一个用于管理公共镜像的好地方,我们可以在上面找到我们想要的镜像,也可以把我们自己的镜像推送上去. 但是有时候我们的使用场景需要我们拥有一个私有的镜 ...

  5. 【Activiti】使用学习

    [Activiti]使用学习 转载: ================================================== 1.下载安装 2.清空表 3.开启sql打印 4. 5. = ...

  6. 【spring boot】配置信息

    ======================================================================== 1.feign 超时配置 2.上传文件大小控制 3.J ...

  7. python学习-class封装

    # 封装 类=属性+行为 抽像 -class StudentV2: # 类属性 所有的实例可以共享 .不属于任何实例的特性. is_people = True # 类方法 1.装饰器.2.参数是cls ...

  8. jq触发oninput事件

    之前一直在用jq的change()方法来处理输入框的值变化事件,以及触发输入框的变化事件. 后来发现change()方法有个弊端,change事件的发生条件是:输入框的值value发生变化,并且输入框 ...

  9. webpack学习3.1由浅入深-打包JS

    打包JS Step1:在新建的文件夹下新建一个sum.js文件,一个app.js文件 sum.js //es module export default function(a,b){ return a ...

  10. cmake常用命令总结

    最近研究了下cmake,总结了一些常用命令,方便以后快速查找. project(projectname [CXX] [C] [Java]): 设置工程名. set(VAR [VALUE] [CACHE ...