诊断：MRP0: Background Media Recovery process shutdown with error ORA-19909

oracle12c data guard，从库无法应用日志，检查alert日至发现

2019-10-21T14:55:40.087819+08:00
MRP0: Background Media Recovery process shutdown (DATA)

查看mrp的trace发现：

Trace file /oracle/diag/rdbms/pdDATA/DATA/trace/DATA_mrp0_37206.trc
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production
Build label: RDBMS_12.2.0.1.0_LINUX.X64_170125
ORACLE_HOME: /oracle/product/12c/db
System name: Linux
Node name: pdfmdm002
Release: 2.6.32-431.el6.x86_64
Version: #1 SMP Sun Nov 10 22:19:54 EST 2013
Machine: x86_64
Instance name: DATA
Redo thread mounted by this instance: 1
Oracle process number: 47
Unix process pid: 37206, image: oracle@pdfmdm002 (MRP0)

*** 2019-10-21T14:39:21.697436+08:00
*** SESSION ID:(1343.12850) 2019-10-21T14:39:21.697470+08:00
*** CLIENT ID:() 2019-10-21T14:39:21.697477+08:00
*** SERVICE NAME:() 2019-10-21T14:39:21.697482+08:00
*** MODULE NAME:() 2019-10-21T14:39:21.697488+08:00
*** ACTION NAME:() 2019-10-21T14:39:21.697493+08:00
*** CLIENT DRIVER:() 2019-10-21T14:39:21.697498+08:00

*** 2019-10-21 14:39:21.696084 5634 krsh.c
MRP0: Background Managed Standby Recovery process started

*** 2019-10-21T14:39:26.702935+08:00
Managed Recovery: Initialization posted.

*** 2019-10-21T14:39:27.596076+08:00
Successfully allocated 8 recovery slaves
Parallel Media Recovery started with 8 slaves
Managed Recovery: Active posted.
LogMerger process exited abnormally.

*** 2019-10-21T14:55:18.881022+08:00
Slave# 8: PR02 exited
Slave# 7: PR04 exited
Slave# 6: PR01 exited
Slave# 5: PR06 exited
Slave# 4: PR07 exited
Slave# 3: PR05 exited
Slave# 2: PR03 exited
Slave# 1: PR00 exited
ksvp2penabled: ep->flg = 0, rpr->slv_flg = 0
ksvp2penabled: ep = 0x7f178ffce408, rpr = 0x463f7f8b8
Managed Recovery: Initialization posted.

*** 2019-10-21T14:55:20.082337+08:00
ksvp2penabled: ep->flg = 0, rpr->slv_flg = 0
ksvp2penabled: ep = 0x7f178ffce408, rpr = 0x463f7f8b8
ORA-19909: datafile 1 belongs to an orphan incarnation
ORA-01110: data file 1: '/oradata/DATA/datafile/system.257.987760395'

*** 2019-10-21T14:55:40.087576+08:00
Managed standby recovery cannot handle orphaned datafiles
*** 2019-10-21 14:55:40.087837 5634 krsh.c
MRP0: Background Media Recovery process shutdown
Managed Recovery: Not Active posted.

检查从库的incarnation

$ rman target /

Recovery Manager: Release 12.2.0.1.0 - Production on Mon Oct 21 15:22:46 2019

Copyright (c) 1982, 2017, Oracle and/or its affiliates. All rights reserved.

connected to target database: DATA (DBID=1936743762, not open)

RMAN> list incarnation of database;

using target database control file instead of recovery catalog

List of Database Incarnations
DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1 1 DATA 1936743762 PARENT 1 26-JAN-17
2 2 DATA 1936743762 PARENT 1408558 25-SEP-18
3 3 DATA 1936743762 ORPHAN 4346574446 23-JUL-19
4 4 DATA 1936743762 CURRENT 5041364391 10-SEP-19

再对比主库：

RMAN> list incarnation of database;

List of Database Incarnations
DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1 1 MUDATA 1936743762 PARENT 1 2017-01-26 13:52:29
2 2 MUDATA 1936743762 CURRENT 1408558 2018-09-25 09:55:33

主库使用的是1408558，将从库重置到1408558即可

备库操作：

RMAN> reset database to incarnation 2;

database reset to incarnation 2

RMAN> list incarnation of database;

List of Database Incarnations
DB Key Inc Key DB Name DB ID STATUS Reset SCN Reset Time
------- ------- -------- ---------------- --- ---------- ----------
1 1 DATA 1936743762 PARENT 1 26-JAN-17
2 2 DATA 1936743762 CURRENT 1408558 25-SEP-18
3 3 DATA 1936743762 ORPHAN 4346574446 23-JUL-19
4 4 DATA 1936743762 ORPHAN 5041364391 10-SEP-19

RMAN>