问题解决：TNS-12543: TNS:destination host unreachable

环境：

11.2.0.3 ADG (db11g\db11gadg\db11gcas)

在自己先前克隆后的环境互相tnsping报错。

tnsping 本机ok，tnsping其他机器均报错：

[oracle@db11g ~]$ tnsping jingyu

TNS Ping Utility for Linux: Version 11.2.0.3.0 - Production on 13-MAY-2023 08:09:11

Copyright (c) 1997, 2011, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.18)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = jingyu)))
OK (0 msec)
[oracle@db11g ~]$ tnsping jyadg

TNS Ping Utility for Linux: Version 11.2.0.3.0 - Production on 13-MAY-2023 08:09:14

Copyright (c) 1997, 2011, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.19)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = jyadg)))
TNS-12543: TNS:destination host unreachable
[oracle@db11g ~]$ tnsping jycas

TNS Ping Utility for Linux: Version 11.2.0.3.0 - Production on 13-MAY-2023 08:09:17

Copyright (c) 1997, 2011, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/product/11.2.0/dbhome_1/network/admin/sqlnet.ora

Used TNSNAMES adapter to resolve the alias
Attempting to contact (DESCRIPTION = (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.20)(PORT = 1521)) (CONNECT_DATA = (SERVER = DEDICATED) (SERVICE_NAME = jycas)))
TNS-12543: TNS:destination host unreachable

• 1.检查防火墙和SElinux状态
• 2.确认克隆的网卡是否正常
• 3.网络层面排查1521端口

1.检查防火墙和SElinux状态

第一反应就是防火墙或SELinux导致，初步检查发现：防火墙未运行，SELinux也禁用掉。

[root@db11g ~]# service iptables status
Redirecting to /bin/systemctl status iptables.service
Unit iptables.service could not be found.

[root@db11g ~]# getenforce
Permissive
[root@db11g ~]# cat /etc/sysconfig/selinux 

# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=disabled
# SELINUXTYPE= can take one of three values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted

配置文件tnsnames.ora同步到3个环境，保持一致：

# tnsnames.ora Network Configuration File: /u01/app/oracle/product/11.2.0/dbhome_1/network/admin/tnsnames.ora
# Generated by Oracle configuration tools.

JINGYU =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.18)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = jingyu)
    )
  )

JYADG =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.19)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = jyadg)
    )
  )

JYCAS =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.1.20)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = jycas)
    )
  )

2.确认克隆的网卡是否正常

主要怀疑克隆是否MAC地址之类做了变化，期望是不一样的MAC，实际也是，在宿主机看arp记录的和目前的都匹配：

[oracle@bogon ~]$ arp -a|grep 1.18
bogon (192.168.1.18) at 52:54:00:6f:92:54 [ether] on br0
[oracle@bogon ~]$ arp -a|grep 1.19
localhost (192.168.1.19) at 52:54:00:d0:49:1b [ether] on br0
[oracle@bogon ~]$ arp -a|grep 1.20
localhost (192.168.1.20) at 52:54:00:71:f1:9a [ether] on br0

可现在就是网络IP可以ping通，但是tnsping说啥不通：

[oracle@bogon ~]$ ping 192.168.1.18
PING 192.168.1.18 (192.168.1.18) 56(84) bytes of data.
64 bytes from 192.168.1.18: icmp_seq=1 ttl=64 time=0.525 ms
64 bytes from 192.168.1.18: icmp_seq=2 ttl=64 time=0.312 ms
^C
--- 192.168.1.18 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1024ms
rtt min/avg/max/mdev = 0.312/0.418/0.525/0.108 ms
[oracle@bogon ~]$ tnsping 192.168.1.18

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 13-MAY-2023 08:39:50

Copyright (c) 1997, 2022, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/product/19.3.0/db_1/network/admin/sqlnet.ora

Used HOSTNAME adapter to resolve the alias
Attempting to contact (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.18)(PORT=1521)))
TNS-12543: TNS:destination host unreachable

3.网络层面排查1521端口

以db11g环境为例说明：

[oracle@db11g ~]$ netstat -ano|grep 1521
tcp6       0      0 :::1521                 :::*                    LISTEN      off (0.00/0/0)
unix  2      [ ACC ]     STREAM     LISTENING     26303    /var/tmp/.oracle/sEXTPROC1521

使用telnet工具定位：

[root@bogon ~]# telnet 192.168.1.18 1521
Trying 192.168.1.18...
telnet: connect to address 192.168.1.18: No route to host

而正常的环境（其他可以tnsping的正常虚机）应该类似这样：

[root@bogon ~]# telnet 192.168.1.11 1521
Trying 192.168.1.11...
Connected to 192.168.1.11.
Escape character is '^]'.

看来的确是网络问题了。

说没有路由可达主机，直接traceroute对比看下有何区别：

# 正常的虚机：
[root@bogon ~]# traceroute 192.168.1.11
traceroute to 192.168.1.11 (192.168.1.11), 30 hops max, 60 byte packets
 1  bogon (192.168.1.11)  0.556 ms  0.507 ms  0.486 ms

# 不正常的虚机：
[root@bogon ~]# traceroute 192.168.1.18
traceroute to 192.168.1.18 (192.168.1.18), 30 hops max, 60 byte packets
 1  bogon (192.168.1.18)  0.516 ms !X  0.450 ms !X  0.404 ms !X

[root@bogon ~]# traceroute 192.168.1.19
traceroute to 192.168.1.19 (192.168.1.19), 30 hops max, 60 byte packets
 1  bogon (192.168.1.19)  0.546 ms !X  0.491 ms !X  0.455 ms !X

[root@bogon ~]# traceroute 192.168.1.20
traceroute to 192.168.1.20 (192.168.1.20), 30 hops max, 60 byte packets
 1  localhost (192.168.1.20)  0.730 ms !X  0.680 ms !X  0.648 ms !X

又回到防火墙。。之前检查命令搞错了，来吧，赶紧正常关闭防火墙：

[root@db11g ~]# systemctl status firewalld
● firewalld.service - firewalld - dynamic firewall daemon
   Loaded: loaded (/usr/lib/systemd/system/firewalld.service; enabled; vendor preset: enabled)
   Active: active (running) since Sat 2023-05-13 08:38:13 CST; 23min ago
     Docs: man:firewalld(1)
 Main PID: 850 (firewalld)
   CGroup: /system.slice/firewalld.service
           └─850 /usr/bin/python2 -Es /usr/sbin/firewalld --nofork --nopid

May 13 08:38:12 db11g systemd[1]: Starting firewalld - dynamic firewall dae.....
May 13 08:38:13 db11g systemd[1]: Started firewalld - dynamic firewall daemon.
May 13 08:38:13 db11g firewalld[850]: WARNING: AllowZoneDrifting is enabled...w.
Hint: Some lines were ellipsized, use -l to show in full.
[root@db11g ~]# systemctl stop firewalld

一切恢复正常：

[root@bogon ~]# traceroute 192.168.1.18
traceroute to 192.168.1.18 (192.168.1.18), 30 hops max, 60 byte packets
 1  bogon (192.168.1.18)  0.559 ms  0.497 ms  0.445 ms
[root@bogon ~]# telnet 192.168.1.18 1521
Trying 192.168.1.18...
Connected to 192.168.1.18.
Escape character is '^]'.

[oracle@bogon ~]$ tnsping 192.168.1.18

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 13-MAY-2023 08:59:56

Copyright (c) 1997, 2022, Oracle.  All rights reserved.

Used parameter files:
/u01/app/oracle/product/19.3.0/db_1/network/admin/sqlnet.ora

Used HOSTNAME adapter to resolve the alias
Attempting to contact (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=tcp)(HOST=192.168.1.18)(PORT=1521)))
OK (0 msec)

克隆的环境全部都关闭下防火墙，并禁用：

systemctl status firewalld
systemctl stop firewalld
systemctl disable firewalld

一切正常！

这个根因找到后着实有点丢人了，折腾半天居然就是最开始怀疑的防火墙问题，但却因为命令搞错误判了。

记得早些时候这个Linux6查看防火墙的命令在Linux7中执行，重定向是可以变成正确命令的。

再就是应该多查下，如果是类似下面的输出才是没有防火墙运行：

[root@db11g ~]# /sbin/iptables -L
Chain INPUT (policy ACCEPT)
target     prot opt source               destination         

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination         

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

不过troubleshooting的有意思之处也就在于此，你总不能每次都可以快速解决问题，哪怕是相对简单的问题，也可能因为一个疏忽而走不少弯路，但这个过程是有意思的！