keepalived脑裂问题查找
在自己环境做keepalived+redis实验时,当重启了备用redies机器后,发现两台redies主机都拿到了VIP
- [root@redis2 ~]# ip addr list
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
- link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- inet 127.0.0.1/8 scope host lo
- inet6 ::1/128 scope host
- valid_lft forever preferred_lft forever
- 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- link/ether 52:54:00:72:6a:7c brd ff:ff:ff:ff:ff:ff
- inet 192.168.122.54/24 brd 192.168.122.255 scope global eth0
- inet 192.168.122.50/32 scope global eth0
- inet6 fe80::5054:ff:fe72:6a7c/64 scope link
- valid_lft forever preferred_lft forever
- [root@localhost ~]# ip addr list
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
- link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- inet 127.0.0.1/8 scope host lo
- inet6 ::1/128 scope host
- valid_lft forever preferred_lft forever
- 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- link/ether 52:54:00:85:7b:a9 brd ff:ff:ff:ff:ff:ff
- inet 192.168.122.96/24 brd 192.168.122.255 scope global eth0
- inet 192.168.122.50/32 scope global eth0
- inet6 fe80::5054:ff:fe85:7ba9/64 scope link
- valid_lft forever preferred_lft forever
也就是出现了keepalived的脑裂现象,检查了两台主机的网络连通状态,发现网络是好的。然后在备机上抓包
- [root@localhost ~]# tcpdump -i eth0|grep VRRP
- tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
- listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
- 15:51:17.146322 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:17.146577 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:17.146972 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:18.147136 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:18.147576 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:25.151399 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:25.151942 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:26.151703 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:26.152623 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:27.152456 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:27.153261 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:28.152955 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:28.153461 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:29.153766 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:29.155652 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:30.154275 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:30.154587 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:31.155042 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:31.155428 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:32.155539 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:32.155986 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:33.156357 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:33.156979 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
- 15:51:34.156801 IP 192.168.122.96 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 50, authtype simple, intvl 1s, length 20
- 15:51:34.156989 IP 192.168.122.54 > vrrp.mcast.net: VRRPv2, Advertisement, vrid 51, prio 160, authtype simple, intvl 1s, length 20
备机能接收到master发过来的VRRP广播,那为什么还会有脑裂现象?发现重启后iptables还开着,检查了防火墙配置
- root@localhost ~]# iptables -S
- -P INPUT ACCEPT
- -P FORWARD ACCEPT
- -P OUTPUT ACCEPT
- -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
- -A INPUT -p icmp -j ACCEPT
- -A INPUT -i lo -j ACCEPT
- -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
- -A INPUT -j REJECT --reject-with icmp-host-prohibited
- -A FORWARD -j REJECT --reject-with icmp-host-prohibited
发现系统不接收VRRP协议,于是修改iptables
- [root@localhost ~]# iptables -I INPUT 4 -p vrrp -j ACCEPT
- [root@localhost ~]# iptables -S
- -P INPUT ACCEPT
- -P FORWARD ACCEPT
- -P OUTPUT ACCEPT
- -A INPUT -m state --state RELATED,ESTABLISHED -j ACCEPT
- -A INPUT -p icmp -j ACCEPT
- -A INPUT -i lo -j ACCEPT
- -A INPUT -p vrrp -j ACCEPT
- -A INPUT -p tcp -m state --state NEW -m tcp --dport 22 -j ACCEPT
- -A INPUT -j REJECT --reject-with icmp-host-prohibited
- -A FORWARD -j REJECT --reject-with icmp-host-prohibited
- [root@localhost ~]# ip addr list
- 1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN
- link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
- inet 127.0.0.1/8 scope host lo
- inet6 ::1/128 scope host
- valid_lft forever preferred_lft forever
- 2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
- link/ether 52:54:00:85:7b:a9 brd ff:ff:ff:ff:ff:ff
- inet 192.168.122.96/24 brd 192.168.122.255 scope global eth0
- inet6 fe80::5054:ff:fe85:7ba9/64 scope link
- valid_lft forever preferred_lft forever
发现VIP没了。虽然问题解决了,但为什么备机明明能抓到master发来的VRRP广播包,但却无法改变自身状态呢?只能说明网卡接收到数据包是在iptables处理数据包之前发生的事情。
keepalived脑裂问题查找的更多相关文章
- 记一次keepalived脑裂问题查找
在自己环境做keepalived+Redis实验时,当重启了备用redies机器后,发现两台redies主机都拿到了VIP [root@redis2 ~]# ip addr list 1: lo: & ...
- Keepalived脑裂
问题描述:开启防火墙后,Keepalived出现脑裂. 背景架构:两台centos7通过Keepalived实现高可用 问题具体表现形式:两台主机通过ip addr (ip a)查看,发现两台主机都 ...
- 解决keepalived脑裂问题
检测思路:正常情况下keepalived的VIP地址是在主节点上的,如果在从节点发现了VIP,就设置报警信息 脚本如下: #!/bin/bash # 检查脑裂的脚本,在备节点上进行部署 LB01_VI ...
- keepalived脑裂问题
一.对脑裂的理解 在高可用(HA)系统中,当联系2个节点的“心跳线”断开时,本来为一整体.动作协调的HA系统,就分裂成为2个独立的个体.由于相互失去了联系,都以为是对方出了故障.两个节点上的HA软件像 ...
- split-brain 脑裂问题(Keepalived)
脑裂(split-brain)指在一个高可用(HA)系统中,当联系着的两个节点断开联系时,本来为一个整体的系统,分裂为两个独立节点,这时两个节点开始争抢共享资源,结果会导致系统混乱,数据损坏.对于无状 ...
- keepalived的脑裂问题与解决
Keepalived的作用是检测服务器的状态,如果有一台web服务器宕机,或工作出现故障,Keepalived将检测到,并将有故障的服务器从系统中剔除,同时使用其他服务器代替该服务器的工作,当服务器工 ...
- keepalive脑裂的处理,从节点发现访问的虚拟IP就报警,同时尝试发送内容到主节点服务器关闭keepalive和nginx,或者关机
解决keepalived脑裂问题 检测思路:正常情况下keepalived的VIP地址是在主节点上的,如果在从节点发现了VIP,就设置报警信息 脚本如下: 1 2 3 4 5 6 7 8 9 10 ...
- keepalived的脑裂问题
keepalived的脑裂问题 学习了:http://blog.51cto.com/10630401/2089847 split-brain 无HA不脑裂
- keepalived 预防脑裂检测脚本
1 检查vip [root@mysql2 keepalived]# cat /etc/keepalived/check_brain_keepalived.sh #!/bin/bash # 检查脑裂的脚 ...
随机推荐
- c# 服务
注:服务里的timer System.Timers.Timer time=new System.Timers.Timer(); time.Interval = 3000; //设置计时器事件间隔执 ...
- Windows server 2008 R2配置多个远程连接
1.右键计算机属性——远程设置——出现系统属性对话框——选择“远程”选项卡,按如下图操作:. 2.默认只有administrator具有远程桌面的权限,其他用户都没有权限远程桌面连接服务器.因此,我们 ...
- [android] 隐式意图激活另外一个activity
随着api的升级,系统的很多应用包名和类名都改掉了,所以很多时候,打开系统应用的时候会报错,隐式意图就是解决组件之间松耦合,描述动作行为 获取Intent对象,通过new出来 调用Intent对象的s ...
- form表单基础知识
form 元素是块级元素 ------------------- ------------------- ----------------------------------------------- ...
- 设置dataGridView单元格颜色、字体、ToolTip、字体颜色
this.dataGridView3.Rows[e.RowIndex].Cells["你的那个要判断的列名"].Style.BackColor = Color.MediumPurp ...
- 大数据时代,Wyn Enterprise和您一起探讨CIO的困境和出路 ZT
这是一篇知识分享帖,如果您致力于成为一名CIO,希望您能够阅读完,信息虽然简略,但我们依然希望可以帮到您. CIO:首席信息官 CIO是干什么的 一.经典的CIO主要负责什么 1.IT战略规划.IT预 ...
- ionic 兼容title居中显示和tab栏在显示底部
在app.js里的 .config 里添加配置,同时在函数中引入 $ionicConfigProvider,具体格式如下所示: .config(function($stateProvider, $ur ...
- ChakraCore/JSRT使用问题汇总
QQ交流群:523723780(ChakraCore) ChakraCore是什么? 一个微软开源的,用于Windows IE/Edge内核的高效JS脚本引擎. 前不久微软开源了ChakraCore, ...
- UVA 1152 4 Values whose Sum is 0 (枚举+中途相遇法)(+Java版)(Java手撕快排+二分)
4 Values whose Sum is 0 题目链接:https://cn.vjudge.net/problem/UVA-1152 ——每天在线,欢迎留言谈论. 题目大意: 给定4个n(1< ...
- MyBatis笔记----报错:Exception in thread "main" org.apache.ibatis.binding.BindingException: Invalid bound statement (not found)解决方法
报错 Exception in thread "main" org.apache.ibatis.binding.BindingException: Invalid bound st ...