LVS之DR模式实战及高可用性
author：JevonWei
版权声明：原创作品
LVS-DR实现同网段调度web模式
- 拓扑环境
网络环境
RS1
RIP 192.168.198.138/24
VIP 192.168.198.100/32
GW 192.168.198.130
RS2
RIP 192.168.198.132/24
VIP 192.168.198.100/32
GW 192.168.198.130
VS
DIP 192.168.198.128/24
VIP 192.168.198.100/32
GW 192.168.198.130
route
192.168.198.130/24
172.16.253.166/16
Client
172.16.254.150/16
GW 172.16.253.166
RS1,RS2的网关指向192.168.198.130
RS1
[root@RS1 html]# route del default gw 192.168.198.128
[root@RS1 html]# route add default gw 192.168.198.130
[root@RS1 ~]# iptables -F
[root@RS1 ~]# yum -y install httpd
[root@RS1 ~]# vim /var/www/html/index.html
welcome to RS1
[root@RS1 ~]# service httpd start
RS2
[root@RS2 network-scripts]# route add -net 172.16.0.0/16 gw 192.168.198.130
[root@RS2 network-scripts]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.0.0 192.168.198.130 255.255.0.0 UG 0 0 0 ens34
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
[root@RS2 ~]# iptables -F
[root@RS2 ~]# yum -y install httpd
[root@RS2 ~]# vim /var/www/html/index.html
welcome to RS2
[root@RS2 ~]# systemctl start httpd
VS
添加网关路由信息
[root@VS ~]# route add default gw 192.168.198.130
[root@VS ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.198.130 0.0.0.0 UG 0 0 0 ens34
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
[root@VS ~]# vim lvs_dr.sh
#! /bin/bash
vip=192.168.198.100
server=$vip:80
rip1=192.168.198.138
rip2=192.168.198.132
sch=wlc
dev=ens34:1 \\绑定网卡ens34
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip \\绑定vip到ens34网卡上
iptables -F
ipvsadm -A -t $server -s $sch
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ipconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@danran ~]# bash lvs_dr.sh start
[root@danran ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.198.100:80 wlc
-> 192.168.198.132:80 Route 1 0 0
-> 192.168.198.138:80 Route 3 0 0
添加RS1和RS2的VIP
dr_vip_rs.sh 为添加RS服务端VIP地址的脚本
[root@RS1 ~]# vim dr_vip_rs.sh
#!/bin/bash
#
vip=192.168.198.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask broadcast $vip up
route add -host $vip dev $dev
echo "VS server is Ready "
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "VS server is Cancel"
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
;;
esac
[root@RS1 ~]# bash dr_vip_rs.sh start
VS server is Ready
[root@RS2 ~]# bash dr_vip_rs.sh start
VS server is Ready
client
添加路由
[root@danran ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 eth0
[root@danran ~]# route del default gw 172.16.0.1
[root@danran ~]# route add default gw 172.16.253.166 \\添加默认路由
[root@danran ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 172.16.253.166 0.0.0.0 UG 0 0 0 eth0
测试
[root@danran ~]# for i in {1..10};do curl --connect-timeout 1 192.168.198.100 ;sleep 1;done
welcome to RS2
welcome to RS1
welcome to RS1
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS1
welcome to RS1
welcome to RS2
welcome to RS1
LVS-DR实现跨网段
网络拓扑
网络环境
RS1
RIP 192.168.198.138/24
VIP 192.168.80.100/32
GW 192.168.198.130
RS2
RIP 192.168.198.132/24
VIP 192.168.80.100/32
GW 192.168.198.130
VS
DIP 192.168.198.128/24
VIP 192.168.198.100/32
GW 192.168.198.130
route
192.168.198.130/24
192.168.80.130/8
172.16.253.166/16
GW 192.168.198.130
Client
172.16.254.150/16
GW 172.16.253.166
RS1,RS2的网关指向192.168.198.130
route
ens38网卡添加第二个IP
[root@route network-scripts]# nmcli connection modify ens38 +ipv4.addresses 192.168.80.130/8
[root@route ~]# nmcli connection up ens38 \\启动ens38网卡
[root@route ~]# ip a
[root@route ~]# route add default gw 192.168.198.130
VS
编辑LVS_DR的配置脚本
[root@VS ~]# vim lvs_dr.sh
#! /bin/bash
vip=192.168.80.100
server=$vip:80
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -t $server -s $sch
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
添加网关及默认路由
[root@VS ~]# route add default gw 192.168.198.130
[root@VS ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.198.130 0.0.0.0 UG 0 0 0 ens34
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
RS1和RS2配置vip IP
[root@RS1 ~]# vim dr_vip_rs.sh
#!/bin/bash
#
vip=192.168.80.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask broadcast $vip up
# route add -host $vip dev $dev
echo "VS server is Ready "
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "VS server is Cancel"
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
;;
esac
[root@RS1 ~]# bash dr_vip_rs.sh start
VS server is Ready
[root@RS2 ~]# bash dr_vip_rs.sh start
VS server is Ready
路由信息
[root@RS2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.198.130 0.0.0.0 UG 100 0 0 ens34
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
[root@RS1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.198.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
0.0.0.0 192.168.198.130 0.0.0.0 UG 0 0 0 eth1
client
[root@client ~]# for i in {1..10};do curl 192.168.80.100 ;done
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
将http和https两个不同的服务打标签,从而使http和https做成一个集群服务
FireWall Mark技术
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
[root@VS ~]# vim lvs_dr_vs_fwm.sh
#! /bin/bash
vip=192.168.80.100
server=10
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -f $server -s $sch
ipvsadm -a -f $server -r $rip1 -g -w 3
ipvsadm -a -f $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@VS ~]# bash lvs_dr_vs_fwm.sh start
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 10 rr
-> 192.168.198.132:0 Route 1 0 0
-> 192.168.198.138:0 Route 3 0 0
client
[root@client ~]# curl 192.168.80.100；curl -k https://192.168.80.100
实现DR持久连接
PFWMC基于防火墙的持久连接
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
[root@VS ~]# vim lvs_dr_vs_fwm.sh
#! /bin/bash
vip=192.168.80.100
server=10
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -f $server -s $sch -p 600 \\-p 设置持久连接为600s
ipvsadm -a -f $server -r $rip1 -g -w 3
ipvsadm -a -f $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
; ;
esac
[root@VS ~]# bash lvs_dr_vs_fwm.sh start
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 10 rr persistent 600 \\持久连接为600s
-> 192.168.198.132:0 Route 1 0 0
-> 192.168.198.138:0 Route 3 0 0
client
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl https://192.168.80.100
welcome to RS2
[root@client ~]# curl https://192.168.80.100
welcome to RS2
PCC基于0端口的持久连接
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
[root@VS ~]# vim lvs_dr_vs_per.sh
#! /bin/bash
vip=192.168.80.100
server=$vip:0
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip netmask 255.255.255.255 broadcast $vip
ipvsadm -A -t $server -s $sch -p 600
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@VS ~]# bash lvs_dr_vs_per.sh start
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.80.100:0 rr persistent 600
-> 192.168.198.132:0 Route 1 0 0
-> 192.168.198.138:0 Route 3 0 0
LVS高可用性
VS(编写脚本判断RS服务器是否故障)
[root@VS ~]# vim lvs_dr_vs.sh
#! /bin/bash
vip=192.168.80.100
server=$vip:80
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -t $server -s $sch
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@VS ~]# bash lvs_dr_vs.sh start
ldirectord实现LVS的高可用性
当RS服务端崩溃时，自动从LVS中删除
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
下载ldirectord软件包(pub/Source/7.x86/crmsh/)
[root@VS ~]# yum -y install ldirectord-3.9.6-0rc1.1.1.x86_64.rpm \\需有完整yum源
[root@VS ~]# rpm -ql ldirectord
/etc/ha.d
/etc/ha.d/resource.d
/etc/ha.d/resource.d/ldirectord
/etc/logrotate.d/ldirectord
/usr/lib/ocf/resource.d/heartbeat/ldirectord
/usr/lib/systemd/system/ldirectord.service
/usr/sbin/ldirectord
/usr/share/doc/ldirectord-3.9.6
/usr/share/doc/ldirectord-3.9.6/COPYING
/usr/share/doc/ldirectord-3.9.6/ldirectord.cf
/usr/share/man/man8/ldirectord.8.gz
[root@VS ~]# cp /usr/share/doc/ldirectord-3.9.6/ldirectord.cf /etc/ha.d
[root@VS ~]# vim /etc/ha.d/ldirectord.cf
checktimeout=3 \\超时时间
checkinterval=1 \\检查间隔
fallback=127.0.0.1:80 \\Sorry Server,错误的网页
autoreload=yes \\自动加载配置文件
logfile="/var/log/ldirectord.log" \\日志文件
quiescent=no \\当RS宕机时是否将RS记录从ipvsadm记录中删除，no表示宕机即删除
virtual=192.168.80.100:80 \\VS服务端IP
real=192.168.198.138:80 gate 2 \\RS服务端IP，gate表示dr类型
real=192.168.198.132:80 gate 1 \\RS服务端IP，gate表示dr类型
fallback=127.0.0.1:80 gate
service=http
scheduler=wrr \\调度算法
protocol=tcp \\tcp协议
checktype=negotiate
checkport=80 \\检查端口
request="index.html" \\检查网页
receive="danran" \\检查网页字符，若包含该字符，则表示RS服务端正常
[root@VS ~]# systemctl start ldirectord
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.80.100:80 rr
-> 192.168.198.132:80 Route 1 0 0
-> 192.168.198.138:80 Route 1 0 0
client
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS1
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS1
使用标签实现ldirectord将多个服务定义为一个集群服务
使用打标签时需删除protocol=tcp选项
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10 \\标签定义为10
[root@VS ~]# iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 41 packets, 3944 bytes)
pkts bytes target prot opt in out source destination
0 0 MARK tcp -- * * 0.0.0.0/0 192.168.80.100 multiport dports 80,443 MARK set 0xa
[root@VS ~]# vim /etc/ha.d/ldirectord.cf
checktimeout=3 \\超时时间
checkinterval=1 \\检查间隔
fallback=127.0.0.1:80 \\Sorry Server,错误的网页
autoreload=yes \\自动加载配置文件
logfile="/var/log/ldirectord.log" \\日志文件
quiescent=no \\当RS宕机时是否将RS记录从ipvsadm记录中删除，no表示宕机即删除
virtual=10 \\VS标签为10
real=192.168.198.138:80 gate 2 \\RS服务端IP，gate表示dr类型
real=192.168.198.132:80 gate 1 \\RS服务端IP，gate表示dr类型
fallback=127.0.0.1:80 gate
service=http
scheduler=wrr \\调度算法
checktype=negotiate
checkport=80 \\检查端口
request="index.html" \\检查网页
receive="danran" \\检查网页字符，若包含该字符，则表示RS服务端正常
[root@VS ~]# systemctl start ldirectord
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 10 rr
-> 192.168.198.132:80 Route 1 0 0
-> 192.168.198.138:80 Route 1 0 0LVS之DR模式实战及高可用性的更多相关文章
- LVS：DR模式（Direct Routing）部署实验
本文介绍怎样在kvm的虚拟环境下,部署实验LVS的DR模式.包含网络结构图,怎样配置.以及使用tcpdump分析ip包. 网络结构图 kvm ...
- LVS的DR模式
DR模式: 请求由LVS接受,由真实提供服务的服务器(RealServer, RS)直接返回给用户,返回的时候不经过LVS. DR模式下需要LVS和绑定同一个VIP(RS通过将VIP绑定在loopba ...
- LVS的DR模式负载均衡
参考项目:http://www.cnblogs.com/along21/p/7833261.html#auto_id_3 LVS的DR模式实现负载均衡 1.环境 lvs-server :192.168 ...
- Lvs Keepalive DR模式高可用配置
Lvs Keepalive DR模式配置 一.环境 #DIP# eth0:192.168.233.145#VIP# eth0:0 192.168.233.250/32 #RIP1:192.168.23 ...
- lvs中dr模式配置脚本
1 dr模式介绍 1.1 lvs的安装 安装具体解释:http://blog.csdn.net/CleverCode/article/details/50586957. 1.2 lvs模式 lvs有三 ...
- lvs部署-DR模式
DR模式 角色 IP地址 备注 LVS负载均衡器 192.168.119.132 VIP:192.168.119.150 ipvsadm http_Real server 192.168.119 ...
- CentOS6.4 配置LVS（DR模式）
DR模式中LVS主机与实际服务器都有一块网卡连在同一物理网段上. IP分配 VIP:10.10.3.170 RIP1:10.10.3.140 RIP2:10.10.3.141 1.安装所需的依赖包 y ...
- LVS的DR模式测试案例<仅个人记录>
初始概念 大家都知道LVS,是章文嵩博士创建的,所以首先推一下主站吧!http://zh.linuxvirtualserver.org/ LVS集群分为三层结构: 负载调度器(load balance ...
- LVS+keepalived DR模式配置高可用负载均衡集群
实验环境 LVS-Master 10.0.100.201 VIP:10.0.100.203 LVS-Slave 10.0.100.204 WEB1-Tomcat 10.0.2.29 gat ...
随机推荐
- AC自动机总结及板子（不带指针）
蒟蒻最近想学个AC自动机简直被网上的板子搞疯了,随便点开一个都是带指针的,然而平时用到指针的时候并不多,看到这些代码也完全是看不懂的状态.只好在大概理解后自己脑补(yy)了一下AC自动机的代码,居然还 ...
- opnet点对点通信模型 分类： opnet 2014-05-26 22:15 246人阅读 评论(3) 收藏
网络包含两个节点,一个发送节点,一个接收节点.发送节点按照某种随机的规律产生数据包(包大小和包间隔可自己定义),然后发送给接收节点.传输过程中会有一些随机的差错(误包率也可自己定义).接收节点收到正确 ...
- AD7928
实验室板子soc-de1自带的7928芯片,下面记录一下它的参数 吞吐速率 : 1MSPS 吞吐速率 : 是指ADC器件单位时间内能处理的任务数或输出结果的数量.单位:SPS(Samples per ...
- RabbitMQ入门-初识RabbitMQ
初识RabbitMQ 要说RabbitMQ,我们不得不先说下AMQP.AMQP,即Advanced Message Queuing Protocol,高级消息队列协议,是应用层协议的一个开放标准,为面 ...
- JAVA多线程之wait/notify
本文主要学习JAVA多线程中的 wait()方法 与 notify()/notifyAll()方法的用法. ①wait() 与 notify/notifyAll 方法必须在同步代码块中使用 ②wait ...
- Redis-key的设计技巧
把表名转换为key前缀, 比如: tag: 第二段放置用于区分key的字段--对应mysql中的主键的列名 第三段放置主键值 第三段写列名 用户表user, 转换为redis的key-value存储 ...
- Java基础之集合框架类及泛型简介
Collection接口 Collection 通用的常见方法 add()添加一个元素,可以指定脚标 addAll()将一个collection放入 clear()清除 remove()删除元素,返回 ...
- 写个 Hello world - 前端从入坑到弃坑系列教程（1）
这是一个系列教程<前端从入坑到弃坑>的第一篇. HTML 是什么 说白了,HTML 就是网页的内容.比如你现在正在阅读的这个网页的内容,就是 HTML.如果你还不明白,请继续往下阅读. 写 ...
- 实例甜点 Unreal Engine 4迷你教程(5)之函数中的静态变量
本小节的教程无前置教程,可直接学习,篇幅很短. 本教程浓缩起来就是一句话:函数中的静态变量在调试过程中保留值.所以需要谨慎对待. 什么意思?请先不要一步一步对着做,而整体地看一遍下面的过程: 第一步: ...
- Given a linked list, reverse the nodes of a linked list k at a time and return its modified list. If the number of nodes is not a multiple of k then left-out nodes in the end should remain as it is.
我用String代替了链表显示,本题的大意是每k个进行逆序处理,剩下的不够k个的就按照原顺序保留下来. public class ReverseNodes { public static void m ...