LVS之DR模式实战及高可用性
author：JevonWei
版权声明：原创作品
LVS-DR实现同网段调度web模式
- 拓扑环境
网络环境
RS1
RIP 192.168.198.138/24
VIP 192.168.198.100/32
GW 192.168.198.130
RS2
RIP 192.168.198.132/24
VIP 192.168.198.100/32
GW 192.168.198.130
VS
DIP 192.168.198.128/24
VIP 192.168.198.100/32
GW 192.168.198.130
route
192.168.198.130/24
172.16.253.166/16
Client
172.16.254.150/16
GW 172.16.253.166
RS1,RS2的网关指向192.168.198.130
RS1
[root@RS1 html]# route del default gw 192.168.198.128
[root@RS1 html]# route add default gw 192.168.198.130
[root@RS1 ~]# iptables -F
[root@RS1 ~]# yum -y install httpd
[root@RS1 ~]# vim /var/www/html/index.html
welcome to RS1
[root@RS1 ~]# service httpd start
RS2
[root@RS2 network-scripts]# route add -net 172.16.0.0/16 gw 192.168.198.130
[root@RS2 network-scripts]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
172.16.0.0 192.168.198.130 255.255.0.0 UG 0 0 0 ens34
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
[root@RS2 ~]# iptables -F
[root@RS2 ~]# yum -y install httpd
[root@RS2 ~]# vim /var/www/html/index.html
welcome to RS2
[root@RS2 ~]# systemctl start httpd
VS
添加网关路由信息
[root@VS ~]# route add default gw 192.168.198.130
[root@VS ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.198.130 0.0.0.0 UG 0 0 0 ens34
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
[root@VS ~]# vim lvs_dr.sh
#! /bin/bash
vip=192.168.198.100
server=$vip:80
rip1=192.168.198.138
rip2=192.168.198.132
sch=wlc
dev=ens34:1 \\绑定网卡ens34
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip \\绑定vip到ens34网卡上
iptables -F
ipvsadm -A -t $server -s $sch
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ipconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@danran ~]# bash lvs_dr.sh start
[root@danran ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.198.100:80 wlc
-> 192.168.198.132:80 Route 1 0 0
-> 192.168.198.138:80 Route 3 0 0
添加RS1和RS2的VIP
dr_vip_rs.sh 为添加RS服务端VIP地址的脚本
[root@RS1 ~]# vim dr_vip_rs.sh
#!/bin/bash
#
vip=192.168.198.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask broadcast $vip up
route add -host $vip dev $dev
echo "VS server is Ready "
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "VS server is Cancel"
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
;;
esac
[root@RS1 ~]# bash dr_vip_rs.sh start
VS server is Ready
[root@RS2 ~]# bash dr_vip_rs.sh start
VS server is Ready
client
添加路由
[root@danran ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 172.16.0.1 0.0.0.0 UG 0 0 0 eth0
[root@danran ~]# route del default gw 172.16.0.1
[root@danran ~]# route add default gw 172.16.253.166 \\添加默认路由
[root@danran ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
169.254.0.0 0.0.0.0 255.255.0.0 U 1002 0 0 eth0
172.16.0.0 0.0.0.0 255.255.0.0 U 0 0 0 eth0
0.0.0.0 172.16.253.166 0.0.0.0 UG 0 0 0 eth0
测试
[root@danran ~]# for i in {1..10};do curl --connect-timeout 1 192.168.198.100 ;sleep 1;done
welcome to RS2
welcome to RS1
welcome to RS1
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS1
welcome to RS1
welcome to RS2
welcome to RS1
LVS-DR实现跨网段
网络拓扑
网络环境
RS1
RIP 192.168.198.138/24
VIP 192.168.80.100/32
GW 192.168.198.130
RS2
RIP 192.168.198.132/24
VIP 192.168.80.100/32
GW 192.168.198.130
VS
DIP 192.168.198.128/24
VIP 192.168.198.100/32
GW 192.168.198.130
route
192.168.198.130/24
192.168.80.130/8
172.16.253.166/16
GW 192.168.198.130
Client
172.16.254.150/16
GW 172.16.253.166
RS1,RS2的网关指向192.168.198.130
route
ens38网卡添加第二个IP
[root@route network-scripts]# nmcli connection modify ens38 +ipv4.addresses 192.168.80.130/8
[root@route ~]# nmcli connection up ens38 \\启动ens38网卡
[root@route ~]# ip a
[root@route ~]# route add default gw 192.168.198.130
VS
编辑LVS_DR的配置脚本
[root@VS ~]# vim lvs_dr.sh
#! /bin/bash
vip=192.168.80.100
server=$vip:80
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -t $server -s $sch
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
添加网关及默认路由
[root@VS ~]# route add default gw 192.168.198.130
[root@VS ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.198.130 0.0.0.0 UG 0 0 0 ens34
192.168.122.0 0.0.0.0 255.255.255.0 U 0 0 0 virbr0
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
RS1和RS2配置vip IP
[root@RS1 ~]# vim dr_vip_rs.sh
#!/bin/bash
#
vip=192.168.80.100
mask='255.255.255.255'
dev=lo:1
case $1 in
start)
echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
ifconfig $dev $vip netmask $mask broadcast $vip up
# route add -host $vip dev $dev
echo "VS server is Ready "
;;
stop)
ifconfig $dev down
echo 0 > /proc/sys/net/ipv4/conf/all/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_ignore
echo 0 > /proc/sys/net/ipv4/conf/all/arp_announce
echo 0 > /proc/sys/net/ipv4/conf/lo/arp_announce
echo "VS server is Cancel"
;;
*)
echo "Usage $(basename $0) start|stop"
exit 1
;;
esac
[root@RS1 ~]# bash dr_vip_rs.sh start
VS server is Ready
[root@RS2 ~]# bash dr_vip_rs.sh start
VS server is Ready
路由信息
[root@RS2 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
0.0.0.0 192.168.198.130 0.0.0.0 UG 100 0 0 ens34
192.168.198.0 0.0.0.0 255.255.255.0 U 100 0 0 ens34
[root@RS1 ~]# route -n
Kernel IP routing table
Destination Gateway Genmask Flags Metric Ref Use Iface
192.168.198.0 0.0.0.0 255.255.255.0 U 0 0 0 eth1
0.0.0.0 192.168.198.130 0.0.0.0 UG 0 0 0 eth1
client
[root@client ~]# for i in {1..10};do curl 192.168.80.100 ;done
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
welcome to RS2
welcome to RS1
将http和https两个不同的服务打标签,从而使http和https做成一个集群服务
FireWall Mark技术
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
[root@VS ~]# vim lvs_dr_vs_fwm.sh
#! /bin/bash
vip=192.168.80.100
server=10
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -f $server -s $sch
ipvsadm -a -f $server -r $rip1 -g -w 3
ipvsadm -a -f $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@VS ~]# bash lvs_dr_vs_fwm.sh start
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 10 rr
-> 192.168.198.132:0 Route 1 0 0
-> 192.168.198.138:0 Route 3 0 0
client
[root@client ~]# curl 192.168.80.100；curl -k https://192.168.80.100
实现DR持久连接
PFWMC基于防火墙的持久连接
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
[root@VS ~]# vim lvs_dr_vs_fwm.sh
#! /bin/bash
vip=192.168.80.100
server=10
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -f $server -s $sch -p 600 \\-p 设置持久连接为600s
ipvsadm -a -f $server -r $rip1 -g -w 3
ipvsadm -a -f $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
; ;
esac
[root@VS ~]# bash lvs_dr_vs_fwm.sh start
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 10 rr persistent 600 \\持久连接为600s
-> 192.168.198.132:0 Route 1 0 0
-> 192.168.198.138:0 Route 3 0 0
client
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl https://192.168.80.100
welcome to RS2
[root@client ~]# curl https://192.168.80.100
welcome to RS2
PCC基于0端口的持久连接
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
[root@VS ~]# vim lvs_dr_vs_per.sh
#! /bin/bash
vip=192.168.80.100
server=$vip:0
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip netmask 255.255.255.255 broadcast $vip
ipvsadm -A -t $server -s $sch -p 600
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@VS ~]# bash lvs_dr_vs_per.sh start
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.80.100:0 rr persistent 600
-> 192.168.198.132:0 Route 1 0 0
-> 192.168.198.138:0 Route 3 0 0
LVS高可用性
VS(编写脚本判断RS服务器是否故障)
[root@VS ~]# vim lvs_dr_vs.sh
#! /bin/bash
vip=192.168.80.100
server=$vip:80
rip1=192.168.198.138
rip2=192.168.198.132
sch=rr
dev=ens34:1
case $1 in
start)
ifconfig $dev $vip/32 broadcast $vip
ipvsadm -A -t $server -s $sch
ipvsadm -a -t $server -r $rip1 -g -w 3
ipvsadm -a -t $server -r $rip2 -g -w 1
;;
stop)
ipvsadm -C
ifconfig $dev down
;;
*)
echo "Usage:$(basename $0) start|stop"
exit 1
;;
esac
[root@VS ~]# bash lvs_dr_vs.sh start
ldirectord实现LVS的高可用性
当RS服务端崩溃时，自动从LVS中删除
VS
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10
下载ldirectord软件包(pub/Source/7.x86/crmsh/)
[root@VS ~]# yum -y install ldirectord-3.9.6-0rc1.1.1.x86_64.rpm \\需有完整yum源
[root@VS ~]# rpm -ql ldirectord
/etc/ha.d
/etc/ha.d/resource.d
/etc/ha.d/resource.d/ldirectord
/etc/logrotate.d/ldirectord
/usr/lib/ocf/resource.d/heartbeat/ldirectord
/usr/lib/systemd/system/ldirectord.service
/usr/sbin/ldirectord
/usr/share/doc/ldirectord-3.9.6
/usr/share/doc/ldirectord-3.9.6/COPYING
/usr/share/doc/ldirectord-3.9.6/ldirectord.cf
/usr/share/man/man8/ldirectord.8.gz
[root@VS ~]# cp /usr/share/doc/ldirectord-3.9.6/ldirectord.cf /etc/ha.d
[root@VS ~]# vim /etc/ha.d/ldirectord.cf
checktimeout=3 \\超时时间
checkinterval=1 \\检查间隔
fallback=127.0.0.1:80 \\Sorry Server,错误的网页
autoreload=yes \\自动加载配置文件
logfile="/var/log/ldirectord.log" \\日志文件
quiescent=no \\当RS宕机时是否将RS记录从ipvsadm记录中删除，no表示宕机即删除
virtual=192.168.80.100:80 \\VS服务端IP
real=192.168.198.138:80 gate 2 \\RS服务端IP，gate表示dr类型
real=192.168.198.132:80 gate 1 \\RS服务端IP，gate表示dr类型
fallback=127.0.0.1:80 gate
service=http
scheduler=wrr \\调度算法
protocol=tcp \\tcp协议
checktype=negotiate
checkport=80 \\检查端口
request="index.html" \\检查网页
receive="danran" \\检查网页字符，若包含该字符，则表示RS服务端正常
[root@VS ~]# systemctl start ldirectord
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.80.100:80 rr
-> 192.168.198.132:80 Route 1 0 0
-> 192.168.198.138:80 Route 1 0 0
client
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS1
[root@client ~]# curl 192.168.80.100
welcome to RS2
[root@client ~]# curl 192.168.80.100
welcome to RS1
使用标签实现ldirectord将多个服务定义为一个集群服务
使用打标签时需删除protocol=tcp选项
[root@VS ~]# iptables -t mangle -A PREROUTING -d 192.168.80.100 -p tcp -m multiport --dports 80,443 -j MARK --set-mark 10 \\标签定义为10
[root@VS ~]# iptables -t mangle -nvL
Chain PREROUTING (policy ACCEPT 41 packets, 3944 bytes)
pkts bytes target prot opt in out source destination
0 0 MARK tcp -- * * 0.0.0.0/0 192.168.80.100 multiport dports 80,443 MARK set 0xa
[root@VS ~]# vim /etc/ha.d/ldirectord.cf
checktimeout=3 \\超时时间
checkinterval=1 \\检查间隔
fallback=127.0.0.1:80 \\Sorry Server,错误的网页
autoreload=yes \\自动加载配置文件
logfile="/var/log/ldirectord.log" \\日志文件
quiescent=no \\当RS宕机时是否将RS记录从ipvsadm记录中删除，no表示宕机即删除
virtual=10 \\VS标签为10
real=192.168.198.138:80 gate 2 \\RS服务端IP，gate表示dr类型
real=192.168.198.132:80 gate 1 \\RS服务端IP，gate表示dr类型
fallback=127.0.0.1:80 gate
service=http
scheduler=wrr \\调度算法
checktype=negotiate
checkport=80 \\检查端口
request="index.html" \\检查网页
receive="danran" \\检查网页字符，若包含该字符，则表示RS服务端正常
[root@VS ~]# systemctl start ldirectord
[root@VS ~]# ipvsadm -Ln
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
FWM 10 rr
-> 192.168.198.132:80 Route 1 0 0
-> 192.168.198.138:80 Route 1 0 0LVS之DR模式实战及高可用性的更多相关文章
- LVS：DR模式（Direct Routing）部署实验
本文介绍怎样在kvm的虚拟环境下,部署实验LVS的DR模式.包含网络结构图,怎样配置.以及使用tcpdump分析ip包. 网络结构图 kvm ...
- LVS的DR模式
DR模式: 请求由LVS接受,由真实提供服务的服务器(RealServer, RS)直接返回给用户,返回的时候不经过LVS. DR模式下需要LVS和绑定同一个VIP(RS通过将VIP绑定在loopba ...
- LVS的DR模式负载均衡
参考项目:http://www.cnblogs.com/along21/p/7833261.html#auto_id_3 LVS的DR模式实现负载均衡 1.环境 lvs-server :192.168 ...
- Lvs Keepalive DR模式高可用配置
Lvs Keepalive DR模式配置 一.环境 #DIP# eth0:192.168.233.145#VIP# eth0:0 192.168.233.250/32 #RIP1:192.168.23 ...
- lvs中dr模式配置脚本
1 dr模式介绍 1.1 lvs的安装 安装具体解释:http://blog.csdn.net/CleverCode/article/details/50586957. 1.2 lvs模式 lvs有三 ...
- lvs部署-DR模式
DR模式 角色 IP地址 备注 LVS负载均衡器 192.168.119.132 VIP:192.168.119.150 ipvsadm http_Real server 192.168.119 ...
- CentOS6.4 配置LVS（DR模式）
DR模式中LVS主机与实际服务器都有一块网卡连在同一物理网段上. IP分配 VIP:10.10.3.170 RIP1:10.10.3.140 RIP2:10.10.3.141 1.安装所需的依赖包 y ...
- LVS的DR模式测试案例<仅个人记录>
初始概念 大家都知道LVS,是章文嵩博士创建的,所以首先推一下主站吧!http://zh.linuxvirtualserver.org/ LVS集群分为三层结构: 负载调度器(load balance ...
- LVS+keepalived DR模式配置高可用负载均衡集群
实验环境 LVS-Master 10.0.100.201 VIP:10.0.100.203 LVS-Slave 10.0.100.204 WEB1-Tomcat 10.0.2.29 gat ...
随机推荐
- hadoop入门，跑出第一个WordCount
1.环境准备 下载:http://mirror.bit.edu.cn/apache/hadoop/common/hadoop-2.7.2/hadoop-2.7.2.tar.gz 解压:解压后,修改et ...
- 浅谈js中如何动态添加表头/表列/表格内容
我想很多童鞋用js动态向表格中添加数据很熟悉,而且也觉得非常简单!是的,对于写页面的童鞋来说,最喜欢写查询的页面了,动态向表格绑定数据.用for循环就可以轻松搞定. 如果我们的业务需求有所变化,可能我 ...
- 如何在github制作一个网页
1.首先得先注册一个github账号,官网:https://github.com/ 2.注册完,登录账号进入首页,点右上角的 ‘+’ 创建新的仓库 3. 点击setting,选择一个主题, 4. 选完 ...
- Linux 系统下安装 rz/sz 命令及使用说明
Linux 系统下安装 rz/sz 命令及使用说明 rz/sz命令,实现将本地的文件上传到服务器或者从服务器上下载文件到本地,但是很多Linux系统初始并没有这两个命令,以下为安装和使用的具体步骤: ...
- Python操作文件和目录
Python操作文件和目录 读写文件比较简单,有一点特别注意就好了 windows下Python默认打开的文件以gbk解码,而一般我们的文件是utf-8编码的,所以如果文本含有中文,就会出现异常或者乱 ...
- C++基本内置类型
C++基本内置类型 基本内置类型包括算术类型和空类型. 算术类型 算术类型包括整型和浮点型. 类型 含义 最小尺寸 bool 布尔型 - char 字符型 8 bit wchar_t 宽字符型 16 ...
- 最新 Zookeeper + Flume + Kafka 简易整合教程
在大数据领域有很多耳熟能详的框架,今天要介绍的就是 zookeeper.flume.kafka.因为平时是做数据接入的,所以对这些实时的数据处理系统不是很熟悉.通过官网的简要介绍,搭建了一套简要的平台 ...
- akoj-1148-小光棍数
小光棍数 Time Limit:1000MS Memory Limit:65536K Total Submit:197 Accepted:94 Description 最近Topcoder的XD遇到 ...
- NOIP2000提高组 单词接龙
题目描述 单词接龙是一个与我们经常玩的成语接龙相类似的游戏,现在我们已知一组单词,且给定一个开头的字母,要求出以这个字母开头的最长的"龙"(每个单词都最多在"龙" ...
- Webapi文档描述-swagger优化
一.前言 最近做的项目使用WebApi,采取前后端分离的方式,后台提供API接口给前端开发人员.这个过程中遇到一个问题后台开发人员怎么提供接口说明文档给前端开发人员,最初打算使用word.Xmind思 ...