Linux学习-LVS跨网段DR模型和FWM多服务绑定

一、实验环境

　系统：CentOS7.6

　主机：5台 (虚拟机)

　　客户端1台：172.16.236.134/24 (NAT网卡)，网关指向 172.16.236.185/24（路由服务器）

　　路由服务器1台：172.16.236.185/24 (NAT)，192.168.214.17/16 (仅主机)，附加绑定IP (10.0.0.200/8)

　　LVS服务器1台：VIP (10.0.0.100/32，绑定在lo网卡上)，DIP (192.168.214.27/16，仅主机)，网关指向 192.168.214.17/16（路由服务器）

　　RS1服务器1台：VIP (10.0.0.100/32，绑定在lo网卡上)，RIP (192.168.214.37/16，仅主机)，网关指向 192.168.214.17/16（路由服务器）

　　RS2服务器1台：VIP (10.0.0.100/32，绑定在lo网卡上)，RIP (192.168.214.47/16，仅主机)，网关指向 192.168.214.17/16（路由服务器）

　软件包：ipvsadm，httpd，mod_ssl（光盘yum源）

二、相关实验

1、实现LVS跨网段DR模型

(1) 按网络规划，配置好各主机的IP

　客户端服务器：eth0：172.16.236.134/24，网关 172.16.236.185

　路由器服务器：eth0：172.16.236.185/24，eth1：192.168.214.17/16，网关不需配

　LVS服务器：eth0：192.168.214.27/16，网关 192.168.214.17

　RS1服务器：eth0：192.168.214.37/16，网关 192.168.214.17

　RS2服务器：eth0：192.168.214.47/16，网关 192.168.214.17

(2) 在路由服务器上开启网络转发功能

[root@centos7-17 ~]# vim /etc/sysctl.conf
net.ipv4.ip_forward=
[root@centos7-17 ~]# sysctl -p
net.ipv4.ip_forward = 

(3) 在LVS服务器上安装ipvsadm包

[root@centos7- ~]# yum install -y ipvsadm

(4) 在LVS服务器上配置LVS，此处用脚本实现，脚本如下

[root@centos7- ~]# vim lvs_dr_vs.sh
#!/bin/bash

vip='10.0.0.100'    #VIP
iface='lo:1'  #VIP绑定接口
mask='255.255.255.255'    #VIP子网掩码
port=''    #端口
rs1='192.168.214.37'    #RS1服务器IP
rs2='192.168.214.47'    #RS2服务器IP
scheduler='wrr'    #调度算法
type='-g'    #LVS类型，-m为nat模式,-g为dr模式，-i为tun模式

case $ in
start)
        ifconfig $iface $vip netmask $mask #broadcast $vip up
        iptables -F
        ipvsadm -A -t ${vip}:${port} -s $scheduler
        ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w
        ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w
        ;;
stop)
        ipvsadm -C
        ifconfig $iface down
        ;;
*)
        echo "Usage $(basename $0) start|stop"
        exit
        ;;
esac

#!/bin/bash

vip='10.0.0.100'
iface='lo:1'
mask='255.255.255.255'
port=''
rs1='192.168.214.37'
rs2='192.168.214.47'
scheduler='wrr'
type='-g'

case $ in
start)
        ifconfig $iface $vip netmask $mask #broadcast $vip up
        iptables -F
        ipvsadm -A -t ${vip}:${port} -s $scheduler
        ipvsadm -a -t ${vip}:${port} -r ${rs1} $type -w
        ipvsadm -a -t ${vip}:${port} -r ${rs2} $type -w
        ;;
stop)
        ipvsadm -C
        ifconfig $iface down
        ;;
*)
        echo "Usage $(basename $0) start|stop"
        exit
        ;;
esac

lvs_dr_vs.sh

(5) 在R1服务器配置httpd服务，并配置VIP绑定与相关内核参数，配置RIP(192.168.214.37)时，记得网关指向 192.168.214.27

[root@centos7- ~]# yum install -y httpd
[root@centos7- ~]# echo 192.168.214.37 RS1 > /var/www/html/index.html
[root@centos7- ~]# systemctl start httpd
#配置VIP绑定与相关内核参数，配置脚本如下
[root@centos7- ~]# vim lvs_dr_rs.sh
#!/bin/bash

vip='10.0.0.100'
mask='255.255.255.255'
dev='lo:1'

case $ in
start)
    echo  > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/all/arp_announce
    echo  > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    ;;
stop)
    ifconfig $dev down
    echo  > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/all/arp_announce
    echo  > /proc/sys/net/ipv4/conf/lo/arp_announce
    ;;
*)
    echo "Usage:$(basename $0) start|stop"
    exit
    ;;
esac

[root@centos7- ~]# bash lvs_dr_rs.sh start

(6) 在R2服务器配置httpd服务，并配置VIP绑定与相关内核参数，配置RIP(192.168.214.37)时，记得网关指向 192.168.214.27

[root@centos7- ~]# yum install -y httpd
[root@centos7- ~]# systemctl start httpd
[root@centos7- ~]# echo 192.168.214.47 RS2 > /var/www/html/index.html
#配置VIP绑定相关内核参数，脚本与R1上一样
[root@centos7- ~]# bash lvs_dr_rs.sh start

#!/bin/bash

vip='10.0.0.100'
mask='255.255.255.255'
dev='lo:1'

case $ in
start)
    echo  > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/all/arp_announce
    echo  > /proc/sys/net/ipv4/conf/lo/arp_announce
    ifconfig $dev $vip netmask $mask #broadcast $vip up
    #route add -host $vip dev $dev
    ;;
stop)
    ifconfig $dev down
    echo  > /proc/sys/net/ipv4/conf/all/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/lo/arp_ignore
    echo  > /proc/sys/net/ipv4/conf/all/arp_announce
    echo  > /proc/sys/net/ipv4/conf/lo/arp_announce
    ;;
*)
    echo "Usage:$(basename $0) start|stop"
    exit
    ;;
esac

lvs_dr_rs.sh

(7) 在LVS服务器上启动配置脚本

[root@centos7- ~]# bash lvs_dr_vs.sh start
[root@centos7- ~]# ipvsadm -Ln
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  10.0.0.100: wrr
  -> 192.168.214.37:            Route
  -> 192.168.214.47:            Route                       

(8) 在客户端上进行测试

[root@centos7 ~]# while true;do curl 10.0.0.100 ;sleep ;done
#可以看到成功了
192.168.214.37 RS1
192.168.214.47 RS2
192.168.214.37 RS1
192.168.214.47 RS2
192.168.214.37 RS1
192.168.214.47 RS2
...

2、实现LVS的FWM多服务绑定

　当http服务同时用到80与443端口时，用DR模式会有点繁琐，此时用FWM (FireWall Mark)是一种不错的选择。

(1) 只需要LVS服务器稍稍修改即可，其它服务器还是按以上配置

　在LVS主机上打标记

　　iptables -t mangle -A PREROUTING -d $vip -p $proto –m multiport --dports $port1,$port2,... -j MARK --set-mark NUMBER

　在LVS主机基于标记定义集群服务

　　ipvsadm -A -f NUMBER [options]

[root@centos7- ~]# bash lvs_dr_vs.sh stop
[root@centos7- ~]# ipvsadm
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
[root@centos7- ~]# ifconfig lo: 10.0.0.100 netmask 255.255.255.255 #绑定VIP
#打标记
[root@centos7- ~]# iptables -t mangle -A PREROUTING -d 10.0.0.100 -p tcp -m multiport --dports , -j MARK --set-mark
#配置集群
[root@centos7- ~]# ipvsadm -A -f  -s rr
[root@centos7- ~]# ipvsadm -a -f  -r 192.168.214.37 -g
[root@centos7- ~]# ipvsadm -a -f  -r 192.168.214.47 -g
[root@centos7- ~]# ipvsadm -Ln
IP Virtual Server version 1.2. (size=)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
FWM   rr
  -> 192.168.214.37:             Route
  -> 192.168.214.47:             Route                    

(2) 在两台RS服务器上安装mod_ssl模块，实现https的访问

[root@centos7- ~]# yum install -y mod_ssl
[root@centos7- ~]# systemctl restart httpd

[root@centos7- ~]# yum install -y mod_ssl
[root@centos7- ~]# systemctl restart httpd

(3) 在客户端上测试80与443端口的调度情况

[root@centos7 ~]# while true;do curl 10.0.0.100 ;curl -k https://10.0.0.100;sleep 1;done
#可以看到成功了，且不分端口进行了统一调度
192.168.214.47 RS2
192.168.214.37 RS1
192.168.214.47 RS2
192.168.214.37 RS1
192.168.214.47 RS2
192.168.214.37 RS1
192.168.214.47 RS2
192.168.214.37 RS1
192.168.214.47 RS2
192.168.214.37 RS1