[转帖]LVS入门篇（四）之LVS实战

LVS入门篇（四）之LVS实战

https://www.cnblogs.com/linuxk/p/9360922.html

• 一、LVS的NAT模式实战

1、环境说明：

HOST	OS	role	remask
192.168.56.12	Centos 7.4	LVS调度器（1.2.7）	VIP：192.168.0.104
192.168.56.11	Centos 7.4	RS1	LNMP
192.168.56.13	Centos 7.4	RS2	LNMP

2、LVS调度器环境配置

（1）增加一块外网网卡eth1

（2）拷贝eth0配置文件为eth1，并修改网卡配置

[root@lb01 ~]# cp /etc/sysconfig/network-scripts/ifcfg-eth0 /etc/sysconfig/network-scripts/ifcfg-eth1
[root@lb01 ~]# vim !$
vim /etc/sysconfig/network-scripts/ifcfg-eth1
TYPE="Ethernet"
PROXY_METHOD="none"
BROWSER_ONLY="no"
BOOTPROTO="static"
DEFROUTE="yes"
IPV4_FAILURE_FATAL="no"
NAME="eth1"
DEVICE="eth1"
ONBOOT="yes"
IPADDR=192.168.0.104
NETMASK=255.255.255.0
GATEWAY=192.168.0.1
DNS1=8.8.8.8
DNS2=114.114.114.114
[root@lb01 ~]# systemctl restart network
[root@lb01 ~]# ifconfig eth1
eth1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.0.104  netmask 255.255.255.0  broadcast 192.168.0.255
        inet6 fe80::20c:29ff:fe95:2993  prefixlen 64  scopeid 0x20<link>
        ether 00:0c:29:95:29:93  txqueuelen 1000  (Ethernet)
        RX packets 2383  bytes 271561 (265.1 KiB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 43  bytes 5606 (5.4 KiB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

（3）YUM安装ipvsadm，并对ipvsadm规则进行配置

1、开启调度器上的路由转发功能
[root@lb01 ~]# echo 1 > /proc/sys/net/ipv4/ip_forward　

2、关闭ICMP重定向　
[root@lb01 ~]# echo 0 > /proc/sys/net/ipv4/conf/all/send_redirects 　　
[root@lb01 ~]# echo 0 > /proc/sys/net/ipv4/conf/default/send_redirects
[root@lb01 ~]# echo 0 > /proc/sys/net/ipv4/conf/eth0/send_redirects
[root@lb01 ~]# echo 0 > /proc/sys/net/ipv4/conf/eth1/send_redirects 

3、配置防火墙的NAT规则
[root@lb01 ~]# iptables -t nat -F　　
[root@lb01 ~]# iptables -t nat -X
[root@lb01 ~]# iptables -t nat -A POSTROUTING -s 192.168.56.0/24 -j MASQUERADE

4、配置ipvsadm并查看调度器状态信息
[root@lb01 ~]# ipvsadm -C
[root@lb01 ~]# ipvsadm -A -t 192.168.0.104:80 -s rr
[root@lb01 ~]# ipvsadm -a -t 192.168.0.104:80 -r 192.168.56.11:80 -m -w 1
[root@lb01 ~]# ipvsadm -a -t 192.168.0.104:80 -r 192.168.56.13:80 -m -w 1
[root@lb01 ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.104:80 rr
  -> 192.168.56.11:80             Masq    1      0          0
  -> 192.168.56.13:80             Masq    1      0          0         

参数说明：

-A：表示增加一个虚拟服务

-a：表示增加一个real server

-t：表示TCP服务

-r：指定real server的ip地址

-s：指定调度算法

-m：选择NAT方式调度

-w：指定权重

（3）修改real server：RS1和RS2的网关

[root@rs1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
GATEWAY=192.168.56.12
[root@rs2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0
GATEWAY=192.168.56.12

（4）测试访问

[root@lb01 ~]# curl 192.168.0.104
<h1>welcome to RS1 192.168.56.11</h1>
[root@lb01 ~]# curl 192.168.0.104
<h1> welcome to use RS2 192.168.56.13</h1>
[root@lb01 ~]# watch -n 1 ipvsadm -L -n　　#动态查看访问记录
Every 1.0s: ipvsadm -L -n                                                                                                                                            Tue Jul 24 04:40:59 2018

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.104:80 rr
  -> 192.168.56.11:80             Masq    1     0          2
  -> 192.168.56.13:80             Masq    1     0          3

• 二、LVS的DR模式实战

1、环境准备和说明

HOST	OS	ROLE	REMASK
192.168.56.12	Centos 7.4	LVS调度器	VIP：192.168.56.200
192.168.56.11	Centos 7.4	RS1	LNMP
192.168.56.13	Centos 7.4	RS2	LNMP

[root@lb01 ~]# ipvsadm -ln　　#查看NAT模式下的ipvsadm规则
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.0.104:80 rr
  -> 192.168.56.11:80             Masq    1      0          0
  -> 192.168.56.13:80             Masq    1      0          0
[root@lb01 ~]# ipvsadm -C　　#清空NAT模式下的规则
[root@lb01 ~]# ipvsadm -ln　　#确认规则是否清空
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
[root@lb01 ~]# iptables -t nat -F　　#情况防火墙的nat表规则
[root@rs1 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0 　　#修改rs1网卡网关
GATEWAY=192.168.56.2
[root@rs2 ~]# vim /etc/sysconfig/network-scripts/ifcfg-eth0 　　#修改rs2网卡网关
GATEWAY=192.168.56.2
[root@rs1 ~]# systemctl restart network　　#重启网络服务
[root@rs2 ~]# systemctl restart network

　[root@lb01 ~]# ifdown eth1　　#DR模式调度器用不到eth1网卡，down掉
  Device 'eth1' successfully disconnected.

2、在LVS调度节点上配置vip

[root@lb01 ~]# ifconfig eth0:1 192.168.56.200/24 up
[root@lb01 ~]# ifconfig eth0:1
eth0:1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 192.168.56.200  netmask 255.255.255.0  broadcast 192.168.56.255
        ether 00:0c:29:95:29:89  txqueuelen 1000  (Ethernet)

3、在lvs调度节点上配置LVS服务并增加2台节点服务器

[root@lb01 ~]# ipvsadm -C　　#清空LVS配置
[root@lb01 ~]# ipvsadm --set 30 5 60　　#设置连接超时时间
[root@lb01 ~]# ipvsadm -A -t 192.168.56.200:80 -s rr -p 20　　#新增lvs服务
[root@lb01 ~]# ipvsadm -a -t 192.168.56.200:80 -r 192.168.56.11 -g -w 1
[root@lb01 ~]# ipvsadm -a -t 192.168.56.200:80 -r 192.168.56.13 -g -w 1
[root@lb01 ~]# ipvsadm -L -n
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.56.200:80 rr persistent 20
  -> 192.168.56.11:80             Route   1      0          0
  -> 192.168.56.13:80             Route   1      0          0 

4、在RS节点上的lo（网卡的回环接口）接口上配置vip（192.168.56.200）

[root@rs1 ~]# ifconfig lo:1 192.168.56.200/32 up
[root@localhost ~]# ifconfig lo:1
lo:1: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 192.168.56.200  netmask 0.0.0.0
        loop  txqueuelen 1  (Local Loopback)
[root@rs2 ~]# ifconfig lo:1 192.168.56.200/32 up
[root@localhost ~]# ifconfig lo:1
lo:1: flags=73<UP,LOOPBACK,RUNNING>  mtu 65536
        inet 192.168.56.200  netmask 0.0.0.0
        loop  txqueuelen 1  (Local Loopback)

5、在RS节点上抑制ARP响应 （RS1和RS2上分别执行）

[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs1 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs1 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/lo/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/lo/arp_announce
[root@rs2 ~]# echo 1 > /proc/sys/net/ipv4/conf/all/arp_ignore
[root@rs2 ~]# echo 2 > /proc/sys/net/ipv4/conf/all/arp_announce

6、测试LVS调度

访问：http://192.168.56.200 ，如图：

[root@lb01 ~]# watch -n 1 ipvsadm -L -n 　　#查看访问状态
Every 1.0s: ipvsadm -L -n                                                                                                                                            Tue Jul 24 05:11:25 2018

IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.56.200:80 rr persistent 20
  -> 192.168.56.11:80             Route   1     0          0
  -> 192.168.56.13:80             Route   1     3          1

7、删除节点和服务

[root@lb01 ~]# ipvsadm -d -t 192.168.56.200:80 -r 192.168.56.13   #删除RS2节点
[root@lb01 ~]# ipvsadm -Ln　　#查看是否已删除RS2
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
  -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
TCP  192.168.56.200:80 rr persistent 20
  -> 192.168.56.11:80             Route   1      0          0
[root@lb01 ~]# ipvsadm -D -t 192.168.56.200:80  　　#直接删除LVS服务

 8、LVS脚本配置（设置为开机启动，确保长期生效）

（1）Server端脚本

[root@lb01 ~]# vim /scripts/ipvs_server.sh
#!/bin/sh
. /etc/init.d/functions
VIP=192.168.56.200
PORT=80
RIP=(
   192.168.56.11
   192.168.56.13
)
start(){
ifconfig eth0:1 $VIP/24 up
route add -host $VIP dev eth0
ipvsadm -C
ipvsadm --set 30 5 60
ipvsadm -A -t $VIP:$PORT -s rr -p 20
for ((i=0;i<${#RIP[*]};i++))
do
ipvsadm -a -t $VIP:$PORT -r ${RIP[$i]} -g -w 1
done
echo "ipvs is started"
}
stop(){
ipvsadm -C
ifconfig eth0:1  down
route del -host $VIP dev eth0
echo "ipvs is stopped"
}
case "$1" in
    start)
        start
        ;;
    stop)
        stop
        ;;
    restart)
        stop
        start
        ;;
    *)
        echo "UASGE: $0  {start|stop|restart}"
esac
[root@lb01 ~]# cp /scripts/ipvs_server.sh /usr/local/sbin/ipvs
[root@lb01 ~]# cd /usr/local/sbin/
[root@lb01 ~]# chmod +x ipvs

（2）Client端脚本

[root@rs1 ~]# vi /scripts/ipvs_client.sh
#!/bin/sh
. /etc/init.d/functions
VIP=(
    192.168.56.200
)

case "$1" in
    start)
        echo "start LVS of RealServer IP"
        for ((i=0;i<`echo ${#VIP[*]}`;i++))
        do
            interface="lo:`echo ${VIP[$i]}|awk -F . '{print $4}'`"
            /sbin/ifconfig $interface ${VIP[$i]} broadcast ${VIP[$i]} netmask 255.255.255.255 up
            route add -host ${VIP[$i]} dev $interface
        done
        echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
        echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
        echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
        echo "1" >/proc/sys/net/ipv4/conf/all/arp_announce
        ;;
    stop)
        for ((i=0;i<`echo ${#VIP[*]}`;i++))
        do
            interface="lo:`echo ${VIP[$i]}|awk -F . '{print $4}'`"
            /sbin/ifconfig $interface ${VIP[$i]} broadcast ${VIP[$i]} netmask 255.255.255.255 down
            route del -host ${VIP[$i]} dev $interface
        done
        echo "stOP LVS of RealServer IP"
        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
        echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
        echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
        echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
        ;;
    *)
        echo "Usage: $0 {start | stop}"
        exit 1
esac