LVS+keepalived结合

LVS+Keepalived实现高可用负载均衡（web集群）

LVS+Keepalived架构图：

 

测试环境：

名称                    操作系统               IP地址
LVS-MASTER             Centos7.x             192.168.1.23
LVS_BACKUP             Centos7.x             192.168.1.25
WebserverA             Centos7.x             192.168.1.20
WebserverB             Centos7.x             192.168.1.21
VIP                                          192.168.1.188

1）安装keepalived (MASTER/BACKUP安装一样):

#解决依赖：
yum install -y openssl openssl-devel popt-devel
#关闭防火墙selinux：
systemctl stop firewalld
setenforce 0

2）在LVS集群环境中应用时，需要用到ipvsadm管理工具：

yum install -y ipvsadm

3）正式编译安装keepalived：

wget https://www.keepalived.org/software/keepalived-1.4.5.tar.gz
tar xf keepalived-1.4.5.tar.gz
cd keepalived-1.4.5
./configure --prefix=/usr/local/keepalived/
make
make install

 mkdir /etc/keepalived/
cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
cp /usr/local/keepalived/etc/sysconfig/keepalived  /etc/sysconfig/
cp keepalived-1.4.5/keepalived/keepalived.service /etc/systemd/system/
ln -s /usr/local/keepalived/sbin/keepalived  /usr/sbin/
cp keepalived-1.4.5/keepalived/etc/init.d/keepalived /etc/init.d/
chmod 755 /etc/init.d/keepalived
systemctl enable keepalived
systemctl start keepalived
ps -ef|gre keepalived

5）配置MASTER 文件：vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
   notification_email {
      wgkgood@163.com
   }
   notification_email_from wgkgood@163.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
# VIP1
vrrp_instance VI_1 {
    state  BACKUP
    interface  ens33
    virtual_router_id 51
    priority 100
    advert_int 5
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.188
    }
}
virtual_server 192.168.1.188 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 192.168.1.20 80 {
        weight 100
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    real_server 192.168.1.21 80 {
        weight 100
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }

6）配置BACKUP 文件：vim /etc/keepalived/keepalived.conf

! Configuration File for keepalived
global_defs {
   notification_email {
      wgkgood@163.com
   }
   notification_email_from wgkgood@163.com
   smtp_server 127.0.0.1
   smtp_connect_timeout 30
   router_id LVS_DEVEL
}
# VIP1
vrrp_instance VI_1 {
    state  BACKUP
    interface  ens33
    virtual_router_id 51
    priority 90
    advert_int 5
    nopreempt
    authentication {
        auth_type PASS
        auth_pass 1111
    }
    virtual_ipaddress {
        192.168.1.188
    }
}
virtual_server 192.168.1.188 80 {
    delay_loop 6
    lb_algo rr
    lb_kind DR
    protocol TCP
    real_server 192.168.1.20 80 {
        weight 100
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    real_server 192.168.1.21 80 {
        weight 100
        TCP_CHECK {
        connect_timeout 10
        nb_get_retry 3
        delay_before_retry 3
        connect_port 80
        }
    }

7）LVS配置均衡:
MASTER:

#开启ip_vs管理模块：
modprobe ip_vs
lsmod |grep -i ip_vs

ipvsadm -A -t 192.168.1.188:80 -s rr

ipvsadm -a -t 192.168.1.188:80 -r 192.168.1.20 -g -w 100
ipvsadm -a -t 192.168.1.188:80 -r 192.168.1.21 -g -w 100
#参数解释：
-A        大A 添加虚拟集群
-t        tcp协议
-s        指定算法rr模式
-a        在虚拟集群中添加后端真实IP
-g       指定DR模式
-w       权重

#查看虚拟集群状态：
ipvsadm -L -n

#删除集群IP：
ipvsadm -d -t 192.168.1.188:80 -r 192.168.1.20
ipvsadm -D -t 192.168.1.188:80

BACKUP:

#开启ip_vs管理模块：
modprobe ip_vs
lsmod |grep -i ip_vs

#添加虚拟集群实例/后端真实IP：
ipvsadm -A -t 192.168.1.188:80 -s rr
ipvsadm -a -t 192.168.1.188:80 -r 192.168.1.20 -g -w 100
ipvsadm -a -t 192.168.1.188:80 -r 192.168.1.21 -g -w 100

7）WebserverA/B配置DR模式：

1）LVS DR原理：用户请求LVS VIP到达director（LB均衡器），director将请求的报文的目标MAC地址改成后端的realserverMAC地址，目标IP为VIP（不变），源IP为用户IP地址（保持不变），如果Director将报文发送到realserver，realserver检测到目标为自己本地VIP，如果在同一个网段，然后将请求直接返给用户。如果用户跟realserver不在一个网段，则通过网关返回用户，如下图所示：

2）LVS DR模式注意事项：

• LVS服务器和后端服务器realserver必须在同网段（内网、公网）；
• LVS修改请求报文的目标MAC，目标（VIP）不修改的；
• 目标IP（VIP）保持不变。在RS后端配置VIP，lo网卡上面配置（不冲突）；
• RS后端服务器的网关指向路由器的下一跳，保证数据能够出去（访问外网）；
• 在所有RS后端服务器，抑制ARP广播，禁止VIP响应解析，而且要保证真实网卡不能抑制ARP广播。

 

5、LVS　DR模式架构部署：

1）Real server后端服务器的网关设路由器出口IP地址（正常设置，保持能够上网），另外配置拷贝ｌｏ网卡为lo：１，配置一个VIP地址（需要在同网段），掩码设置为４个２５５，否则网卡起不来（所有流量都会走ens３３真实网卡的）配置在ｌｏ网卡是为了VIP地址不冲突：

#拷贝网卡：
cp /etc/sysconfig/network-scripts/ifcfg-lo /etc/sysconfig/network-srcipts/ifcfg-lo:1
#编辑 添加如下内容 vim /etc/sysconfig/network-srcipts/ifcfg-lo:1
DEVICE=lo
IPADDR=192.168.1.188
NETMASK=255.255.255.255
ONBOOT=yes
NAME=loopback

#需要重启网卡：
ifup lo:1
systemctl restart network

2）为了实现均衡，需要在后端服务器上配置抑制arp广播，禁止arp响应解析，而且要保证真实网卡能接受arp广播：

cat /proc/sys/net/ipv4/conf/lo/arp_ignore  默认是0需要运行如下命令：

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
#解释：
arp ignore参数（1）含义：只响应目标IP配置在真实网卡；
arp announce参数（2）含义：忽略报文得源IP地址，使用主机上能够跟用户通信的真实网卡发送数据。

realserver配置DR脚本:

#!/bin/sh
#LVS Client Server
VIP=192.168.1.188
case  $1  in                                                                                                                                                                                                                                                   
start)                                                                                                                                                                                                                                           
    ifconfig lo:0 $VIP netmask 255.255.255.255 broadcast $VIP
    /sbin/route add -host $VIP dev lo:0
    echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
    sysctl -p >/dev/null 2>&1
    echo "RealServer Start OK"
    exit 0
;;                                                                                                                                                                                                                                        
stop)
    ifconfig lo:0 down
    route del $VIP >/dev/null 2>&1
    echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
    echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
    echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
    echo "RealServer Stoped OK"
    exit 1
;;
*)
    echo "Usage: $0 {start|stop}"
;;
esac

测试均衡情况：

while true;do curl http://192.168.1.188 ;sleep ３;done