005.HAProxy+Keepalived高可用负载均衡

一 基础准备

1.1 部署环境及说明

系统OS：CentOS 6.8 64位

HAProxy软件：HA-Proxy version 1.5.18

Keepalived软件：keepalived-1.3.6.tar.gz

官方链接：http://www.haproxy.org/ （国内可能无法打开）

下载连接：http://pkgs.fedoraproject.org/repo/pkgs/haproxy/

部署说明：当用户访问对应的域名时，HAProxy能将请求发送到对应的后端主机上，同时当主HAProxy服务器发生故障后，能立刻将负载均衡服务切换到备用HAProxy服务器上。

主机名	主机IP地址	集群角色	虚机IP/域名
master	172.24.8.10	主HAProxy服务器	172.24.8.100
backup	172.24.8.11	备HAProxy服务器
webapp1	172.24.8.30	后端web服务器	www.lz.com
webapp2	172.24.8.31		static.lz.com
webapp3	172.24.8.32		video.lz.com

1.2 架构规划

二 后端httpd集群部署

2.1 部署httpd集群

 [root@webapp1 ~]# yum -y install httpd
 [root@webapp1 ~]# vi /var/www/html/index.html
 This is my www.lz.com!
 [root@webapp1 ~]# vi /var/www/html/index.html
 This is my static.lz.com!
 [root@webapp1 ~]# vi /var/www/html/index.html
 This is my video.lz.com!
 [root@webapp1 ~]# systemctl start httpd.service
 [root@webapp1 ~]# systemctl enable httpd.service
 [root@webapp1 ~]# systemctl stop firewalld.service
 [root@webapp1 ~]# systemctl disable firewalld.service
 [root@webapp1 ~]# vi /etc/selinux/config
 SELINUX=disabled
 [root@webapp1 ~]# setenforce 0			#关闭SELinux及防火墙

 

注意：后端real server节点都需要安装，本环境针对httpd简单安装即可，无需过多配置。

三 基础NTP部署

3.1 NTP部署

 [root@webapp1 ~]# yum -y install ntp
 [root@webapp1 ~]# systemctl start  ntpd.service

 

建议：替换附件中的ntp配置文件，建议采用阿里云时钟进行同步。

注意：为了保证集群的稳定性，强烈建议在所有节点均部署NTP同步服务，保证所有时钟一致。

四 Keepalived部署

4.1 编译环境

安装基础环境及依赖：

 # yum -y install gcc gcc-c++ make kernel-devel kernel-tools kernel-tools-libs kernel libnl libnl-devel libnfnetlink-devel openssl-devel wget openssh-clients

4.2 安装Keepalived

 [root@haproxy_master ~]# wget http://www.keepalived.org/software/keepalived-1.3.6.tar.gz
 [root@haproxy_master ~]# tar -zxvf keepalived-1.3.6.tar.gz
 [root@haproxy_master ~]# cd keepalived-1.3.6/
 [root@lvsmaster keepalived-1.3.6]# ./configure --prefix=/usr/local/keepalived
 [root@haproxy_master keepalived-1.3.6]# make && make install

 

注意：CentOS6.8安装高于1.3.6版本会出现未知错误。

4.3 添加Keepalived启动相关服务

 [root@haproxy_master ~]# mkdir /etc/keepalived
 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
 [root@haproxy_master ~]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
 [root@haproxy_master ~]# vi /etc/init.d/keepalived	#创建Keepalived启动脚本，见附件
 [root@haproxy_master ~]# chmod u+x /etc/rc.d/init.d/keepalived

 

4.4 配置Keepalived

 [root@haproxy_master ~]# vi /etc/keepalived/keepalived.conf
 global_defs {
    notification_email {
 #……
    }
    notification_email_from Alexandre.Cassen@firewall.loc
    smtp_server 192.168.200.1
    smtp_connect_timeout 30
    router_id LVS_DEVEL
 }

 vrrp_script check_haproxy {
     script "/usr/bin/killall -0 haproxy"
     interval    2
     weight      21
     }

 vrrp_instance HAProxy_HA {
     state BACKUP			#在HAProxy主备均设置为BACKUP
     interface eth0
     virtual_router_id 80
     priority 100
     advert_int 2
     nopreempt				#不抢占模式
     authentication {
         auth_type PASS
         auth_pass 1111
     }

     notify_master "/etc/keepalived/master.sh"
     notify_backup "/etc/keepalived/backup.sh"
     notify_fault "/etc/keepalived/fault.sh"

     track_script {
     check_haproxy
     }

     virtual_ipaddress {
         172.24.8.100 dev eth0
     }
 }

 

4.5 编写Keepalived脚本

 [root@haproxy_master ~]# vi /etc/keepalived/master.sh
 #!/bin/bash
 LOGFILE=/var/log/keepalived-mysql-state/log
 date >>$LOGFILE
 echo "[Master]" >>$LOGFILE
 [root@haproxy_master ~]# vi /etc/keepalived/backup.sh
 #!/bin/bash
 LOGFILE=/var/log/keepalived-mysql-state/log
 date >>$LOGFILE
 echo "[BACKUP]" >>$LOGFILE
 [root@haproxy_master ~]# vi /etc/keepalived/fault.sh
 #!/bin/bash
 LOGFILE=/var/log/keepalived-mysql-state/log
 date >>$LOGFILE
 echo "[FAULT]" >>$LOGFILE
 [root@haproxy_master ~]# chmod u+x /etc/keepalived/backup.sh
 [root@haproxy_master ~]# chmod u+x /etc/keepalived/master.sh
 [root@haproxy_master ~]# chmod u+x /etc/keepalived/fault.sh

 

提示：本环境采用测试脚本，真实环境建议采用自动发送邮件通知运维员的脚本。

4.5 Bakcup节点配置

 [root@haproxy_master ~]# scp /etc/keepalived/keepalived.conf 172.24.8.11:/etc/keepalived/keepalived.conf    #将配置好的Master节点的配置文件复制到Backup节点
 [root@haproxy_slave ~]# vi /etc/keepalived/keepalived.conf
 state BACKUP
 priority 80

 

注意：在HAProxy备节点也设置为BACKUP，priority修改为低于HAProxy主角色的优先级即可，同时去掉nopreempt。

 [root@haproxy_master ~]# scp /etc/keepalived/*.sh 172.24.8.11:/etc/keepalived/
 #将对应的脚本也复制至backup节点。

 

五 HAProxy部署

5.1 HAProxy安装

 [root@haproxy_master ~]# yum -y install haproxy

提示：需要在主备HAProxy节点都安装。

5.2 HAProxy配置

 [root@haproxy_master ~]# vi /etc/haproxy/haproxy.cfg
 global
     log         127.0.0.1 local0 info
     chroot      /var/lib/haproxy
     pidfile     /var/run/haproxy.pid
     maxconn     4096
     user        haproxy
     group       haproxy
     daemon
     nbproc      1
 defaults
     mode                    http
     log                     global
     retries                 3
     timeout connect         5s
     timeout client          30s
     timeout server          30s
     timeout check           2s
 frontend        www
     bind        172.24.8.100:80
     mode        http
     option      httplog
     option      forwardfor
     log         global

     acl host_www        hdr_dom(host)   -i  www.lz.com	#配置不同域名分发不同后端策略
     acl host_static     hdr_dom(host)   -i  static.lz.com
     acl host_video      hdr_dom(host)   -i  video.lz.com

     use_backend     server_www      if      host_www	#配置不同域名分发不同后端策略
     use_backend     server_static   if      host_static
     use_backend     server_video    if      host_video
 backend         server_www		#后端真是服务器
     mode        http
     option      redispatch
     option      abortonclose
     balance     roundrobin
     option      httpchk GET /index.html
     server      webapp1 172.24.8.30:80  weight 6 check inter 2000 rise 2 fall 3
 backend         server_static		#后端真是服务器
     mode        http
     option      redispatch
     option      abortonclose
     balance     roundrobin
     option      httpchk GET /index.html
     server      webapp2 172.24.8.31:80  weight 6 check inter 2000 rise 2 fall 3
 backend         server_video		#后端真是服务器
     mode        http
     option      redispatch
     option      abortonclose
     balance     roundrobin
     option      httpchk GET /index.html
     server      webapp3 172.24.8.32:80  weight 6 check inter 2000 rise 2 fall 3
 [root@haproxy_master ~]# scp /etc/haproxy/haproxy.cfg root@172.24.8.11:/etc/haproxy/haproxy.cfg		#将配置文件复制至HAProxy备节点

 

5.3 打开转发

 [root@haproxy_master ~]# vi /etc/sysctl.conf
 net.ipv4.ip_nonlocal_bind = 1
 [root@haproxy_master ~]# sysctl -p

 

注意：绑定非本机的IP必须在sysctl.conf文件中配置。

六 启动服务

 [root@webapp1 ~]# systemctl start httpd

提示：三个节点的httpd服务均启动。

 [root@haproxy_master ~]# service haproxy start

提示：必须先启动HAProxy服务，之后启动Keepalived，因为Keepalived会先检测HAProxy服务进程。

 [root@haproxy_master ~]# service keepalived start

七 验证测试

7.1 高可用验证

 [root@haproxy_master ~]# ip add

 [root@haproxy_master ~]# service haproxy stop		#停止HAProxy主节点的HAProxy进程
 [root@haproxy_master ~]# tail -f /var/log/messages	#观察HAProxy主节点日志

 

 [root@haproxy_slave ~]# ip addr				#查看备HAProxy节点的IP

结论：通过测试可知当主HAProxy节点服务异常，Keepalived会检测到，同时HAProxy会将vip从主节点移除，备HAProxy会接管。

 [root@haproxy_master ~]# service haproxy start

结论：由于配置了非抢占模式，主HAProxy恢复进程之后，依旧由备HAProxy提供服务，vip不会切回至主HAProxy。

7.2 负载均衡测试

主机hosts中添加如下解析：

 172.24.8.100	www.lz.com
 172.24.8.100	static.lz.com
 172.24.8.100	video.lz.com

 

浏览器分别访问不用的三个域名：