HAProxy+Keepalived高可用负载均衡

一基础准备

1.1 部署环境及说明

系统OS：CentOS 6.8 64位

HAProxy软件：HA-Proxy version 1.5.18

Keepalived软件：keepalived-1.3.6.tar.gz

官方链接：http://www.haproxy.org/ （国内可能无法打开）

下载连接：http://pkgs.fedoraproject.org/repo/pkgs/haproxy/

部署说明：当用户访问对应的域名时，HAProxy能将请求发送到对应的后端主机上，同时当主HAProxy服务器发生故障后，能立刻将负载均衡服务切换到备用HAProxy服务器上。

主机名	主机IP地址	集群角色	虚机IP/域名
master	172.24.8.10	主HAProxy服务器	172.24.8.100
backup	172.24.8.11	备HAProxy服务器	172.24.8.100
webapp1	172.24.8.30	后端web服务器	www.lz.com
webapp2	172.24.8.31		static.lz.com
webapp3	172.24.8.32		video.lz.com

1.2 架构规划

二后端httpd集群部署

2.1 部署httpd集群

  1 [root@webapp1 ~]# yum -y install httpd

  2 [root@webapp1 ~]# vi /var/www/html/index.html

  3 This is my www.lz.com!

  4 [root@webapp1 ~]# vi /var/www/html/index.html

  5 This is my static.lz.com!

  6 [root@webapp1 ~]# vi /var/www/html/index.html

  7 This is my video.lz.com!

  8 [root@webapp1 ~]# systemctl start httpd.service

  9 [root@webapp1 ~]# systemctl enable httpd.service

 10 [root@webapp1 ~]# systemctl stop firewalld.service

 11 [root@webapp1 ~]# systemctl disable firewalld.service

 12 [root@webapp1 ~]# vi /etc/selinux/config

 13 SELINUX=disabled

 14 [root@webapp1 ~]# setenforce 0			#关闭SELinux及防火墙

注意：后端real server节点都需要安装，本环境针对httpd简单安装即可，无需过多配置。

三基础NTP部署

3.1 NTP部署

  1 [root@webapp1 ~]# yum -y install ntp

  2 [root@webapp1 ~]# systemctl start  ntpd.service

建议：替换附件中的ntp配置文件，建议采用阿里云时钟进行同步。

注意：为了保证集群的稳定性，强烈建议在所有节点均部署NTP同步服务，保证所有时钟一致。

四 Keepalived部署

4.1 编译环境

安装基础环境及依赖：

  1 # yum -y install gcc gcc-c++ make kernel-devel kernel-tools kernel-tools-libs kernel libnl libnl-devel libnfnetlink-devel openssl-devel wget openssh-clients

4.2 安装Keepalived

  1 [root@haproxy_master ~]# wget http://www.keepalived.org/software/keepalived-1.3.6.tar.gz

  2 [root@haproxy_master ~]# tar -zxvf keepalived-1.3.6.tar.gz

  3 [root@haproxy_master ~]# cd keepalived-1.3.6/

  4 [root@lvsmaster keepalived-1.3.6]# ./configure --prefix=/usr/local/keepalived

  5 [root@haproxy_master keepalived-1.3.6]# make && make install

注意：CentOS6.8安装高于1.3.6版本会出现未知错误。

4.3 添加Keepalived启动相关服务

  1 [root@haproxy_master ~]# mkdir /etc/keepalived

  2 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

  3 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

  4 [root@haproxy_master ~]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/

  5 [root@haproxy_master ~]# vi /etc/init.d/keepalived	#创建Keepalived启动脚本，见附件

  6 [root@haproxy_master ~]# chmod u+x /etc/rc.d/init.d/keepalived

4.4 配置Keepalived

  1 [root@haproxy_master ~]# vi /etc/keepalived/keepalived.conf

  2 global_defs {

  3    notification_email {

  4 #……

  5    }

  6    notification_email_from Alexandre.Cassen@firewall.loc

  7    smtp_server 192.168.200.1

  8    smtp_connect_timeout 30

  9    router_id LVS_DEVEL

 10 }

 11

 12 vrrp_script check_haproxy {

 13     script "/usr/bin/killall -0 haproxy"

 14     interval    2

 15     weight      21

 16     }

 17

 18 vrrp_instance HAProxy_HA {

 19     state BACKUP			#在HAProxy主备均设置为BACKUP

 20     interface eth0

 21     virtual_router_id 80

 22     priority 100

 23     advert_int 2

 24     nopreempt				#不抢占模式

 25     authentication {

 26         auth_type PASS

 27         auth_pass 1111

 28     }

 29

 30     notify_master "/etc/keepalived/master.sh"

 31     notify_backup "/etc/keepalived/backup.sh"

 32     notify_fault "/etc/keepalived/fault.sh"

 33

 34     track_script {

 35     check_haproxy

 36     }

 37

 38     virtual_ipaddress {

 39         172.24.8.100 dev eth0

 40     }

 41 }

4.5 编写Keepalived脚本

  1 [root@haproxy_master ~]# vi /etc/keepalived/master.sh

  2 #!/bin/bash

  3 LOGFILE=/var/log/keepalived-mysql-state/log

  4 date >>$LOGFILE

  5 echo "[Master]" >>$LOGFILE

  6 [root@haproxy_master ~]# vi /etc/keepalived/backup.sh

  7 #!/bin/bash

  8 LOGFILE=/var/log/keepalived-mysql-state/log

  9 date >>$LOGFILE

 10 echo "[BACKUP]" >>$LOGFILE

 11 [root@haproxy_master ~]# vi /etc/keepalived/fault.sh

 12 #!/bin/bash

 13 LOGFILE=/var/log/keepalived-mysql-state/log

 14 date >>$LOGFILE

 15 echo "[FAULT]" >>$LOGFILE

 16 [root@haproxy_master ~]# chmod u+x /etc/keepalived/backup.sh

 17 [root@haproxy_master ~]# chmod u+x /etc/keepalived/master.sh

 18 [root@haproxy_master ~]# chmod u+x /etc/keepalived/fault.sh

提示：本环境采用测试脚本，真实环境建议采用自动发送邮件通知运维员的脚本。

4.5 Bakcup节点配置

  1 [root@haproxy_master ~]# scp /etc/keepalived/keepalived.conf 172.24.8.11:/etc/keepalived/keepalived.conf    #将配置好的Master节点的配置文件复制到Backup节点

  2 [root@haproxy_slave ~]# vi /etc/keepalived/keepalived.conf

  3 state BACKUP

  4 priority 80

注意：在HAProxy备节点也设置为BACKUP，priority修改为低于HAProxy主角色的优先级即可，同时去掉nopreempt。

  1 [root@haproxy_master ~]# scp /etc/keepalived/*.sh 172.24.8.11:/etc/keepalived/

  2 #将对应的脚本也复制至backup节点。

五 HAProxy部署

5.1 HAProxy安装

  1 [root@haproxy_master ~]# yum -y install haproxy

提示：需要在主备HAProxy节点都安装。

5.2 HAProxy配置

  1 [root@haproxy_master ~]# vi /etc/haproxy/haproxy.cfg

  2 global

  3     log         127.0.0.1 local0 info

  4     chroot      /var/lib/haproxy

  5     pidfile     /var/run/haproxy.pid

  6     maxconn     4096

  7     user        haproxy

  8     group       haproxy

  9     daemon

 10     nbproc      1

 11 defaults

 12     mode                    http

 13     log                     global

 14     retries                 3

 15     timeout connect         5s

 16     timeout client          30s

 17     timeout server          30s

 18     timeout check           2s

 19 frontend        www

 20     bind        172.24.8.100:80

 21     mode        http

 22     option      httplog

 23     option      forwardfor

 24     log         global

 25

 26     acl host_www        hdr_dom(host)   -i  www.lz.com	#配置不同域名分发不同后端策略

 27     acl host_static     hdr_dom(host)   -i  static.lz.com

 28     acl host_video      hdr_dom(host)   -i  video.lz.com

 29

 30     use_backend     server_www      if      host_www	#配置不同域名分发不同后端策略

 31     use_backend     server_static   if      host_static

 32     use_backend     server_video    if      host_video

 33 backend         server_www		#后端真是服务器

 34     mode        http

 35     option      redispatch

 36     option      abortonclose

 37     balance     roundrobin

 38     option      httpchk GET /index.html

 39     server      webapp1 172.24.8.30:80  weight 6 check inter 2000 rise 2 fall 3

 40 backend         server_static		#后端真是服务器

 41     mode        http

 42     option      redispatch

 43     option      abortonclose

 44     balance     roundrobin

 45     option      httpchk GET /index.html

 46     server      webapp2 172.24.8.31:80  weight 6 check inter 2000 rise 2 fall 3

 47 backend         server_video		#后端真是服务器

 48     mode        http

 49     option      redispatch

 50     option      abortonclose

 51     balance     roundrobin

 52     option      httpchk GET /index.html

 53     server      webapp3 172.24.8.32:80  weight 6 check inter 2000 rise 2 fall 3

 54 [root@haproxy_master ~]# scp /etc/haproxy/haproxy.cfg root@172.24.8.11:/etc/haproxy/haproxy.cfg		#将配置文件复制至HAProxy备节点

5.3 打开转发

  1 [root@haproxy_master ~]# vi /etc/sysctl.conf

  2 net.ipv4.ip_nonlocal_bind = 1

  3 [root@haproxy_master ~]# sysctl -p

注意：绑定非本机的IP必须在sysctl.conf文件中配置。

六启动服务

  1 [root@webapp1 ~]# systemctl start httpd

提示：三个节点的httpd服务均启动。

  1 [root@haproxy_master ~]# service haproxy start

提示：必须先启动HAProxy服务，之后启动Keepalived，因为Keepalived会先检测HAProxy服务进程。

  1 [root@haproxy_master ~]# service keepalived start

七验证测试

7.1 高可用验证

  1 [root@haproxy_master ~]# ip add

  1 [root@haproxy_master ~]# service haproxy stop		#停止HAProxy主节点的HAProxy进程

  2 [root@haproxy_master ~]# tail -f /var/log/messages	#观察HAProxy主节点日志

  1 [root@haproxy_slave ~]# ip addr				#查看备HAProxy节点的IP

结论：通过测试可知当主HAProxy节点服务异常，Keepalived会检测到，同时HAProxy会将vip从主节点移除，备HAProxy会接管。

  1 [root@haproxy_master ~]# service haproxy start

结论：由于配置了非抢占模式，主HAProxy恢复进程之后，依旧由备HAProxy提供服务，vip不会切回至主HAProxy。

7.2 负载均衡测试

主机hosts中添加如下解析：

  1 172.24.8.100	www.lz.com

  2 172.24.8.100	static.lz.com

  3 172.24.8.100	video.lz.com

浏览器分别访问不用的三个域名：