HAProxy+Keepalived高可用负载均衡

一 基础准备

1.1 部署环境及说明

系统OS：CentOS 6.8 64位

HAProxy软件：HA-Proxy version 1.5.18

Keepalived软件：keepalived-1.3.6.tar.gz

官方链接：http://www.haproxy.org/ （国内可能无法打开）

下载连接：http://pkgs.fedoraproject.org/repo/pkgs/haproxy/

部署说明：当用户访问对应的域名时，HAProxy能将请求发送到对应的后端主机上，同时当主HAProxy服务器发生故障后，能立刻将负载均衡服务切换到备用HAProxy服务器上。

主机名	主机IP地址	集群角色	虚机IP/域名
master	172.24.8.10	主HAProxy服务器	172.24.8.100
backup	172.24.8.11	备HAProxy服务器
webapp1	172.24.8.30	后端web服务器	www.lz.com
webapp2	172.24.8.31		static.lz.com
webapp3	172.24.8.32		video.lz.com

1.2 架构规划

二 后端httpd集群部署

2.1 部署httpd集群

  1 [root@webapp1 ~]# yum -y install httpd
  2 [root@webapp1 ~]# vi /var/www/html/index.html
  3 This is my www.lz.com!
  4 [root@webapp1 ~]# vi /var/www/html/index.html
  5 This is my static.lz.com!
  6 [root@webapp1 ~]# vi /var/www/html/index.html
  7 This is my video.lz.com!
  8 [root@webapp1 ~]# systemctl start httpd.service
  9 [root@webapp1 ~]# systemctl enable httpd.service
 10 [root@webapp1 ~]# systemctl stop firewalld.service
 11 [root@webapp1 ~]# systemctl disable firewalld.service
 12 [root@webapp1 ~]# vi /etc/selinux/config
 13 SELINUX=disabled
 14 [root@webapp1 ~]# setenforce 0			#关闭SELinux及防火墙

 

注意：后端real server节点都需要安装，本环境针对httpd简单安装即可，无需过多配置。

三 基础NTP部署

3.1 NTP部署

  1 [root@webapp1 ~]# yum -y install ntp
  2 [root@webapp1 ~]# systemctl start  ntpd.service

 

建议：替换附件中的ntp配置文件，建议采用阿里云时钟进行同步。

注意：为了保证集群的稳定性，强烈建议在所有节点均部署NTP同步服务，保证所有时钟一致。

四 Keepalived部署

4.1 编译环境

安装基础环境及依赖：

  1 # yum -y install gcc gcc-c++ make kernel-devel kernel-tools kernel-tools-libs kernel libnl libnl-devel libnfnetlink-devel openssl-devel wget openssh-clients

4.2 安装Keepalived

  1 [root@haproxy_master ~]# wget http://www.keepalived.org/software/keepalived-1.3.6.tar.gz
  2 [root@haproxy_master ~]# tar -zxvf keepalived-1.3.6.tar.gz
  3 [root@haproxy_master ~]# cd keepalived-1.3.6/
  4 [root@lvsmaster keepalived-1.3.6]# ./configure --prefix=/usr/local/keepalived
  5 [root@haproxy_master keepalived-1.3.6]# make && make install

 

注意：CentOS6.8安装高于1.3.6版本会出现未知错误。

4.3 添加Keepalived启动相关服务

  1 [root@haproxy_master ~]# mkdir /etc/keepalived
  2 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/
  3 [root@haproxy_master ~]# cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
  4 [root@haproxy_master ~]# cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
  5 [root@haproxy_master ~]# vi /etc/init.d/keepalived	#创建Keepalived启动脚本，见附件
  6 [root@haproxy_master ~]# chmod u+x /etc/rc.d/init.d/keepalived

 

4.4 配置Keepalived

  1 [root@haproxy_master ~]# vi /etc/keepalived/keepalived.conf
  2 global_defs {
  3    notification_email {
  4 #……
  5    }
  6    notification_email_from Alexandre.Cassen@firewall.loc
  7    smtp_server 192.168.200.1
  8    smtp_connect_timeout 30
  9    router_id LVS_DEVEL
 10 }
 11
 12 vrrp_script check_haproxy {
 13     script "/usr/bin/killall -0 haproxy"
 14     interval    2
 15     weight      21
 16     }
 17
 18 vrrp_instance HAProxy_HA {
 19     state BACKUP			#在HAProxy主备均设置为BACKUP
 20     interface eth0
 21     virtual_router_id 80
 22     priority 100
 23     advert_int 2
 24     nopreempt				#不抢占模式
 25     authentication {
 26         auth_type PASS
 27         auth_pass 1111
 28     }
 29
 30     notify_master "/etc/keepalived/master.sh"
 31     notify_backup "/etc/keepalived/backup.sh"
 32     notify_fault "/etc/keepalived/fault.sh"
 33
 34     track_script {
 35     check_haproxy
 36     }
 37
 38     virtual_ipaddress {
 39         172.24.8.100 dev eth0
 40     }
 41 }

 

4.5 编写Keepalived脚本

  1 [root@haproxy_master ~]# vi /etc/keepalived/master.sh
  2 #!/bin/bash
  3 LOGFILE=/var/log/keepalived-mysql-state/log
  4 date >>$LOGFILE
  5 echo "[Master]" >>$LOGFILE
  6 [root@haproxy_master ~]# vi /etc/keepalived/backup.sh
  7 #!/bin/bash
  8 LOGFILE=/var/log/keepalived-mysql-state/log
  9 date >>$LOGFILE
 10 echo "[BACKUP]" >>$LOGFILE
 11 [root@haproxy_master ~]# vi /etc/keepalived/fault.sh
 12 #!/bin/bash
 13 LOGFILE=/var/log/keepalived-mysql-state/log
 14 date >>$LOGFILE
 15 echo "[FAULT]" >>$LOGFILE
 16 [root@haproxy_master ~]# chmod u+x /etc/keepalived/backup.sh
 17 [root@haproxy_master ~]# chmod u+x /etc/keepalived/master.sh
 18 [root@haproxy_master ~]# chmod u+x /etc/keepalived/fault.sh

 

提示：本环境采用测试脚本，真实环境建议采用自动发送邮件通知运维员的脚本。

4.5 Bakcup节点配置

  1 [root@haproxy_master ~]# scp /etc/keepalived/keepalived.conf 172.24.8.11:/etc/keepalived/keepalived.conf    #将配置好的Master节点的配置文件复制到Backup节点
  2 [root@haproxy_slave ~]# vi /etc/keepalived/keepalived.conf
  3 state BACKUP
  4 priority 80

 

注意：在HAProxy备节点也设置为BACKUP，priority修改为低于HAProxy主角色的优先级即可，同时去掉nopreempt。

  1 [root@haproxy_master ~]# scp /etc/keepalived/*.sh 172.24.8.11:/etc/keepalived/
  2 #将对应的脚本也复制至backup节点。

 

五 HAProxy部署

5.1 HAProxy安装

  1 [root@haproxy_master ~]# yum -y install haproxy

提示：需要在主备HAProxy节点都安装。

5.2 HAProxy配置

  1 [root@haproxy_master ~]# vi /etc/haproxy/haproxy.cfg
  2 global
  3     log         127.0.0.1 local0 info
  4     chroot      /var/lib/haproxy
  5     pidfile     /var/run/haproxy.pid
  6     maxconn     4096
  7     user        haproxy
  8     group       haproxy
  9     daemon
 10     nbproc      1
 11 defaults
 12     mode                    http
 13     log                     global
 14     retries                 3
 15     timeout connect         5s
 16     timeout client          30s
 17     timeout server          30s
 18     timeout check           2s
 19 frontend        www
 20     bind        172.24.8.100:80
 21     mode        http
 22     option      httplog
 23     option      forwardfor
 24     log         global
 25
 26     acl host_www        hdr_dom(host)   -i  www.lz.com	#配置不同域名分发不同后端策略
 27     acl host_static     hdr_dom(host)   -i  static.lz.com
 28     acl host_video      hdr_dom(host)   -i  video.lz.com
 29
 30     use_backend     server_www      if      host_www	#配置不同域名分发不同后端策略
 31     use_backend     server_static   if      host_static
 32     use_backend     server_video    if      host_video
 33 backend         server_www		#后端真是服务器
 34     mode        http
 35     option      redispatch
 36     option      abortonclose
 37     balance     roundrobin
 38     option      httpchk GET /index.html
 39     server      webapp1 172.24.8.30:80  weight 6 check inter 2000 rise 2 fall 3
 40 backend         server_static		#后端真是服务器
 41     mode        http
 42     option      redispatch
 43     option      abortonclose
 44     balance     roundrobin
 45     option      httpchk GET /index.html
 46     server      webapp2 172.24.8.31:80  weight 6 check inter 2000 rise 2 fall 3
 47 backend         server_video		#后端真是服务器
 48     mode        http
 49     option      redispatch
 50     option      abortonclose
 51     balance     roundrobin
 52     option      httpchk GET /index.html
 53     server      webapp3 172.24.8.32:80  weight 6 check inter 2000 rise 2 fall 3
 54 [root@haproxy_master ~]# scp /etc/haproxy/haproxy.cfg root@172.24.8.11:/etc/haproxy/haproxy.cfg		#将配置文件复制至HAProxy备节点

 

5.3 打开转发

  1 [root@haproxy_master ~]# vi /etc/sysctl.conf
  2 net.ipv4.ip_nonlocal_bind = 1
  3 [root@haproxy_master ~]# sysctl -p

 

注意：绑定非本机的IP必须在sysctl.conf文件中配置。

六 启动服务

  1 [root@webapp1 ~]# systemctl start httpd

提示：三个节点的httpd服务均启动。

  1 [root@haproxy_master ~]# service haproxy start

提示：必须先启动HAProxy服务，之后启动Keepalived，因为Keepalived会先检测HAProxy服务进程。

  1 [root@haproxy_master ~]# service keepalived start

七 验证测试

7.1 高可用验证

  1 [root@haproxy_master ~]# ip add

  1 [root@haproxy_master ~]# service haproxy stop		#停止HAProxy主节点的HAProxy进程
  2 [root@haproxy_master ~]# tail -f /var/log/messages	#观察HAProxy主节点日志

 

  1 [root@haproxy_slave ~]# ip addr				#查看备HAProxy节点的IP

结论：通过测试可知当主HAProxy节点服务异常，Keepalived会检测到，同时HAProxy会将vip从主节点移除，备HAProxy会接管。

  1 [root@haproxy_master ~]# service haproxy start

结论：由于配置了非抢占模式，主HAProxy恢复进程之后，依旧由备HAProxy提供服务，vip不会切回至主HAProxy。

7.2 负载均衡测试

主机hosts中添加如下解析：

  1 172.24.8.100	www.lz.com
  2 172.24.8.100	static.lz.com
  3 172.24.8.100	video.lz.com

 

浏览器分别访问不用的三个域名：