LVS+keepalived实现负载均衡

背景:
       
随着你的网站业务量的增长你网站的服务器压力越来越大？需要负载均衡方案！商业的硬件如F5又太贵，你们又是创业型互联公司如何有效节约成本，节省不必要
的浪费？同时实现商业硬件一样的高性能高可用的功能？有什么好的负载均衡可伸张可扩展的方案吗？答案是肯定的！有！我们利用
LVS+Keepalived基于完整开源软件的架构可以为你提供一个负载均衡及高可用的服务器。

LVS+Keepalived 介绍

LVS

        LVS是Linux Virtual
Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项
目之一.目前有三种IP负载均衡技术（VS/NAT、VS/TUN和VS/DR）；

八种调度算法（rr,wrr,lc,wlc,lblc,lblcr,dh,sh）。

Keepalvied

        Keepalived在这里主要用作RealServer的健康状态检查以及LoadBalance主机和BackUP主机之间failover的实现

二. 网站负载均衡构架图

IP信息列表：

名称                             IP

LVS-DR-Master        61.164.122.6       

LVS-DR-BACKUP        61.164.122.7       

LVS-DR-VIP           61.164.122.8       

WEB1-Realserver      61.164.122.9       

WEB2-Realserver      61.164.122.10       

GateWay              61.164.122.1 

三. 安装LVS和Keepalvied软件包

1. 下载相关软件包

    #mkdir /usr/local/src/lvs
    #cd /usr/local/src/lvs
    #wget http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
    #wget http://www.keepalived.org/software/keepalived-1.1.15.tar.gz  

2.安装LVS和Keepalived

    #lsmod |grep ip_vs
    #uname -r
    2.6.-.el5PAE
    #ln -s /usr/src/kernels/2.6.-.el5PAE-i686/  /usr/src/linux  

    #tar zxvf ipvsadm-1.24.tar.gz
    #cd ipvsadm-1.24
    #make && make install
    #find / -name ipvsadm  # 查看ipvsadm的位置  

     #tar zxvf keepalived-1.1..tar.gz
    #cd keepalived-1.1.
    #./configure  && make && make install
    #find / -name keepalived  # 查看keepalived位置                  

     #cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/
    #cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/
    #mkdir /etc/keepalived
    #cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/
    #cp /usr/local/sbin/keepalived /usr/sbin/
    #service keepalived start|stop     #做成系统启动服务方便管理.  

四. 配置LVS实现负载均衡
  1． LVS-DR，配置LVS脚本实现负载均衡

    #vi /usr/local/sbin/lvs-dr.sh
    #!/bin/bash
      # description: start LVS of DirectorServer
      #Written by :NetSeek http://www.linuxtone.org  

      GW=61.164.122.1
      # website director vip.
      SNS_VIP=61.164.122.8
      SNS_RIP1=61.164.122.9
      SNS_RIP2=61.164.122.10  

      ./etc/rc.d/init.d/functions  

      logger $ called with $  

      case "$1" in  

      start)
         # set squid vip
         /sbin/ipvsadm --set
                 /sbin/ifconfig eth0: $SNS_VIP broadcast $SNS_VIP netmask 255.255.255.255
      broadcast $SNS_VIP up
         /sbin/route add -host $SNS_VIP dev eth0:
         /sbin/ipvsadm -A -t $SNS_VIP: -s wrr -p
         /sbin/ipvsadm -a -t $SNS_VIP: -r $SNS_RIP1: -g -w
         /sbin/ipvsadm -a -t $SNS_VIP: -r $SNS_RIP2: -g -w
         touch /var/lock/subsys/ipvsadm >/dev/null >&  

     ;;
      stop)
         /sbin/ipvsadm -C
         /sbin/ipvsadm -Z
         ifconfig eth0: down
         ifconfig eth0: down
         route del $SNS_VIP
         route del $SS_VIP
         rm -rf /var/lock/subsys/ipvsadm >/dev/null >&
         echo "ipvsadm stoped"
         ;;  

      status)  

         if [ ! -e /var/lock/subsys/ipvsadm ];then
              echo "ipvsadm stoped"
              exit
         else
              echo "ipvsadm OK"
         fi
         ;;  

      *)
        echo "Usage: $0 {start|stop|status}"
         exit
      esac
      exit   

． 配置Realserver脚本.
Java代码  收藏代码

    #vi /usr/local/sbin/realserver.sh
    #!/bin/bash
    # description: Config realserver lo and apply noarp
    #Written by :NetSeek http://www.linuxtone.org  

    SNS_VIP=61.164.122.8  

    . /etc/rc.d/init.d/functions  

    case "$1" in
    start)
           ifconfig lo: $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
           /sbin/route add -host $SNS_VIP dev lo:
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_announce
           echo "" >/proc/sys/net/ipv4/conf/all/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/all/arp_announce
           sysctl -p >/dev/null >&
           echo "RealServer Start OK"  

           ;;
    stop)
           ifconfig lo: down
           route del $SNS_VIP >/dev/null >&
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_announce
           echo "" >/proc/sys/net/ipv4/conf/all/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/all/arp_announce
           echo "RealServer Stoped"
           ;;
    *)
           echo "Usage: $0 {start|stop}"
           exit
    esac  

    exit   

或者采用secondary ip address方式配置

    #vi /usr/local/sbin/lvs-dr.sh
    #!/bin/bash
      # description: start LVS of DirectorServer
      #Written by :NetSeek http://www.linuxtone.org  

      GW=61.164.122.1
      # website director vip.
      SNS_VIP=61.164.122.8
      SNS_RIP1=61.164.122.9
      SNS_RIP2=61.164.122.10  

      ./etc/rc.d/init.d/functions  

      logger $ called with $  

      case "$1" in  

      start)
         # set squid vip
         /sbin/ipvsadm --set
                 /sbin/ifconfig eth0: $SNS_VIP broadcast $SNS_VIP netmask 255.255.255.255
      broadcast $SNS_VIP up
         /sbin/route add -host $SNS_VIP dev eth0:
         /sbin/ipvsadm -A -t $SNS_VIP: -s wrr -p
         /sbin/ipvsadm -a -t $SNS_VIP: -r $SNS_RIP1: -g -w
         /sbin/ipvsadm -a -t $SNS_VIP: -r $SNS_RIP2: -g -w
         touch /var/lock/subsys/ipvsadm >/dev/null >&  

     ;;
      stop)
         /sbin/ipvsadm -C
         /sbin/ipvsadm -Z
         ifconfig eth0: down
         ifconfig eth0: down
         route del $SNS_VIP
         route del $SS_VIP
         rm -rf /var/lock/subsys/ipvsadm >/dev/null >&
         echo "ipvsadm stoped"
         ;;  

      status)  

         if [ ! -e /var/lock/subsys/ipvsadm ];then
              echo "ipvsadm stoped"
              exit
         else
              echo "ipvsadm OK"
         fi
         ;;  

      *)
        echo "Usage: $0 {start|stop|status}"
         exit
      esac
      exit   

． 配置Realserver脚本.
Java代码  收藏代码

    #vi /usr/local/sbin/realserver.sh
    #!/bin/bash
    # description: Config realserver lo and apply noarp
    #Written by :NetSeek http://www.linuxtone.org  

    SNS_VIP=61.164.122.8  

    . /etc/rc.d/init.d/functions  

    case "$1" in
    start)
           ifconfig lo: $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
           /sbin/route add -host $SNS_VIP dev lo:
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_announce
           echo "" >/proc/sys/net/ipv4/conf/all/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/all/arp_announce
           sysctl -p >/dev/null >&
           echo "RealServer Start OK"  

           ;;
    stop)
           ifconfig lo: down
           route del $SNS_VIP >/dev/null >&
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/lo/arp_announce
           echo "" >/proc/sys/net/ipv4/conf/all/arp_ignore
           echo "" >/proc/sys/net/ipv4/conf/all/arp_announce
           echo "RealServer Stoped"
           ;;
    *)
           echo "Usage: $0 {start|stop}"
           exit
    esac  

    exit   

3. 启动lvs-dr脚本和realserver启本，在DR上可以查看LVS当前状态:

#watch ipvsadm –ln  

五．利用Keepalvied实现负载均衡和和高可用性
  1.配置在主负载均衡服务器上配置keepalived.conf

#vi /etc/keepalived/keepalived.conf  

    ! Configuration File for keepalived  

    global_defs {
       notification_email {
             cnseek@gmail.com
       }
       notification_email_from sns-lvs@gmail.com
       smtp_server 127.0.0.1
      # smtp_connect_timeout
       router_id LVS_DEVEL
    }  

    #  written by :netseek
    # VIP1
    vrrp_instance VI_1 {
        state MASTER             #备份服务器上将MASTER改为BACKUP
        interface eth0
        virtual_router_id
        priority     # 备份服务上将100改为99
        advert_int
        authentication {
            auth_type PASS
            auth_pass
        }
        virtual_ipaddress {
            61.164.122.8
            #(如果有多个VIP，继续换行填写.)
        }
    }  

    virtual_server 61.164.122.8  {
        delay_loop                   #(每隔10秒查询realserver状态)
        lb_algo wrr                  #(lvs 算法)
        lb_kind DR                  #(Direct Route)
        persistence_timeout         #(同一IP的连接60秒内被分配到同一台realserver)
        protocol TCP                #(用TCP协议检查realserver状态)  

        real_server 61.164.122.9  {
            weight                #(权重)
            TCP_CHECK {
            connect_timeout        #(10秒无响应超时)
            nb_get_retry
            delay_before_retry
            connect_port
            }
        }
        real_server 61.164.122.10  {
            weight
            TCP_CHECK {
            connect_timeout
            nb_get_retry
            delay_before_retry
            connect_port
            }
         }
    }  

2. BACKUP服务器同上配置，先安装lvs再按装keepalived,仍后配置/etc/keepalived/keepalived.conf，只需将红色标示的部分改一下即可.

    vi /etc/rc.local
    #/usr/local/sbin/lvs-dr.sh  将lvs-dr.sh这个脚本注释掉。步
    #/usr/local/sbin/lvs-dr.sh stop 停止lvs-dr脚本
    #/etc/init.d/keepalived start  启动keepalived 服务，keepalived就能利用  

keepalived.conf 配置文件，实现负载均衡和高可用.

4. 查看lvs服务是否正常

   #watch ipvsadm –ln  

IP Virtual Server version 1.2. (size=)
Prot LocalAddressort Scheduler Flags
  -> RemoteAddressort           Forward Weight ActiveConn InActConn
TCP  61.164.122.8: wrr persistent
  -> 61.164.122.10:            Route
  -> 61.164.122.9:             Route                    

#tail –f /var/log/message  

监听日志，查看状态，测试LVS负载均衡及高可用性是否有效。

5．停Master服务器的keepalived服务，查看BAKCUP服务器是否能正常接管服务。

四．相关参考

  1．LVS 基础知识汇总

LVS的算法介绍               http://www.linuxtone.org/viewthread.php?tid=69

学习LVS的三种转发模式       http://www.linuxtone.org/viewthread.php?tid=77

LVS中的IP负载均衡技术       http://www.linuxtone.org/viewthread.php?tid=68

更多的请到http://www.linuxtone.org 负载均衡版查看

Keepalived 相关参考资料。

   http://www.keepalived.org/documentation.html