LVS主从部署配置和使用

　　LVS是Linux Virtual Server的简写，意即Linux虚拟服务器，是一个虚拟的服务器集群系统。本项目在1998年5月由章文嵩博士成立，是中国国内最早出现的自由软件项目之一。

　　LVS是Linux Virtual Server的简称，也就是Linux虚拟服务器, 是一个由章文嵩博士发起的自由软件项目，它的官方站点是www.linuxvirtualserver.org。现在LVS已经是 Linux标准内核的一部分，在Linux2.4内核以前，使用LVS时必须要重新编译内核以支持LVS功能模块，但是从Linux2.4内核以后，已经完全内置了LVS的各个功能模块，无需给内核打任何补丁，可以直接使用LVS提供的各种功能。
使用LVS技术要达到的目标是：通过LVS提供的负载均衡技术和Linux操作系统实现一个高性能、高可用的服务器群集，它具有良好可靠性、可扩展性和可操作性。从而以低廉的成本实现最优的服务性能。

　　使用LVS架设的服务器集群系统有三个部分组成：最前端的负载均衡层，用Load Balancer表示，中间的服务器群组层，用Server Array表示，最底端的数据共享存储层，用Shared Storage表示，在用户看来，所有的内部应用都是透明的，用户只是在使用一个虚拟服务器提供的高性能服务。

　　接下来开始部署配置LVS主从：

　　免责声明：小伙伴在上生产环境前先测试有没有问题哈！！！

　　

本次试验共计使用4台虚拟机，都是centos6.3.

192.168.146.164

192.168.146.160

192.168.146.162

192.168.146.163

具体实验环境配置如下：

192.168.146.101  LVS_VIP

192.168.146.164  LVS_MASTER

192.168.146.160  LVS_SALVE

192.168.146.162  WEB1

192.168.146.163  WEB2

192.168.146.164  LVS_MASTER：

安装IPVSADM：

[root@localhost ~]# yum -y install ipvsadm

为了测试方便，暂时关闭防火墙：

[root@localhost ~]# service iptables stop

安装KeepAlived：

[root@localhost ~]# cd /usr/src

[root@localhost src]# yum -y install openssl-devel

root@localhost src]# wget http://www.keepalived.org/software/keepalived-1.2.7.tar.gz

[root@localhost src]# wget http://mirror.centos.org/centos/6/os/x86_64/Packages/popt-static-1.13-7.el6.x86_64.rpm

[root@localhost src]# yum -y install popt-static-1.13-7.el6.x86_64.rpm

[root@localhost src]# yum -y install kernel-devel make gcc openssl-devel libnl* popt*

[root@localhost src]# ln -s /usr/src/kernels/2.6.32-220.13.1.el6.x86_64/ /usr/src/linux

[root@localhost src]# tar zxvf keepalived-1.2.7.tar.gz

[root@localhost src]# cd keepalived-1.2.7

[root@localhost keepalived-1.2.7]# ./configure --with-kernel-dir=/usr/src/kernels/2.6.32-358.2.1.el6.x86_64/

[root@localhost keepalived-1.2.7]# make && make install

[root@localhost keepalived-1.2.7]# cp /usr/local/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/

[root@localhost keepalived-1.2.7]# cp /usr/local/etc/sysconfig/keepalived /etc/sysconfig/

[root@localhost keepalived-1.2.7]# mkdir /etc/keepalived

[root@localhost keepalived-1.2.7]# cp /usr/local/etc/keepalived/keepalived.conf /etc/keepalived/

[root@localhost keepalived-1.2.7]# cp /usr/local/sbin/keepalived /usr/sbin/

KeepAlived安装完毕，然后进行配置。

打开IP Forward 功能（LVS现有三种负载均衡规则都需要打开此功能，如果不打开此功能，下面的配置配得再好都无济于事。）

打开后修改里面“net.ipv4.ip_forward = 1”

[root@localhost keepalived-1.2.7]# cat /etc/sysctl.conf

# Kernel sysctl configuration file for Red Hat Linux

#

# For binary values, 0 is disabled, 1 is enabled.  See sysctl(8) and

# sysctl.conf(5) for more details.

# Controls IP packet forwarding

net.ipv4.ip_forward = 1

执行如下命令使设置立即生效：

[root@localhost keepalived-1.2.7]# sysctl -p

net.ipv4.ip_forward = 1

net.ipv4.conf.default.rp_filter = 1

net.ipv4.conf.default.accept_source_route = 0

kernel.sysrq = 0

kernel.core_uses_pid = 1

net.ipv4.tcp_syncookies = 1

error: "net.bridge.bridge-nf-call-ip6tables" is an unknown key

error: "net.bridge.bridge-nf-call-iptables" is an unknown key

error: "net.bridge.bridge-nf-call-arptables" is an unknown key

kernel.msgmnb = 65536

kernel.msgmax = 65536

kernel.shmmax = 68719476736

kernel.shmall = 4294967296

配置keepalived:

配置文件在这个位置： /etc/keepalived/keepalived.conf

[root@localhost keepalived-1.2.7]# cat /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

notification_email {

test@sina.com

}

notification_email_from admin@test.com

smtp_server 127.0.0.1

smtp_connect_timeout 30

router_id LVS_DEVEL

}

vrrp_instance VI_1 {

state MASTER

interface eth1

virtual_router_id 60

priority 100

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

virtual_ipaddress {

192.168.146.101

}

}

virtual_server 192.168.146.101 8080 {

delay_loop 6

lb_algo rr

lb_kind DR

nat_mask 255.255.255.0

persistence_timeout 50

protocol TCP

real_server 192.168.146.162 8080 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

}

}

real_server 192.168.146.163 8080 {

weight 1

TCP_CHECK {

connect_timeout 3

nb_get_retry 3

delay_before_retry 3

}

}

}

至此，keepalived配置完成。

把keepalived加入开机自启动，并开启keepalived服务：

[root@localhost keepalived-1.2.7]# chkconfig keepalived on

[root@localhost keepalived-1.2.7]# service keepalived start

Starting keepalived:                                       [  OK  ]

查看keepalived进程：

[root@localhost keepalived-1.2.7]# ps aux | grep keepalived

root      1117  0.0  0.0  42172  1000 ?        Ss   14:37   0:00 keepalived -D

root      1119  0.0  0.2  44400  2296 ?        S    14:37   0:00 keepalived -D

root      1120  0.0  0.1  44276  1664 ?        S    14:37   0:00 keepalived -D

root      1165  0.0  0.0 103268   884 pts/5    S+   14:48   0:00 grep keepalived

查看虚拟IP是否加上：

[root@localhost keepalived-1.2.7]# ip a

1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN

link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00

inet 127.0.0.1/8 scope host lo

inet6 ::1/128 scope host

valid_lft forever preferred_lft forever

2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000

link/ether 00:0c:29:b2:88:f7 brd ff:ff:ff:ff:ff:ff

inet 192.168.146.164/24 brd 192.168.146.255 scope global eth1

inet 192.168.146.101/32 scope global eth1

inet6 fe80::20c:29ff:feb2:88f7/64 scope link

valid_lft forever preferred_lft forever

红色部分显示已经自动加上虚拟IP了。

至此，LVS_MASTER服务器已经配置好并启动了。

接下来开始配置部署LVS从服务器：

192.168.146.160  LVS_SALVE：

具体过程同192.168.146.164  LVS_MASTER！！

接下里配置部署两台web服务器：

配置部署192.168.146.162 web1:

配置虚拟IP启动脚本:

[root@localhost ~]# cat /etc/init.d/realserver.sh

SNS_VIP=192.168.146.101

. /etc/rc.d/init.d/functions

case "$1" in

start)

ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP

/sbin/route add -host $SNS_VIP dev lo:0

echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce

sysctl -p >/dev/null 2>&1

echo "RealServer Start OK"

;;

stop)

ifconfig lo:0 down

route del $SNS_VIP >/dev/null 2>&1

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce

echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore

echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce

echo "RealServer Stoped"

;;

*)

echo "Usage: $0 {start|stop}"

exit 1

esac

exit 0

配置部署tomcat：

[root@localhost ~]# yum -y install tomcat6 tomcat6-webapps tomcat6-admin-webapps

[root@localhost ~]# chkconfig tomcat6 on

[root@localhost ~]# service tomcat6 start

Starting tomcat6:                                          [  OK  ]

为了测试方便暂时关闭防火墙：

[root@localhost ~]# service iptables stop

打开浏览器测试：http://192.168.146.142:8080

会看到TOMCAT的熟悉页面了。

为了测试负载均衡，我们将这个页面改下，以更好的标识这个网页是本服务器的

Tomcat6安装目录位于/usr/share/tomcat6，所以我们要编辑tomcat下的webapps/ROOT/index.html这个文件。

[root@localhost ~]# cd /usr/share/tomcat6/webapps/ROOT/

[root@localhost ROOT]# cat /dev/null > index.html

[root@localhost ROOT]# cat index.html

web1 192.168.146.162

启动虚拟IP的脚本：

[root@localhost ROOT]# sh /etc/init.d/realserver.sh start

RealServer Start OK

eth1      Link encap:Ethernet  HWaddr 00:0C:29:1F:0F:F3

inet addr:192.168.146.162  Bcast:192.168.146.255  Mask:255.255.255.0

inet6 addr: fe80::20c:29ff:fe1f:ff3/64 Scope:Link

UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

RX packets:89543 errors:0 dropped:0 overruns:0 frame:0

TX packets:47914 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:1000

RX bytes:112074876 (106.8 MiB)  TX bytes:2870905 (2.7 MiB)

lo        Link encap:Local Loopback

inet addr:127.0.0.1  Mask:255.0.0.0

inet6 addr: ::1/128 Scope:Host

UP LOOPBACK RUNNING  MTU:16436  Metric:1

RX packets:22 errors:0 dropped:0 overruns:0 frame:0

TX packets:22 errors:0 dropped:0 overruns:0 carrier:0

collisions:0 txqueuelen:0

RX bytes:1408 (1.3 KiB)  TX bytes:1408 (1.3 KiB)

lo:0      Link encap:Local Loopback

inet addr:192.168.146.101  Mask:255.255.255.255

UP LOOPBACK RUNNING  MTU:16436  Metric:1

运行后会看到网络有了一个虚拟IP，即红色标注部分。

去LVS_MASTER服务器的终端查看下ipvsadm，查看已经连接上了WEB1服务器（即红色标注部分）：

[root@localhost keepalived-1.2.7]# ipvsadm -ln

IP Virtual Server version 1.2.1 (size=4096)

Prot LocalAddress:Port Scheduler Flags

-> RemoteAddress:Port           Forward Weight ActiveConn InActConn

TCP  192.168.146.101:8080 rr persistent 50

-> 192.168.146.162:8080         Route   1      0          0

已经可以看到有服务器加入进来了。

此时我们访问网页http://192.168.146.101:8080，出现界面显示web1 192.168.146.162；或者使用命令行测试转发情况都可以查看。

配置部署192.168.146.163 web2:

具体配置部署过程同web1！！