nginx配置负载

一.系统优化

1、修改/etc/sysctl.conf，优化tcp连接数

net.ipv4.tcp_fin_timeout = 30

net.ipv4.tcp_keepalive_time = 1200

net.ipv4.tcp_syncookies = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_tw_recycle = 1

net.ipv4.ip_local_port_range = 1024 65000

net.ipv4.tcp_max_syn_backlog = 8192

net.ipv4.tcp_max_tw_buckets = 5000

2、修改/etc/security/limits.conf，修改最大文件打开数为65535

添加如下两行：

*      soft  nofile       65535

*      hard nofile       65535

*      soft  nproc       16384

*      hard nproc       16384

二.安装JDK

chmod +x jdk-6u38-linux-x64.bin

# ./jdk-6u38-linux-x64.bin

#添加环境变量

# vi /etc/profile

export JAVA_HOME=/kjt/java/jdk1.6.0_38

export CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

export PATH=$JAVA_HOME/bin:$PATH

#使环境变量生效

# source /etc/profile

#查看java版本

# java -version

java version "1.6.0_38"

Java(TM) SE Runtime Environment (build 1.6.0_27-b07)

Java HotSpot(TM) 64-Bit Server VM (build 20.2-b06, mixed mode)

三.安装Tomcat

1、确保防火墙已关闭

#service iptables stop

#chkconfig iptables off

#vi /etc/selinux/config

selinux=disabled

2、安装tomcat

# tar zxvf apache-tomcat-6.0.36.tar.gz

# cd apache-tomcat-6.0.36

#修改/bin/catalina.sh，在cygwin=false前加路径

export JAVA_HOME=/kjt/java/jdk1.6.0_38/

export PATH=$JAVA_HOME/bin:$PATH

export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export TOMCAT_HOME=/kjt/java/apache-tomcat-6.0.36

export CATALINA_HOME=/kjt/java/apache-tomcat-6.0.36

export CATALINA_BASE=/kjt/java/apache-tomcat-6.0.36

3、修改/conf/server.xml，做tomcat线程池优化

<Connector port="8081" protocol="HTTP/1.1"

connectionTimeout="15000" maxThreads="500" minSpareThreads="50" maxSpareTheads="250" maxProcessors="600"

acceptCount="1000" enableLookups="false" compression="on"

redirectPort="8443" URIEncoding="utf-8" useBodyEncodingForURI="true"/>

<!-- A "Connector" using the shared thread pool-->

4、启动tomcat

cd /kjt/java/apache-tomcat-6.0.35/bin

./startup

5、查看tomcat进程

ps –ef|grep tomcat

四.Nginx配置

1.确定Nginx所需pcre库、openssl库等依赖包已经安装，若没有安装，安装nginx过程中将报错。

2.安装Nginx

tar zxvf nginx-1.0.15.tar.gz

cd nginx-1.0.15/

./configure  --prefix=/kjt/app/nginx --with-http_stub_status_module --with-http_ssl_module

make

make install

3.Nginx配置文件如下：

4.测试配置文件

/kjt/app/nginx/sbin/nginx -t

显示以下信息为正确的

the configuration file /usr/local/nginx/conf/nginx.conf syntax is ok

configuration file /usr/local/nginx/conf/nginx.conf test is successful

5、启动nginx服务

/kjt/app/nginx/sbin/nginx

6、查看nginx进程

ps -ef|grep nginx

五.安装配置Keepalived

1、安装keepalived

tar -zxvf keepalived-1.2.7.tar.gz

cd keepalived-1.2.7

./configure --prefix=/kjt/app/keepalived

make

make install

2、将keepalived设置为开机启动

cp /kjt/app/keepalived/etc/rc.d/init.d/keepalived /etc/rc.d/init.d/

cp/kjt/app/keepalived/etc/sysconfig/keepalived /etc/sysconfig/

cp /kjt/app/keepalived/sbin/keepalived /usr/sbin/

chkconfig --add keepalived

chkconfig keepalived on

3.修改配置文件为以下内容:

mkdir /etc/keepalived

cp /kjt/app/keepalived/etc/keepalived/keepalived.conf /etc/keepalived/

vim  /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

router_id Nginx_Id_1

}

vrrp_script Monitor_Nginx {

script "/kjt/app/keepalived/etc/keepalived/samples/monitor_nginx.sh
"  #脚本存放位置

interval 2     #检查间隔为2秒

weight 2

}

vrrp_instance VI_1 {

state
BACKUP  #设置nopreempt，表示之前挂掉的master重新恢复后不再切换为master，直接成为BACKUP，nopreempt只能配置在BACKUP状态下

nopreempt

interface eth0

virtual_router_id 51

priority 100    #优先级较高的成为MASTER

advert_int 5   #健康检查间隔为5秒

authentication {

auth_type PASS

auth_pass kjt2013

}

virtual_ipaddress {

10.46.86.60

}

track_script {

Monitor_Nginx

}

}

注：monitor_nginx.sh为监控nginx进程的脚本,内容如下：当检测到nginx主进程挂掉之后，系统将自动重启nginx，若nginx主进程重启失败，延迟5秒后杀掉keepalived进程，进行切换。

#!/bin/bash

if [ "$(ps -ef | grep "nginx:
master process"| grep -v grep )" == "" ]

then

/kjt/app/nginx/sbin/nginx

sleep 5

if [ "$(ps -ef | grep "nginx: master process"| grep -v
grep )" == "" ]

then

killall keepalived

fi

fi

5.启动keepalived

service keepalived start

注:备机的keepalived的安装和上面一样，只要把配置文件中的priority改为小于100的数即可

6、配置Keepalived监控脚本，每分钟执行一次，当检测到原先MASTER上的Keepalived进程挂掉之后，将重启Keepalived进程

vi  /root/shell/keepalived.sh

touch  /root/shell/keepalived.log

chmod 755 *

新增计划任务

crontab –e

* * * * * (sh /root/shell/keepalived.sh
>> /root/shell/keepalived.log)

六．测试步骤

1．访问VIP看是否能够正常访问后端的tomcat

2．停止其中一个tomcat看是否能将访问转到另一台上

3．停止两台nginx上任何一个nginx进程看监控进程脚本是否会自动启动nginx

VRRP_Script(Monitor_Nginx) timed out

VRRP_Script(Monitor_Nginx) succeeded

4．停止任何一台nginx上的keepalived进程看另一台是否接管vip

比如停止Master上的keepalived,例如如下killall keepalived,查看BACKUP机器是否已经接管，如果BACKUP接管后，BACKUP机器日志会是出下情况

tail 
-f  /var/log/messages

Keepalived_vrrp: VRRP_Instance(VI_1)
Transition to MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
Entering MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
setting protocol VIPs.

Keepalived_vrrp: VRRP_Instance(VI_1)
Sending gratuitous ARPs on eth0 for 10.46.86.60

MASTER机器上日志会显示

Keepalived_vrrp: Terminating VRRP child
process on signal

Keepalived_vrrp: VRRP_Instance(VI_1)
removing protocol VIPs.

现在把原先MASTER上的Keepalived重新启动，会看到原先MASTER直接成为BACKUP，不再做一次切换。

现在的BACKUP日志如下：

Keepalived_vrrp: VRRP_Instance(VI_1)
Received higher prio advert

Keepalived_vrrp: VRRP_Instance(VI_1)
Entering BACKUP STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
removing protocol VIPs.

Master日志如下：

Keepalived_vrrp: VRRP_Script(Monitor_Nginx)
succeeded

Keepalived_vrrp: VRRP_Instance(VI_1)
Transition to MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
Entering MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
setting protocol VIPs.

Keepalived_vrrp: VRRP_Instance(VI_1)
Sending gratuitous ARPs on eth0 for 10.46.86.60

一.系统优化

1、修改/etc/sysctl.conf，优化tcp连接数

net.ipv4.tcp_fin_timeout = 30

net.ipv4.tcp_keepalive_time = 1200

net.ipv4.tcp_syncookies = 1

net.ipv4.tcp_tw_reuse = 1

net.ipv4.tcp_tw_recycle = 1

net.ipv4.ip_local_port_range = 1024 65000

net.ipv4.tcp_max_syn_backlog = 8192

net.ipv4.tcp_max_tw_buckets = 5000

2、修改/etc/security/limits.conf，修改最大文件打开数为65535

添加如下两行：

*      soft  nofile       65535

*      hard nofile       65535

*      soft  nproc       16384

*      hard nproc       16384

二.安装JDK

chmod +x jdk-6u38-linux-x64.bin

# ./jdk-6u38-linux-x64.bin

#添加环境变量

# vi /etc/profile

export JAVA_HOME=/kjt/java/jdk1.6.0_38

export
CLASSPATH=.:$JAVA_HOME/lib/tools.jar:$JAVA_HOME/lib/dt.jar

export PATH=$JAVA_HOME/bin:$PATH

#使环境变量生效

# source /etc/profile

#查看java版本

# java -version

java version "1.6.0_38"

Java(TM) SE Runtime Environment (build
1.6.0_27-b07)

Java HotSpot(TM) 64-Bit Server VM (build
20.2-b06, mixed mode)

三.安装Tomcat

1、确保防火墙已关闭

#service iptables stop

#chkconfig iptables off

#vi /etc/selinux/config

selinux=disabled

2、安装tomcat

# tar zxvf apache-tomcat-6.0.36.tar.gz

# cd apache-tomcat-6.0.36

#修改/bin/catalina.sh，在cygwin=false前加路径

export JAVA_HOME=/kjt/java/jdk1.6.0_38/

export PATH=$JAVA_HOME/bin:$PATH

export
CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export
TOMCAT_HOME=/kjt/java/apache-tomcat-6.0.36

export
CATALINA_HOME=/kjt/java/apache-tomcat-6.0.36

export
CATALINA_BASE=/kjt/java/apache-tomcat-6.0.36

3、修改/conf/server.xml，做tomcat线程池优化

<Connector port="8081" protocol="HTTP/1.1"

connectionTimeout="15000" maxThreads="500"
minSpareThreads="50" maxSpareTheads="250"
maxProcessors="600"

acceptCount="1000"
enableLookups="false" compression="on"

redirectPort="8443"
URIEncoding="utf-8" useBodyEncodingForURI="true"/>

<!-- A
"Connector" using the shared thread pool-->

4、启动tomcat

cd /kjt/java/apache-tomcat-6.0.35/bin

./startup

5、查看tomcat进程

ps –ef|grep tomcat

四.Nginx配置

1.确定Nginx所需pcre库、openssl库等依赖包已经安装，若没有安装，安装nginx过程中将报错。

2.安装Nginx

tar zxvf nginx-1.0.15.tar.gz

cd nginx-1.0.15/

./configure  --prefix=/kjt/app/nginx --with-http_stub_status_module
--with-http_ssl_module

make

make install

3.Nginx配置文件如下：

4.测试配置文件

/kjt/app/nginx/sbin/nginx -t

显示以下信息为正确的

the configuration file
/usr/local/nginx/conf/nginx.conf syntax is ok

configuration file
/usr/local/nginx/conf/nginx.conf test is successful

5、启动nginx服务

/kjt/app/nginx/sbin/nginx

6、查看nginx进程

ps -ef|grep nginx

五.安装配置Keepalived

1、安装keepalived

tar -zxvf keepalived-1.2.7.tar.gz

cd keepalived-1.2.7

./configure --prefix=/kjt/app/keepalived

make

make install

2、将keepalived设置为开机启动

cp /kjt/app/keepalived/etc/rc.d/init.d/keepalived
/etc/rc.d/init.d/

cp/kjt/app/keepalived/etc/sysconfig/keepalived
/etc/sysconfig/

cp /kjt/app/keepalived/sbin/keepalived
/usr/sbin/

chkconfig --add keepalived

chkconfig keepalived on

3.修改配置文件为以下内容:

mkdir /etc/keepalived

cp /kjt/app/keepalived/etc/keepalived/keepalived.conf
/etc/keepalived/

vim  /etc/keepalived/keepalived.conf

! Configuration File for keepalived

global_defs {

router_id Nginx_Id_1

}

vrrp_script Monitor_Nginx {

script "/kjt/app/keepalived/etc/keepalived/samples/monitor_nginx.sh
"  #脚本存放位置

interval 2     #检查间隔为2秒

weight 2

}

vrrp_instance VI_1 {

state
BACKUP  #设置nopreempt，表示之前挂掉的master重新恢复后不再切换为master，直接成为BACKUP，nopreempt只能配置在BACKUP状态下

nopreempt

interface eth0

virtual_router_id 51

priority 100    #优先级较高的成为MASTER

advert_int 5   #健康检查间隔为5秒

authentication {

auth_type PASS

auth_pass kjt2013

}

virtual_ipaddress {

10.46.86.60

}

track_script {

Monitor_Nginx

}

}

注：monitor_nginx.sh为监控nginx进程的脚本,内容如下：当检测到nginx主进程挂掉之后，系统将自动重启nginx，若nginx主进程重启失败，延迟5秒后杀掉keepalived进程，进行切换。

#!/bin/bash

if [ "$(ps -ef | grep "nginx:
master process"| grep -v grep )" == "" ]

then

/kjt/app/nginx/sbin/nginx

sleep 5

if [ "$(ps -ef | grep "nginx: master process"| grep -v
grep )" == "" ]

then

killall keepalived

fi

fi

5.启动keepalived

service keepalived start

注:备机的keepalived的安装和上面一样，只要把配置文件中的priority改为小于100的数即可

6、配置Keepalived监控脚本，每分钟执行一次，当检测到原先MASTER上的Keepalived进程挂掉之后，将重启Keepalived进程

vi  /root/shell/keepalived.sh

touch  /root/shell/keepalived.log

chmod 755 *

新增计划任务

crontab –e

* * * * * (sh /root/shell/keepalived.sh
>> /root/shell/keepalived.log)

六．测试步骤

1．访问VIP看是否能够正常访问后端的tomcat

2．停止其中一个tomcat看是否能将访问转到另一台上

3．停止两台nginx上任何一个nginx进程看监控进程脚本是否会自动启动nginx

VRRP_Script(Monitor_Nginx) timed out

VRRP_Script(Monitor_Nginx) succeeded

4．停止任何一台nginx上的keepalived进程看另一台是否接管vip

比如停止Master上的keepalived,例如如下killall keepalived,查看BACKUP机器是否已经接管，如果BACKUP接管后，BACKUP机器日志会是出下情况

tail 
-f  /var/log/messages

Keepalived_vrrp: VRRP_Instance(VI_1)
Transition to MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
Entering MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
setting protocol VIPs.

Keepalived_vrrp: VRRP_Instance(VI_1)
Sending gratuitous ARPs on eth0 for 10.46.86.60

MASTER机器上日志会显示

Keepalived_vrrp: Terminating VRRP child
process on signal

Keepalived_vrrp: VRRP_Instance(VI_1)
removing protocol VIPs.

现在把原先MASTER上的Keepalived重新启动，会看到原先MASTER直接成为BACKUP，不再做一次切换。

现在的BACKUP日志如下：

Keepalived_vrrp: VRRP_Instance(VI_1)
Received higher prio advert

Keepalived_vrrp: VRRP_Instance(VI_1)
Entering BACKUP STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
removing protocol VIPs.

Master日志如下：

Keepalived_vrrp: VRRP_Script(Monitor_Nginx)
succeeded

Keepalived_vrrp: VRRP_Instance(VI_1)
Transition to MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
Entering MASTER STATE

Keepalived_vrrp: VRRP_Instance(VI_1)
setting protocol VIPs.

Keepalived_vrrp: VRRP_Instance(VI_1)
Sending gratuitous ARPs on eth0 for 10.46.86.60