nginx+keeplived+tomcat

1，宣告操作系统版本，nginx，java，tomcat，keeplived版本

操作系统	用途	VIP	IP地址	软件版本
CentOS 7.3 mini	NTP服务器	无	192.168.197.31	官方yum源版本
CentOS 7.3 mini	Nginx主	192.168.197.30	192.168.197.32	nginx-1.14.2.tar.gz
				keepalived-1.3.5.tar.gz
CentOS 7.3 mini	Nginx备		192.168.197.33	nginx-1.14.2.tar.gz
				keepalived-1.3.5.tar.gz
CentOS 7.3 mini	Tomcat1	无	192.168.197.34	jdk-8u102-linux-x64.rpm
				apache-tomcat-8.5.32.tar.gz
CentOS 7.3 mini	Tomcat2		192.168.197.35	jdk-8u102-linux-x64.rpm
				apache-tomcat-8.5.32.tar.gz

2，配置服务器IP，确定主nginx服务器的vip

[root@localhost ~]# more /etc/sysconfig/network-scripts/ifcfg-ens33

TYPE=Ethernet

BOOTPROTO=none

DEFROUTE=yes

PEERDNS=yes

PEERROUTES=yes

IPV4_FAILURE_FATAL=no

IPV6INIT=yes

IPV6_AUTOCONF=yes

IPV6_DEFROUTE=yes

IPV6_PEERDNS=yes

IPV6_PEERROUTES=yes

IPV6_FAILURE_FATAL=no

IPV6_ADDR_GEN_MODE=stable-privacy

NAME=ens33

UUID=d7cca43a-afd4-4bb3-aabb-90a2318fe28e

DEVICE=ens33

ONBOOT=yes

IPADDR=192.168.197.31

PREFIX=24

GATEWAY=192.168.197.2

DNS1=114.114.114.114

根据实际情况配置ip地址，需要需要vip的再配置文件中加入以下信息

IPADDR1=192.168.197.30

重启网络服务

[root@localhost ~]# service network restart

测试外网连接性

[root@localhost ~]# ping www.baidu.com

PING www.a.shifen.com (111.13.100.92) 56(84) bytes of data.

64 bytes from 111.13.100.92 (111.13.100.92): icmp_seq=2 ttl=128 time=29.4 ms

64 bytes from 111.13.100.92 (111.13.100.92): icmp_seq=3 ttl=128 time=25.7 ms

64 bytes from 111.13.100.92 (111.13.100.92): icmp_seq=4 ttl=128 time=26.0 ms

永久所有服务器的关闭防火墙和selinux

[root@localhost ~]# systemctl disable firewalld

[root@localhost ~]# systemctl stop firewalld

[root@localhost ~]# setenforce 0

[root@localhost ~]# vi /etc/selinux/config

SELINUX=disabled

3，配置chrony时间服务器和客户端

Centos7已经预装chrony软件，如果未安装可使用yum install –y chrony 来安装

检查是否安装可使用以下命令

rpm –ap |grep chron

或

yum list installed |grep chron

chrony服务端配置

[root@localhost ~]# vi /etc/chrony.conf

allow 192.168.0.0/16

其他配置保持不变，添加以上行表示允许192.168.0.0/16网络的使用此chronyd服务

重启chronyd服务，并添加到开机启动

[root@localhost ~]# systemctl enable chronyd.service

[root@localhost ~]# systemctl restart chronyd.service

chrony所有客户端配置

[root@localhost ~]# vi /etc/chrony.conf

# Use public servers from the pool.ntp.org project.

# Please consider joining the pool (http://www.pool.ntp.org/join.html).

#server 0.centos.pool.ntp.org iburst

#server 1.centos.pool.ntp.org iburst

#server 2.centos.pool.ntp.org iburst

#server 3.centos.pool.ntp.org iburst

server 192.168.197.31 iburst

注释掉此部分所有的server项，添加server 192.168.197.31 iburst项目，192.168.197.31为chrony服务器的IP地址

检查chrony同步的状态

[root@localhost ~]# chronyc sources -v

210 Number of sources = 1

.-- Source mode '^' = server, '=' = peer, '#' = local clock.

/ .- Source state '*' = current synced, '+' = combined , '-' = not combined,

| / '?' = unreachable, 'x' = time may be in error, '~' = time too variable.

|| .- xxxx [ yyyy ] +/- zzzz

|| Reachability register (octal) -. | xxxx = adjusted offset,

|| Log2(Polling interval) --. | | yyyy = measured offset,

|| \ | | zzzz = estimated error.

|| | | \

MS Name/IP address Stratum Poll Reach LastRx Last sample

===============================================================================

^* 192.168.197.31 3 6 17 43 -11us[ -63us] +/- 113ms

4，安装nginx，测试nginx

使用xftp客户端将nginx和keeplived包上传到两台服务器的/software目录下，解压缩软件包

[root@localhost software]# tar -zxvf nginx-1.14.2.tar.gz

在线安装依赖包

yum -y install gcc gcc-++ autoconf automake pcre-devel openssl openssl-devel

进入解压后的目录，编译安装

[root@localhost software]# cd nginx-1.14.2

[root@localhost nginx-1.14.2]# ./configure --prefix=/usr/local/nginx

[root@localhost nginx-1.14.2]# make & make install

启动nginx服务

[root@localhost nginx-1.14.2]# /usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

查看nginx的监听端口

[root@localhost nginx-1.14.2]# netstat -nutpl |grep nginx

tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 5008/nginx: master

如果没有netstat命令，需要安装net-tools，在线安装net-tools的命令为

[root@localhost nginx-1.14.2]# yum install -y net-tools

得知ngxin监听的端口是80，使用浏览器访问两台的80端口上的内容

5，安装两台tomcat服务器的java，tomcat软件，并测试tomcat是否正常运行

首先上传jdk包和tomcat包至/software目录

在安装前可先检查系统是否已安装jdk，使用以下命令检查

[root@localhost ~]# rpm -qa |grep jdk

或者

[root@localhost ~]# yum list installed |grep jdk

如果有则通过

[root@localhost ~]# rpm –e –nodeps 包名

或

[root@localhost ~]# yum –y remove 包名

来删除

安装jdk的rpm包

[root@localhost software]# rpm -ivh jdk-8u102-linux-x64.rpm

移动安装后的目录

[root@localhost software]# mv /usr/java/jdk1.8.0_102 /usr/local/jdk1.8

编辑环境变量加入以下代码

[root@localhost software]# vi /etc/profile

JAVA_HOME=/usr/local/jdk1.8

PATH=$JAVA_HOME/bin:$PATH

CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

export JAVA_HOME

export PATH

export CLASSPATH

使配置文件立即生效

[root@localhost software]# source /etc/profile

执行java –version 看是否输出版本信息，如输出版本信息则证明安装成功

[root@localhost software]# java -version

java version "1.8.0_102"

Java(TM) SE Runtime Environment (build 1.8.0_102-b14)

Java HotSpot(TM) 64-Bit Server VM (build 25.102-b14, mixed mode)

安装tomcat，解压缩tomcat包

[root@localhost software]# tar -zxvf apache-tomcat-8.5.32.tar.gz

移动并更名tomcat

[root@localhost software]# mv apache-tomcat-8.5.32 /usr/local/tomcat8

启动tomcat

[root@localhost software]# cd /usr/local/tomcat8/bin/

[root@localhost bin]# ./startup.sh

Using CATALINA_BASE: /usr/local/tomcat8

Using CATALINA_HOME: /usr/local/tomcat8

Using CATALINA_TMPDIR: /usr/local/tomcat8/temp

Using JRE_HOME: /usr/local/jdk1.8

Using CLASSPATH: /usr/local/tomcat8/bin/bootstrap.jar:/usr/local/tomcat8/bin/tomcat-juli.jar

Tomcat started.

默认情况下tomcat监听端口为8080，现在可使用浏览器访问两台tomcat上的8080端口，如果无法访问请使用netstat检查8080端口的监听情况，检查selinux的开关和防火墙的开关。

6，在两台nginx上安装keeplived以及依赖包

解压keeplived包

[root@localhost software]# tar -zxvf keepalived-1.3.5.tar.gz

在线安装依赖包

[root@localhost software]# yum install -y gcc openssl-devel popt-devel libnl libnl-devel libnfnetlink-devel

编译安装

[root@localhost software]# cd keepalived-1.3.5

[root@localhost keepalived-1.3.5]# ./configure --prefix=/usr/local/keepalived

[root@localhost keepalived-1.3.5]# make & make install

创建连接

[root@localhost keepalived-1.3.5]# cp /software/keepalived-1.3.5/keepalived/etc/init.d/keepalived /etc/init.d/keepalived

添加权限

[root@localhost keepalived-1.3.5]# chmod +x /etc/init.d/keepalived

编辑配置文件

[root@localhost keepalived-1.3.5]# vi /etc/init.d/keepalived

寻找大约15行左右的. /etc/sysconfig/keepalived, 修改为：

. /usr/local/keepalived/etc/sysconfig/keepalived, 即指向正确的配置文件位置。

配置环境变量

[root@localhost keepalived-1.3.5]# vi /etc/profile

KEEPALIVED_HOME=/usr/local/keepalived

PATH=$KEEPALIVED_HOME/sbin:$PATH

export PATH

使环境变量生效

[root@localhost keepalived-1.3.5]# source /etc/profile

建立可执行文件链接：

[root@localhost keepalived-1.3.5]# ln -s /usr/local/keepalived/sbin/keepalived /usr/sbin/

修改/usr/local/keepalived/etc/sysconfig/keepalived文件，设置正确的服务启动参数

KEEPALIVED_OPTIONS="-D -f /usr/local/keepalived/etc/keepalived/keepalived.conf"

[root@localhost sbin]# vi /usr/local/keepalived/etc/sysconfig/keepalived

KEEPALIVED_OPTIONS="-D -f /usr/local/keepalived/etc/keepalived/keepalived.conf"

修改keepalived.service

[root@localhost sbin]# vi /lib/systemd/system/keepalived.service

PIDFile=/var/run/keepalived.pid

重新载入 systemd，扫描新的或有变动的单元

[root@localhost sbin]# systemctl daemon-reload

重启keepalived服务

[root@localhost sbin]# service keepalived restart

设置开机启动

[root@localhost sbin]# chkconfig keepalived on

7，配置nginx和keepalived实现nginx间的故障转移和tomcat间的负载均衡

两台nginx的配置

#user nobody;

worker_processes 1;

#error_log logs/error.log;

#error_log logs/error.log notice;

error_log logs/error.log info;

#pid logs/nginx.pid;

events {

worker_connections 1024;

}

http {

include mime.types;

default_type application/octet-stream;

#log_format main '$remote_addr - $remote_user [$time_local] "$request" '

# '$status $body_bytes_sent "$http_referer" '

# '"$http_user_agent" "$http_x_forwarded_for"';

#access_log logs/access.log main;

sendfile on;

#tcp_nopush on;

#keepalive_timeout 0;

keepalive_timeout 65;

#gzip on;

upstream web123{

server 192.168.197.34:8080 weight=1;

server 192.168.197.35:8080 weight=1;

}

server {

listen 80;

server_name localhost;

#charset koi8-r;

#access_log logs/host.access.log main;

location / {

proxy_pass http://web123;

root html;

index index.html index.htm;

}

#error_page 404 /404.html;

# redirect server error pages to the static page /50x.html

#

error_page 500 502 503 504 /50x.html;

location = /50x.html {

root html;

}

# proxy the PHP scripts to Apache listening on 127.0.0.1:80

#

#location ~ \.php$ {

# proxy_pass http://127.0.0.1;

#}

# pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000

#

#location ~ \.php$ {

# root html;

# fastcgi_pass 127.0.0.1:9000;

# fastcgi_index index.php;

# fastcgi_param SCRIPT_FILENAME /scripts$fastcgi_script_name;

# include fastcgi_params;

#}

# deny access to .htaccess files, if Apache's document root

# concurs with nginx's one

#

#location ~ /\.ht {

# deny all;

#}

}

# another virtual host using mix of IP-, name-, and port-based configuration

#

#server {

# listen 8000;

# listen somename:8080;

# server_name somename alias another.alias;

# location / {

# root html;

# index index.html index.htm;

# }

#}

# HTTPS server

#

#server {

# listen 443 ssl;

# server_name localhost;

# ssl_certificate cert.pem;

# ssl_certificate_key cert.key;

# ssl_session_cache shared:SSL:1m;

# ssl_session_timeout 5m;

# ssl_ciphers HIGH:!aNULL:!MD5;

# ssl_prefer_server_ciphers on;

# location / {

# root html;

# index index.html index.htm;

# }

#}

}

Keepalived的配置

! Configuration File for keepalived

global_defs {

notification_email {

acassen@firewall.loc

failover@firewall.loc

sysadmin@firewall.loc

}

notification_email_from Alexandre.Cassen@firewall.loc

smtp_server 192.168.200.1

smtp_connect_timeout 30

router_id nginx_b

vrrp_skip_check_adv_addr

#vrrp_strict

vrrp_garp_interval 0

vrrp_gna_interval 0

}

vrrp_script check_nginx {

script "/usr/local/keepalived/etc/keepalived/check_nginx.sh"

interval 1

weight 2

}

vrrp_instance VI_1 {

state BACKUP

interface ens33

mcast_src_ip 192.168.197.33

virtual_router_id 51

priority 99

advert_int 1

authentication {

auth_type PASS

auth_pass 1111

}

virtual_ipaddress {

192.168.197.30

}

}

Keepalived中检测脚本的配置

#!/bin/bash

#代码一定注意空格，逻辑就是：如果nginx进程不存在则启动nginx,如果nginx无法启动则kill掉keepalived所有进程

A=`ps -C nginx --no-header |wc -l`

if [ $A -eq 0 ];then

/etc/init.d/nginx start

sleep 3

if [ `ps -C nginx --no-header |wc -l`-eq 0 ];then

killall keepalived

fi

fi

8，相关服务的启动与开机自启

Nginx启动

/usr/local/nginx/sbin/nginx -c /usr/local/nginx/conf/nginx.conf

设置Nginx开机启动

在/lib/systemd/system/目录创建nginx.service启动文件并编辑配置以下内容

[root@localhost nginx-1.14.2]# cd /lib/systemd/system

[root@localhost system]# touch nginx.service

[root@localhost system]# vi nginx.service

[Unit]

Description=nginx

After=network.target

[Service]

Type=forking

ExecStart=/usr/local/nginx/sbin/nginx

ExecReload=/usr/local/nginx/sbin/nginx -s reload

ExecStop=/usr/local/nginx/sbin/nginx -s quit

PrivateTmp=true

[Install]

WantedBy=multi-user.target

设置开机启动

[root@localhost system]# systemctl enable nginx.service

然后通过systemctl status nginx.service命令查询启动状态，通过restart，start，stop命令来重启，启动，停止服务

Tomcat的启动

切换到tomcat主目录下的bin目录

启动tomcat服务

方式一：直接启动 ./startup.sh

方式二：作为服务启动 nohup ./startup.sh &

方式三：控制台动态输出方式启动 ./catalina.sh run 动态地显示tomcat后台的控制台输出信息,Ctrl+C后退出并关闭服务

解释：

通过方式一、方式三启动的tomcat有个弊端，当客户端连接断开的时候，tomcat服务也会立即停止，通过方式二可以作为linux服务一直运行

通过方式一、方式二方式启动的tomcat，其日志会写到相应的日志文件中，而不能动态地查看tomcat控制台的输出信息与错误情况，通过方式三可以以控制台模式启动tomcat服务，

直接看到程序运行时后台的控制台输出信息，不必每次都要很麻烦的打开catalina.out日志文件进行查看，这样便于跟踪查阅后台输出信息。tomcat控制台信息包括log4j和System.out.println()等输出的信息。

关闭tomcat服务

./shutdown.sh

Tomcat的开机自启配置

获取java home目录，并编辑catalina.sh配置文件在OS specific support前面添加下内容：

[root@localhost bin]# echo $JAVA_HOME

/usr/local/jdk1.8

[root@localhost bin]# vi /usr/local/tomcat8/bin/catalina.sh

JAVA_HOME=/usr/local/jdk1.8

JRE_HOME=$JAVA_HOME/jre

创建tomcat8.service启动文件，并编辑加入以下内容：

[root@localhost bin]# cd /lib/systemd/system

[root@localhost system]# touch tomcat8.service

[root@localhost system]# vi tomcat8.service

[Unit]

Description=Tomcat

After=syslog.target network.target remote-fs.target nss-lookup.target

[Service]

Type=oneshot

ExecStart=/usr/local/tomcat8/bin/startup.sh

ExecStop=/usr/local/tomcat8/bin/shutdown.sh

ExecReload=/bin/kill -s HUP $MAINPID

RemainAfterExit=yes

[Install]

WantedBy=multi-user.target

配置开机自启

[root@localhost system]# systemctl enable tomcat8.service