Nginx+Tomcat8+Memcached实现负载均衡及session共享

1> 基础环境

简易拓扑图：

2> 部署Tomcat

[root@node01 ~]# ll -h ~ |egrep 'jdk|tomcat'
-rw-r--r-- 1 root root 9.3M May 4 17:40 apache-tomcat-8.5.41.tar.gz
-rw-r--r-- 1 root root 183M Jul 15 2018 jdk-8u171-linux-x64.tar.gz
[root@node01 ~]# tar zxf jdk-8u171-linux-x64.tar.gz
[root@node01 ~]# mv jdk1.8.0_171 /usr/local/
[root@node01 ~]# vim /etc/profile.d/java.sh

JAVA_HOME=/usr/local/jdk1.8.0_171
JAVA_BIN=/usr/local/jdk1.8.0_171/bin
JRE_HOME=/usr/local/jdk1.8.0_171/jre
PATH=$PATH:/usr/local/jdk1.8.0_171/bin:/usr/local/jdk1.8.0_171/jre/bin
CLASSPATH=/usr/local/jdk1.8.0_171/jre/lib:/usr/local/jdk1.8.0_171/lib:/usr/local/jdk1.8.0_171/jre/lib/charsets.jar

[root@node01 ~]# . /etc/profile.d/java.sh
[root@node01 ~]# java -version
[root@node01 ~]# tar zxf apache-tomcat-8.5.41.tar.gz
[root@node01 ~]# mv apache-tomcat-8.5.41 /opt/tomcat8
[root@node01 ~]# cd /opt/tomcat8/
[root@node01 ~]# > webapps/ROOT/index.jsp
[root@node01 ~]# vim webapps/ROOT/index.jsp

<html>
  <head>
    <title>web session test</title>
  </head>
  <body>
    <h1>10.100.1.121:8080</h1>
    <h2>session: <%=session.getId()%> </h2>
  </body>
</html>

[root@node01 ~]# ./bin/startup.sh

node02和node03和node01的部署方法一致，注意index.jsp信息要区分开。

3> Nginx代理Tomcat

[root@node01 ~]# cd ~
[root@node01 ~]# yum install -y pcre pcre-devel openssl openssl-devel
[root@node01 ~]# yum install -y nginx
[root@node01 ~]# vim /etc/nginx/nginx.conf
[root@node01 ~]# systemctl start nginx
[root@node01 ~]# systemctl enable nginx

[root@node01 ~]# /usr/sbin/nginx -s reload

访问http://10.100.1.121的请求会按权重轮流分发至各tomcat，但sessionID会随着页面刷新而变动。

4> 安装memcached

[root@node01 ~]# yum -y install libevent libevent-devel memcached
[root@node01 ~]# systemctl start memcached
[root@node01 ~]# systemctl enable memcached
[root@node01 ~]# ps -ef|grep memcached
[root@node01 ~]# netstat -lnpt

5> 实现session共享

MSM(memcached session manager)，MSM是一款实现Tomcat会话保持的管理组件，支持粘性和无粘性的配置，并且支持Memcached会话故障转移。
[root@node01 ~]# yum install -y git
[root@node01 ~]# git clone https://github.com/magro/memcached-session-manager.git
以下为我整理的jar包，下载拷贝到各tomcat下lib目录，可为你省去很多麻烦。
参考：https://github.com/magro/memcached-session-manager/wiki/SetupAndConfiguration#decide-which-serialization-strategy-to-use

wget http://repo1.maven.org/maven2/de/javakaffee/msm/memcached-session-manager/1.9.7/memcached-session-manager-1.9.7.jar
wget http://repo1.maven.org/maven2/de/javakaffee/msm/memcached-session-manager-tc8/1.9.7/memcached-session-manager-tc8-1.9.7.jar
wget http://repo1.maven.org/maven2/net/spy/spymemcached/2.12.2/spymemcached-2.12.2.jar
wget http://repo1.maven.org/maven2/com/esotericsoftware/kryo/4.0.0/kryo-4.0.0.jar
wget http://repo1.maven.org/maven2/org/ow2/asm/asm/5.2/asm-5.2.jar
wget http://repo1.maven.org/maven2/de/javakaffee/kryo-serializers/0.38/kryo-serializers-0.38.jar
wget http://repo1.maven.org/maven2/com/esotericsoftware/minlog/1.3.0/minlog-1.3.0.jar
wget http://repo1.maven.org/maven2/com/esotericsoftware/reflectasm/1.11.3/reflectasm-1.11.3.jar
wget http://repo1.maven.org/maven2/org/objenesis/objenesis/2.4/objenesis-2.4.jar
wget http://repo1.maven.org/maven2/de/javakaffee/msm/msm-kryo-serializer/1.9.7/msm-kryo-serializer-1.9.7.jar

配置context.xml
[root@node01 ~]# cd /opt/tomcat8/conf/
[root@node01 ~]# cp context.xml context.xml.bk
[root@node01 ~]# vim context.xml

    <Manager className="de.javakaffee.web.msm.MemcachedBackupSessionManager"
         memcachedNodes="n1:10.100.1.121:11211,n2:10.100.1.122:11211,n3:10.100.1.123:11211"
         lockingMode="auto"
         sticky="false"
         sessionBackupAsync="false"
         sessionBackupTimeout= "1000"
         copyCollectionsForSerialization="true"
         requestUriIgnorePattern=".*\.(ico|png|gif|jpg|css|js)$"
         transcoderFactoryClass="de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"
    />

[root@node01 ~]# scp context.xml 10.100.1.122:/opt/tomcat8/conf/context.xml
[root@node01 ~]# scp context.xml 10.100.1.123:/opt/tomcat8/conf/context.xml

重启tomcat服务后检测效果，n3表示当前session保存在node03节点的mencached上，

停掉node03节点的memcached服务，刷新页面，
[root@node03 tomcat8]# systemctl stop memcached
[root@node03 tomcat8]# ps -ef |grep memcached
root 7412 6754 0 19:46 pts/0 00:00:00 grep --color=auto memcached
刷新页面，session会话转移到n1这个memcached上了，但sessionID依然没有改变，说明session已共享。关闭任一节点的tomcat服务，发现从nginx负载过来的请求未分发至关停的tomcat上，sessionID也不会变。

关闭任一节点的tomcat服务，发现从nginx负载过来的请求未分发至关停的tomcat上，sessionID也不会变。

查看catalina.out日志,
[root@node01 logs]# tailf catalina.out

至此，Nginx+Tomcat+Memcached实现负载均衡及session共享就实现了。