nginx优化实践与验证

nginx优化实践

实践场景1: 单台nginx 2核4G

实践场景2: 三台nginx 2核4G

压测工具:WRK

初始安装的nginx压测:

yum install -y nginx

安装WRK压测工具

git clone https://gitee.com/mirrors/wrk.git
cd wrk
make
cp -a  wrk  /usr/local/bin/

wrk参数:

使用方法: wrk <选项> <被测HTTP服务的URL>
  Options:
    -c, --connections <N>  跟服务器建立并保持的TCP连接数量
    -d, --duration    <T>  压测时间
    -t, --threads     <N>  使用多少个线程进行压测   

    -s, --script      <S>  指定Lua脚本路径
    -H, --header      <H>  为每一个HTTP请求添加HTTP头
        --latency          在压测结束后，打印延迟统计信息
        --timeout     <T>  超时时间
    -v, --version          打印正在使用的wrk的详细版本信息

  <N>代表数字参数，支持国际单位 (1k, 1M, 1G)
  <T>代表时间参数，支持时间单位 (2s, 2m, 2h)

云服务器配置

3台Linux服务器-绑定了一个公网IP39.98.77.148用于配置服务器
1台负载均衡slb

3台服务器安装nginx

yum isntall -y nginx

标识不同nginx
echo "nginx-171" >/usr/share/nginx/html/index.html
echo "nginx-171" >/usr/share/nginx/html/index.html
echo "nginx-171" >/usr/share/nginx/html/index.html

#nginx配置[每台服务器都使用最基础的配置]:
egrep -v "^$|#" /etc/nginx/nginx.conf.default >/etc/nginx/nginx.conf
nginx -s reload

#配置概览:
worker_processes  1;
events {
    worker_connections  1024;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

检查轮询算法:
[root@nginx-172 ~]# curl 192.168.0.171
nginx-171
[root@nginx-172 ~]# curl 192.168.0.172
nginx-172
[root@nginx-172 ~]# curl 192.168.0.173
nginx-173

wrk压测 初始[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208

初始配置压测:
[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   246.22ms  289.79ms   2.00s    85.67%
    Req/Sec     1.12k   427.18     3.51k    72.62%
  265348 requests in 2.00m, 1.34GB read
  Socket errors: connect 0, read 20, write 7873, timeout 3620
Requests/sec:   2210.37
Transfer/sec:     11.41MB

初始数据:
延迟  246.22ms

优化1: work进程连接数优化:

worker_processes  1;
events {
    worker_connections  100000;    <----连接数配置,3台都配置成这个,重启后测试
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

结果:

[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   238.58ms  279.94ms   2.00s    84.64%
    Req/Sec     1.06k   420.61     2.89k    70.18%
  250585 requests in 2.00m, 1.26GB read
  Socket errors: connect 0, read 12, write 10299, timeout 2497
Requests/sec:   2086.60
Transfer/sec:     10.77MB
--------------------------------------
结果:
延迟降低   289.79ms --> 238.58ms
丢包率减少 3620 --> 2497

优化2: work进程数量优化:

worker_processes  auto;   <--- 这里直接改为了auto,而不是1
events {
    worker_connections  100000;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

结果:

[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   263.22ms  298.89ms   2.00s    86.82%
    Req/Sec     1.10k   382.66     2.81k    71.77%
  259948 requests in 2.00m, 1.31GB read
  Socket errors: connect 0, read 10, write 6921, timeout 3454
Requests/sec:   2165.50
Transfer/sec:     11.18MB

结果:
每秒处理请求数   2086.60 --> 2165.50

优化3: cpu亲和力和优先级:

worker_processes  2;
worker_cpu_affinity 0101 1010;  <--- 亲和力
worker_priority -20;     <---优先级
events {
    worker_connections  100000;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

结果:

[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   222.80ms  267.85ms   2.00s    84.34%
    Req/Sec   805.34    333.47     2.61k    73.99%
  190706 requests in 2.00m, 0.96GB read
  Socket errors: connect 0, read 15, write 16445, timeout 1272    # timeout明显减少
Requests/sec:   1588.88
Transfer/sec:      8.20MB

优化4: gzip优化:

worker_processes  2;
worker_cpu_affinity 0101 1010;  <--- 亲和力
worker_priority -20;     <---优先级
events {
    worker_connections  100000;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    gzip on;
    gzip_min_length 500;
    gzip_buffers 4 256;
    gzip_http_version 1.1;
    gzip_comp_level 2;
    gzip_types text/plain application/javascript application/x-javascript text/javascript text/css application/xml application/xml+rss;
    gzip_vary on;
    gzip_proxied expired no-cache no-store private auth;
    gzip_disable "MSIE [1-6]\.";
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
}

[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   232.83ms  277.62ms   2.00s    84.19%
    Req/Sec     0.95k   346.60     2.94k    75.79%
  225026 requests in 2.00m, 1.14GB read
  Socket errors: connect 0, read 13, write 12248, timeout 2490
Requests/sec:   1873.72
Transfer/sec:      9.71MB

优化5 CPU开销优化

events {
    worker_connections  100000;
    multi_accept on;
    accept_mutex on;
    accept_mutex_delay 1ms;
}

[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   213.94ms  260.76ms   2.00s    84.76%
    Req/Sec     0.87k   371.67     3.34k    77.82%
  206969 requests in 2.00m, 1.05GB read
  Socket errors: connect 0, read 13, write 15329, timeout 1713
Requests/sec:   1724.03
Transfer/sec:      8.93MB

优化6 日志优化

access_log /var/log/nginx/access.log aaa buffer=1m;

[root@master wrk]# wrk -t2 -c800 -d120s http://39.99.217.208
Running 2m test @ http://39.99.217.208
  2 threads and 800 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency   246.08ms  278.00ms   2.00s    84.53%
    Req/Sec   804.20    420.24     3.23k    75.12%
  189759 requests in 2.00m, 0.96GB read
  Socket errors: connect 0, read 30, write 14936, timeout 1451
Requests/sec:   1580.08
Transfer/sec:      8.19MB