webpy/flask/bottle性能测试

这三个都是Python WSGI的web开发框架，到底用哪个呢？单纯从性能的角度而言，可能哪个快就用哪个，但是这也不是绝对的。比如我就比较喜欢webpy的router配置放在一个文件中，而flask和bottle的配置分散到各个文件中，从开发角度，写在哪里无所谓，但是从阅读的角度，webpy就比较方便了。因为在工作中本着拿来主义的角度使用了webpy，但是看起来flask也很火，而bottle据说更是一个精简的文件，所以本文简单测试了三者的性能，供参考。

测试方法：用三者分别写一个URL API，这个API直接返回“hello world!”字符串，用uwsgi启动python应用。客户端采用apache benchmark工具：ab，对server执行500000次请求，并发为1000，然后比较ab的测试结果，同时用统计机器的CPU消耗。

测试脚本：http://pan.baidu.com/s/1URGJ0，提取密码：6thp

测试结果：

CPU曲线图（测试顺序：webpy，sleep 120秒，flask，sleep 120秒，bottle）

ab的测试结果：

• webpy

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
……
Completed 50000 requests
Completed …… requests
Finished 500000 requests
……
Server Port:            8010
Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   1533.489 seconds
Complete requests:      500000
Failed requests:        24246
   (Connect: 0, Receive: 0, Length: 24246, Exceptions: 0)
Write errors:           0
Total transferred:      14748374 bytes
HTML transferred:       5709048 bytes
Requests per second:    326.05 [#/sec] (mean)
Time per request:       3066.978 [ms] (mean)
Time per request:       3.067 [ms] (mean, across all concurrent requests)
Transfer rate:          9.39 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    1  10.1      1    3000
Processing:   111 2962 7581.0    645   69933
Waiting:        0 1629 3949.2    642   68942
Total:        218 2963 7581.6    646   69933

Percentage of the requests served within a certain time (ms)
  50%    646
......
 100%  69933 (longest request)

• flask

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
......
Completed 50000 requests
......
Completed 500000 requests
Finished 500000 requests
......
Server Port:            8020

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   862.997 seconds
Complete requests:      500000
Failed requests:        17067
   (Connect: 0, Receive: 0, Length: 17067, Exceptions: 0)
Write errors:           0
Total transferred:      43946903 bytes
HTML transferred:       5795196 bytes
Requests per second:    579.38 [#/sec] (mean)
Time per request:       1725.993 [ms] (mean)
Time per request:       1.726 [ms] (mean, across all concurrent requests)
Transfer rate:          49.73 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    2  42.4      1    3002
Processing:    40 1635 6311.8    196   69819
Waiting:        0  725 3301.8    195   69030
Total:         71 1636 6312.6    197   69819

Percentage of the requests served within a certain time (ms)
  50%    197
......
 100%  69819 (longest request)

• bottle

This is ApacheBench, Version 2.3 <$Revision: 655654 $>
......
Completed 50000 requests
......
Completed 500000 requests
Finished 500000 requests
......
Server Port:            8030

Document Path:          /
Document Length:        12 bytes

Concurrency Level:      1000
Time taken for tests:   419.576 seconds
Complete requests:      500000
Failed requests:        4111
   (Connect: 0, Receive: 0, Length: 4111, Exceptions: 0)
Write errors:           0
Total transferred:      45125899 bytes
HTML transferred:       5950668 bytes
Requests per second:    1191.68 [#/sec] (mean)
Time per request:       839.153 [ms] (mean)
Time per request:       0.839 [ms] (mean, across all concurrent requests)
Transfer rate:          105.03 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0   21 256.5      1    9015
Processing:     9  709 3949.7    117   69845
Waiting:        0  460 2441.9    116   67652
Total:         29  730 3957.6    119   69846

Percentage of the requests served within a certain time (ms)
  50%    119
......
 100%  69846 (longest request)

结果分析：

（1）从平均的response time：bottle(0.839) < flask(1.726) < webpy(3.067)

（2）从TPS：bottle(1191.68) > flask(579.38) > webpy(326.05)

（3）从Failed请求数：bottle(1191.68) > flask(579.38) > webpy(326.05)

[注]：为什么会有fail？从uwsgi的log看来，报了一些（Write IO Error），这个错误的原因是当uwsgi处理完请求返回结果时，发现客户端已经断开了连接，结果无处可送，则Writer IO Error。一般情况下，这是由于客户端响应太慢，导致了客户端timeout所致。看起来响应还是足够快的，并且并发只有1000，这里为何会timeout，不是十分确定根本原因。

（4）从CPU：

• webpy最耗CPU，其中黄色部分是（CPU system time），看起来系统调用比其余两个高很多。
• bottle比flask略高一些，总体是上，相差不算过大。

结论：性能排序结果为webpy < flask < bottle。

那是不是就意味着应该选性能最好的bottle？这也不应太绝对，因为在开发项目时，不同的人可能对不同的项目的熟悉程度不一样，另外flask可能有广泛的插件支持，所以选择最适合自己的，而不是一昧追求性能最快。

说明：本文试图比较三者之间的性能，而不是测试三个模块的性能极限（例如我都没有说明测试机的配置，因为这里测试的是同一配置下的相对值）。如果测试三个模块的性能极限，可能server配置、以及uwsgi的配置均需要根据测试进行优化，在这里不是重点，不再赘述。