Python开发【Tornado】：异步Web服务（一）
异步Web服务
前言：
到目前为止，我们已经看到了许多使Tornado成为一个Web应用强有力框架的功能。它的简单性、易用性和便捷性使其有足够的理由成为许多Web项目的不错的选择。然而，Tornado受到最多关注的功能是其异步取得和提供内容的能力，它有着很好的理由：它使得处理非阻塞请求更容易，最终导致更高效的处理以及更好的可扩展性。在本章中，我们将看到Tornado异步请求的基础，以及一些推送技术，这种技术可以使你使用更少的资源来提供更多的请求以编写更简单的Web应用。
大部分Web应用（包括我们之前的例子）都是阻塞性质的，也就是说当一个请求被处理时，这个进程就会被挂起直至请求完成。在大多数情况下，Tornado处理的Web请求完成得足够快使得这个问题并不需要被关注。然而，对于那些需要一些时间来完成的操作（像大数据库的请求或外部API），这意味着应用程序被有效的锁定直至处理结束，很明显这在可扩展性上出现了问题。默认情况下tornado是单线程运行，处理完一个请求再处理下一个
不过，Tornado给了我们更好的方法来处理这种情况。应用程序在等待第一个处理完成的过程中，让I/O循环打开以便服务于其他客户端，直到处理完成时启动一个请求并给予反馈，而不再是等待请求完成的过程中挂起进程。
我们将展示这个应用的三个不同版本：首先，是一个使用同步HTTP请求的版本，然后是一个使用带有回调函数的Tornado异步HTTP客户端版本。最后，我们将展示如何使用Tornado 2.1版本新增的gen模块来使异步HTTP请求更加清晰和易实现。
笔记：
- 默认情况下tornado是单线程阻塞模式，如果阻塞所有请求都需要等待
- tornado.web.asynchronous可以异步使用，得益于AsyncHTTPClient模块的配合使用，两者缺一不可
- tornado.gen.coroutine严重依赖第三方库的使用，如果没有第三方库的支持则依然是阻塞模式
- Tornado 提供了多种的异步编写形式：回调、Future、协程等，其中以协程模式最是简单和用的最多
- Tornado 实现异步的多种方式：coroutine配合第三方库、启用多线程、使用celery等
1、同步开始（对外进行api请求）
记住我们在顶部导入了Tornado的httpclient模块：我们将使用这个模块的HTTPClient类来执行HTTP请求。之后，我们将使用这个模块的AsyncHTTPClient
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int) class IndexHandler(tornado.web.RequestHandler): def get(self):
client = tornado.httpclient.HTTPClient()
response = client.fetch("http://www.cnblogs.com/lianzhilei") # 访问url，并返回response
self.write("""
<div style="text-align: center">
<div style="font-size: 72px">Time Cost</div>
<div style="font-size: 72px">%s</div>
</div>"""%(response.request_time) ) # 访问开销 if __name__ == "__main__":
tornado.options.parse_command_line()
app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
这个程序的结构现在对你而言应该已经很熟悉了：我们有一个RequestHandler类和一个处理到应用根路径请求的IndexHandler。在IndexHandler的get方法中，实例化了一个Tornado的HTTPClient类，然后调用结果对象的fetch方法，fetch方法会返回一个HTTPResponse对象，fetch方法返回的HTTPResponse对象允许你访问HTTP响应的任何部分
到目前为止，我们已经编写了 一个请求API并向浏览器返回结果的简单Tornado应用。尽管应用程序本身响应相当快，但是向API发送请求到获得返回的搜索数据之间有相当大的滞后。在同步（到目前为止，我们假定为单线程）应用，这意味着同时只能提供一个请求。所以，如果你的应用涉及一个2秒的API请求，你将每间隔一秒才能提供（最多！）一个请求。这并不是你所称的高可扩展性应用，即便扩展到多线程和/或多服务器 。
测试：
为了更具体的看出这个问题，我们对刚编写的例子进行基准测试。你可以使用任何基准测试工具来验证这个应用的性能，不过在这个例子中我们使用优秀的Siege utility工具进行测试。它可以这样使用：
[root@localhost siege-4.0.2]# siege http://192.168.1.210:8000/ -c100 -t3s
[root@localhost siege-4.0.2]# siege http://192.168.1.210:8000/ -c100 -t3s
** SIEGE 4.0.2
** Preparing 100 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200 0.09 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.19 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.27 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.34 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.44 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.54 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.62 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.72 secs: 207 bytes ==> GET /
HTTP/1.1 200 0.79 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.87 secs: 208 bytes ==> GET /
HTTP/1.1 200 0.95 secs: 208 bytes ==> GET /
HTTP/1.1 200 1.02 secs: 207 bytes ==> GET /
HTTP/1.1 200 1.10 secs: 208 bytes ==> GET /
HTTP/1.1 200 1.17 secs: 207 bytes ==> GET /
HTTP/1.1 200 1.29 secs: 207 bytes ==> GET /
HTTP/1.1 200 1.36 secs: 207 bytes ==> GET /
HTTP/1.1 200 1.44 secs: 208 bytes ==> GET /
HTTP/1.1 200 1.56 secs: 207 bytes ==> GET /
HTTP/1.1 200 1.64 secs: 208 bytes ==> GET /
HTTP/1.1 200 1.79 secs: 207 bytes ==> GET /
HTTP/1.1 200 1.92 secs: 207 bytes ==> GET /
HTTP/1.1 200 2.08 secs: 207 bytes ==> GET /
HTTP/1.1 200 2.23 secs: 207 bytes ==> GET /
HTTP/1.1 200 2.34 secs: 207 bytes ==> GET /
HTTP/1.1 200 2.42 secs: 208 bytes ==> GET /
HTTP/1.1 200 2.52 secs: 208 bytes ==> GET /
HTTP/1.1 200 2.67 secs: 207 bytes ==> GET /
HTTP/1.1 200 2.86 secs: 207 bytes ==> GET /
HTTP/1.1 200 2.94 secs: 208 bytes ==> GET / Lifting the server siege...
Transactions: 29 hits
Availability: 100.00 %
Elapsed time: 2.99 secs
Data transferred: 0.01 MB
Response time: 1.39 secs
Transaction rate: 9.70 trans/sec
Throughput: 0.00 MB/sec
Concurrency: 13.43
Successful transactions: 29
Failed transactions: 0
Longest transaction: 2.94
Shortest transaction: 0.00
基准测试输出
结果分析：100并发，请求3s，共完成29次请求，每秒处理9.7个请求，单线程下处理，请求要排队处理。这个例子只提供了一个非常简单的网页。如果你要添加其他Web服务或数据库的调用的话，结果会更糟糕。这种代码如果被 用到网站上，即便是中等强度的流量都会导致请求增长缓慢，甚至发生超时或失败。
2、基础异步（对外进行api请求）
幸运的是，Tornado包含一个AsyncHTTPClient类
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient from tornado.options import define, options
define("port", default=8000, help="run on the given port", type=int) class IndexHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous # 添加异步装饰器
def get(self):
client = tornado.httpclient.AsyncHTTPClient() # 生成AsyncHTTPClient实例,必须使用此类，否则无效果
client.fetch("http://www.cnblogs.com/lianzhilei",callback=self.on_response) # 绑定回调 def on_response(self,response): # response访问返回结果
self.write("""
<div style="text-align: center">
<div style="font-size: 72px">Time Cost</div>
<div style="font-size: 72px">%s</div>
</div>"""%(response.request_time) )
self.finish() # 结束 if __name__ == "__main__":
tornado.options.parse_command_line()
app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
syncHTTPClient的fetch方法并不返回调用的结果。取而代之的是它指定了一个callback参数；你指定的方法或函数将在HTTP请求完成时被调用，并使用HTTPResponse作为其参数。
client = tornado.httpclient.AsyncHTTPClient() # 生成AsyncHTTPClient实例
client.fetch("http://www.cnblogs.com/lianzhilei",callback=self.on_response) # 绑定回调
在这个例子中，我们指定on_response方法作为回调函数。我们之前使用期望的输出转化请求到网页中的所有逻辑被搬到了on_response函数中。还需要注意的是@tornado.web.asynchronous装饰器的使用（在get方法的定义之前）以及在回调方法结尾处调用的self.finish()。我们稍后将简要的讨论他们的细节
测试：
这个版本的应用拥有和之前同步版本相同的外观，但其性能更加优越。有多好呢？让我们看看基准测试的结果吧
[root@localhost ~]#siege http://192.168.1.210:8000/ -c100 -t3s
** SIEGE 3.0.
** Preparing concurrent users for battle.
The server is now under siege...
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.15 secs: bytes ==> GET /
HTTP/1.1 0.15 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.22 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.25 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.33 secs: bytes ==> GET /
HTTP/1.1 0.34 secs: bytes ==> GET /
HTTP/1.1 0.34 secs: bytes ==> GET /
HTTP/1.1 0.35 secs: bytes ==> GET /
HTTP/1.1 0.36 secs: bytes ==> GET /
HTTP/1.1 0.37 secs: bytes ==> GET /
HTTP/1.1 0.37 secs: bytes ==> GET /
HTTP/1.1 0.37 secs: bytes ==> GET /
HTTP/1.1 0.40 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.42 secs: bytes ==> GET /
HTTP/1.1 0.42 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.43 secs: bytes ==> GET /
HTTP/1.1 0.45 secs: bytes ==> GET /
HTTP/1.1 0.45 secs: bytes ==> GET /
HTTP/1.1 0.46 secs: bytes ==> GET /
HTTP/1.1 0.45 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.36 secs: bytes ==> GET /
HTTP/1.1 0.38 secs: bytes ==> GET /
HTTP/1.1 0.36 secs: bytes ==> GET /
HTTP/1.1 0.37 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.30 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.33 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.27 secs: bytes ==> GET /
HTTP/1.1 0.25 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.28 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.26 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.28 secs: bytes ==> GET /
HTTP/1.1 0.25 secs: bytes ==> GET /
HTTP/1.1 0.25 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.26 secs: bytes ==> GET /
HTTP/1.1 0.25 secs: bytes ==> GET /
HTTP/1.1 0.64 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.18 secs: bytes ==> GET /
HTTP/1.1 0.26 secs: bytes ==> GET /
HTTP/1.1 0.19 secs: bytes ==> GET /
HTTP/1.1 0.19 secs: bytes ==> GET /
HTTP/1.1 0.21 secs: bytes ==> GET /
HTTP/1.1 0.18 secs: bytes ==> GET /
HTTP/1.1 0.57 secs: bytes ==> GET /
HTTP/1.1 0.15 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.18 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.19 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.17 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.17 secs: bytes ==> GET /
HTTP/1.1 0.18 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.21 secs: bytes ==> GET /
HTTP/1.1 0.21 secs: bytes ==> GET /
HTTP/1.1 0.22 secs: bytes ==> GET /
HTTP/1.1 0.22 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.26 secs: bytes ==> GET /
HTTP/1.1 0.26 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.30 secs: bytes ==> GET /
HTTP/1.1 0.30 secs: bytes ==> GET /
HTTP/1.1 0.30 secs: bytes ==> GET /
HTTP/1.1 0.32 secs: bytes ==> GET /
HTTP/1.1 0.35 secs: bytes ==> GET /
HTTP/1.1 0.38 secs: bytes ==> GET /
HTTP/1.1 0.39 secs: bytes ==> GET /
HTTP/1.1 0.39 secs: bytes ==> GET /
HTTP/1.1 0.40 secs: bytes ==> GET /
HTTP/1.1 0.44 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.49 secs: bytes ==> GET /
HTTP/1.1 0.49 secs: bytes ==> GET /
HTTP/1.1 0.52 secs: bytes ==> GET /
HTTP/1.1 0.54 secs: bytes ==> GET /
HTTP/1.1 0.57 secs: bytes ==> GET /
HTTP/1.1 0.57 secs: bytes ==> GET /
HTTP/1.1 0.58 secs: bytes ==> GET /
HTTP/1.1 0.61 secs: bytes ==> GET /
HTTP/1.1 0.62 secs: bytes ==> GET /
HTTP/1.1 0.64 secs: bytes ==> GET /
HTTP/1.1 0.66 secs: bytes ==> GET /
HTTP/1.1 0.69 secs: bytes ==> GET /
HTTP/1.1 0.70 secs: bytes ==> GET /
HTTP/1.1 0.70 secs: bytes ==> GET /
HTTP/1.1 0.71 secs: bytes ==> GET /
HTTP/1.1 0.72 secs: bytes ==> GET /
HTTP/1.1 0.73 secs: bytes ==> GET /
HTTP/1.1 0.78 secs: bytes ==> GET /
HTTP/1.1 0.78 secs: bytes ==> GET /
HTTP/1.1 0.79 secs: bytes ==> GET /
HTTP/1.1 0.79 secs: bytes ==> GET /
HTTP/1.1 0.80 secs: bytes ==> GET /
HTTP/1.1 0.81 secs: bytes ==> GET /
HTTP/1.1 0.85 secs: bytes ==> GET /
HTTP/1.1 0.85 secs: bytes ==> GET /
HTTP/1.1 0.86 secs: bytes ==> GET /
HTTP/1.1 0.87 secs: bytes ==> GET /
HTTP/1.1 0.88 secs: bytes ==> GET /
HTTP/1.1 0.88 secs: bytes ==> GET /
HTTP/1.1 0.95 secs: bytes ==> GET /
HTTP/1.1 0.91 secs: bytes ==> GET /
HTTP/1.1 0.91 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.86 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.91 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.94 secs: bytes ==> GET /
HTTP/1.1 0.94 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.93 secs: bytes ==> GET /
HTTP/1.1 0.95 secs: bytes ==> GET /
HTTP/1.1 0.94 secs: bytes ==> GET /
HTTP/1.1 0.96 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET /
HTTP/1.1 0.94 secs: bytes ==> GET /
HTTP/1.1 0.95 secs: bytes ==> GET /
HTTP/1.1 0.95 secs: bytes ==> GET /
HTTP/1.1 0.94 secs: bytes ==> GET /
HTTP/1.1 0.92 secs: bytes ==> GET / Lifting the server siege... done. Transactions: hits
Availability: 100.00 %
Elapsed time: 2.32 secs
Data transferred: 0.04 MB
Response time: 0.44 secs
Transaction rate: 75.86 trans/sec
Throughput: 0.02 MB/sec
Concurrency: 33.71
Successful transactions:
Failed transactions:
Longest transaction: 0.96
Shortest transaction: 0.12 FILE: /root/siege.log
You can disable this annoying message by editing
the .siegerc file in your home directory; change
the directive 'show-logfile' to false.
基准测试输出
我们从同步版本的每秒3.20个事务提升到了75.86个，在相同的时间内总共提供了176次请求。这真是一个非常大的改善！正如你所想象的，当扩展到更多用户和更长时间时，它将能够提供更多连接，并且不会遇到同步版本遭受的变慢的问题。
3、异步装饰器和finish方法
Tornado默认在函数处理返回时关闭客户端的连接。在通常情况下，这正是你想要的。但是当我们处理一个需要回调函数的异步请求时，我们需要连接保持开启状态直到回调函数执行完毕。你可以在你想改变其行为的方法上面使用@tornado.web.asynchronous装饰器来告诉Tornado保持连接开启，正如我们在异步版本的推率例子中IndexHandler的get方法中所做的。下面是相关的代码片段：
@tornado.web.asynchronous # 添加异步装饰器
def get(self):
client = tornado.httpclient.AsyncHTTPClient() # 生成AsyncHTTPClient实例
client.fetch("http://www.cnblogs.com/lianzhilei",callback=self.on_response) # 绑定回调
记住当你使用@tornado.web.asynchonous装饰器时，Tornado永远不会自己关闭连接。你必须在你的RequestHandler对象中调用finish方法来显式地告诉Tornado关闭连接。（否则，请求将可能挂起，浏览器可能不会显示我们已经发送给客户端的数据。）在前面的异步示例中，我们在on_response函数的write后面调用了finish方法：
def on_response(self,response): # response访问返回结果
self.write("""
<div style="text-align: center">
<div style="font-size: 72px">Time Cost</div>
<div style="font-size: 72px">%s</div>
</div>"""%(response.request_time) )
self.finish() # 结束
4、异步生成器
现在，我们的推率程序的异步版本运转的不错并且性能也很好。不幸的是，它有点麻烦：为了处理请求 ，我们不得不把我们的代码分割成两个不同的方法。当我们有两个或更多的异步请求要执行的时候，编码和维护都显得非常困难，每个都依赖于前面的调用：不久你就会发现自己调用了一个回调函数的回调函数的回调函数。下面就是一个构想出来的（但不是不可能的）例子：
def get(self):
client = AsyncHTTPClient()
client.fetch("http://example.com", callback=on_response) def on_response(self, response):
client = AsyncHTTPClient()
client.fetch("http://another.example.com/", callback=on_response2) def on_response2(self, response):
client = AsyncHTTPClient()
client.fetch("http://still.another.example.com/", callback=on_response3) def on_response3(self, response):
[etc., etc.]
幸运的是，Tornado 2.1版本引入了tornado.gen模块，可以提供一个更整洁的方式来执行异步请求。代码清单5-3就是使用了tornado.gen版本的推率应用源代码。让我们先来看一下，然后讨论它是如何工作的。
import tornado.httpserver
import tornado.ioloop
import tornado.options
import tornado.web
import tornado.httpclient
import tornado.gen from tornado.options import define, options define("port", default=8000, help="run on the given port", type=int) class IndexHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.gen.engine
def get(self):
client = tornado.httpclient.AsyncHTTPClient()
response = yield tornado.gen.Task(client.fetch,"http://www.cnblogs.com/lianzhilei" )
# 访问url，并返回response
self.write("""
<div style="text-align: center">
<div style="font-size: 72px">Time Cost</div>
<div style="font-size: 72px">%s</div>
</div>""" % (response.request_time)) # 访问开销
self.finish() if __name__ == "__main__":
tornado.options.parse_command_line()
app = tornado.web.Application(handlers=[(r"/", IndexHandler)])
http_server = tornado.httpserver.HTTPServer(app)
http_server.listen(options.port)
tornado.ioloop.IOLoop.instance().start()
正如你所看到的，这个代码和前面两个版本的代码非常相似。主要的不同点是我们如何调用Asynchronous对象的fetch方法。下面是相关的代码部分：
@tornado.web.asynchronous
@tornado.gen.engine
def get(self):
client = tornado.httpclient.AsyncHTTPClient()
response = yield tornado.gen.Task(client.fetch,"http://www.cnblogs.com/lianzhilei" )
# 访问url，并返回response
我们使用Python的yield关键字以及tornado.gen.Task对象的一个实例，将我们想要的调用和传给该调用函数的参数传递给那个函数。这里，yield的使用返回程序对Tornado的控制，允许在HTTP请求进行中执行其他任务。当HTTP请求完成时，RequestHandler方法在其停止的地方恢复。这种构建的美在于它在请求处理程序中返回HTTP响应，而不是回调函数中。因此，代码更易理解：所有请求相关的逻辑位于同一个位置。而HTTP请求依然是异步执行的，所以我们使用tornado.gen可以达到和使用回调函数的异步请求版本相同的性能
[root@localhost ~]# siege http://192.168.0.2:8000/ -c100 -t3s
** SIEGE 3.0.
** Preparing concurrent users for battle.
The server is now under siege...
HTTP/1.1 0.09 secs: bytes ==> GET /
HTTP/1.1 0.09 secs: bytes ==> GET /
HTTP/1.1 0.11 secs: bytes ==> GET /
HTTP/1.1 0.11 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.13 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.21 secs: bytes ==> GET /
HTTP/1.1 0.22 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.22 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.23 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.30 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.33 secs: bytes ==> GET /
HTTP/1.1 0.33 secs: bytes ==> GET /
HTTP/1.1 0.35 secs: bytes ==> GET /
HTTP/1.1 0.38 secs: bytes ==> GET /
HTTP/1.1 0.39 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.40 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.40 secs: bytes ==> GET /
HTTP/1.1 0.41 secs: bytes ==> GET /
HTTP/1.1 0.43 secs: bytes ==> GET /
HTTP/1.1 0.45 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.40 secs: bytes ==> GET /
HTTP/1.1 0.35 secs: bytes ==> GET /
HTTP/1.1 0.34 secs: bytes ==> GET /
HTTP/1.1 0.49 secs: bytes ==> GET /
HTTP/1.1 0.31 secs: bytes ==> GET /
HTTP/1.1 0.19 secs: bytes ==> GET /
HTTP/1.1 0.25 secs: bytes ==> GET /
HTTP/1.1 0.21 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.46 secs: bytes ==> GET /
HTTP/1.1 0.27 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.24 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.14 secs: bytes ==> GET /
HTTP/1.1 0.60 secs: bytes ==> GET /
HTTP/1.1 0.21 secs: bytes ==> GET /
HTTP/1.1 0.34 secs: bytes ==> GET /
HTTP/1.1 0.15 secs: bytes ==> GET /
HTTP/1.1 0.16 secs: bytes ==> GET /
HTTP/1.1 0.08 secs: bytes ==> GET /
HTTP/1.1 0.08 secs: bytes ==> GET /
HTTP/1.1 0.09 secs: bytes ==> GET /
HTTP/1.1 0.07 secs: bytes ==> GET /
HTTP/1.1 0.08 secs: bytes ==> GET /
HTTP/1.1 0.09 secs: bytes ==> GET /
HTTP/1.1 0.08 secs: bytes ==> GET /
HTTP/1.1 0.08 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.12 secs: bytes ==> GET /
HTTP/1.1 0.19 secs: bytes ==> GET /
HTTP/1.1 0.19 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.20 secs: bytes ==> GET /
HTTP/1.1 0.26 secs: bytes ==> GET /
HTTP/1.1 0.28 secs: bytes ==> GET /
HTTP/1.1 0.28 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.29 secs: bytes ==> GET /
HTTP/1.1 0.34 secs: bytes ==> GET /
HTTP/1.1 0.36 secs: bytes ==> GET /
HTTP/1.1 0.36 secs: bytes ==> GET /
HTTP/1.1 0.36 secs: bytes ==> GET /
HTTP/1.1 0.37 secs: bytes ==> GET /
HTTP/1.1 0.42 secs: bytes ==> GET /
HTTP/1.1 0.43 secs: bytes ==> GET /
HTTP/1.1 0.43 secs: bytes ==> GET /
HTTP/1.1 0.44 secs: bytes ==> GET /
HTTP/1.1 0.45 secs: bytes ==> GET /
HTTP/1.1 0.48 secs: bytes ==> GET /
HTTP/1.1 0.49 secs: bytes ==> GET /
HTTP/1.1 0.50 secs: bytes ==> GET /
HTTP/1.1 0.51 secs: bytes ==> GET /
HTTP/1.1 0.52 secs: bytes ==> GET /
HTTP/1.1 0.56 secs: bytes ==> GET /
HTTP/1.1 0.57 secs: bytes ==> GET /
HTTP/1.1 0.58 secs: bytes ==> GET /
HTTP/1.1 0.59 secs: bytes ==> GET /
HTTP/1.1 0.59 secs: bytes ==> GET /
HTTP/1.1 0.64 secs: bytes ==> GET /
HTTP/1.1 0.65 secs: bytes ==> GET /
HTTP/1.1 0.65 secs: bytes ==> GET /
HTTP/1.1 0.67 secs: bytes ==> GET /
HTTP/1.1 0.67 secs: bytes ==> GET /
HTTP/1.1 0.71 secs: bytes ==> GET /
HTTP/1.1 0.72 secs: bytes ==> GET /
HTTP/1.1 0.73 secs: bytes ==> GET /
HTTP/1.1 0.74 secs: bytes ==> GET /
HTTP/1.1 0.76 secs: bytes ==> GET /
HTTP/1.1 0.79 secs: bytes ==> GET /
HTTP/1.1 0.80 secs: bytes ==> GET /
HTTP/1.1 0.81 secs: bytes ==> GET /
HTTP/1.1 0.82 secs: bytes ==> GET /
HTTP/1.1 0.83 secs: bytes ==> GET /
HTTP/1.1 0.88 secs: bytes ==> GET /
HTTP/1.1 0.89 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.90 secs: bytes ==> GET /
HTTP/1.1 0.86 secs: bytes ==> GET /
HTTP/1.1 0.88 secs: bytes ==> GET /
HTTP/1.1 0.88 secs: bytes ==> GET /
HTTP/1.1 0.87 secs: bytes ==> GET /
HTTP/1.1 0.87 secs: bytes ==> GET /
HTTP/1.1 0.91 secs: bytes ==> GET /
HTTP/1.1 0.95 secs: bytes ==> GET /
HTTP/1.1 0.94 secs: bytes ==> GET / Lifting the server siege... done. Transactions: hits
Availability: 100.00 %
Elapsed time: 2.14 secs \Data transferred: 0.03 MB
Response time: 0.40 secs
Transaction rate: 63.08 trans/sec
Throughput: 0.01 MB/sec
Concurrency: 25.20
Successful transactions:
Failed transactions:
Longest transaction: 0.95
Shortest transaction: 0.07 FILE: /root/siege.log
You can disable this annoying message by editing
the .siegerc file in your home directory; change
the directive 'show-logfile' to false.
基准测试输出
记住@tornado.gen.engine装饰器的使用需要刚好在get方法的定义之前；这将提醒Tornado这个方法将使用tornado.gen.Task类。tornado.gen模块还哟一些其他类和函数可以方便Tornado的异步编程。查阅一下文档[1]是非常值得的。
总结：
正如我们在前面的例子中所看到的，Tornado异步Web发服务不仅容易实现也在实践中有着不容小觑的能力。使用异步处理可以让我们的应用在长时间的API和数据库请求中免受阻塞之苦，最终更快地提供更多请求。尽管不是所有的处理都能从异步中受益--并且实际上尝试整个程序非阻塞会迅速使事情变得复杂--但Tornado的非阻塞功能可以非常方便的创建依赖于缓慢查询或外部服务的Web应用。
不过，值得注意的是，这些例子都非常的做作。如果你正在设计一个任何规模下带有该功能的应用，你可能希望客户端浏览器来执行Twitter搜索请求（使用JavaScript），而让Web服务器转向提供其他请求。在大多数情况下，你至少希望将结果缓存以便两次相同搜索项的请求不会导致再次向远程API执行完整请求。通常，如果你在后端执行HTTP请求提供网站内容，你可能希望重新思考如何建立你的应用。
考虑到这一点，在下一组示例中，我们将看看如何在前端使用像JavaScript这样的工具处理异步应用，让客户端承担更多工作，以提高你应用的扩展性。
*/*上面的异步例子没有什么卵用，生产环境下也很少用到服务器端向外部api发出请求，支撑起上面异步功能实现的一个重要点是AsyncHTTPClient异步模块，换成其他业务处理（大量生产应用涉及与数据库交互）时根本无法套用，只能作为一个示例展示。下面会有生产环境下的具体应用案例
《Tornado异步二》------跳转
文件上传 multipart/form-data
#!/usr/bin/env python
# -*- coding:utf-8 -*- import os
import uuid
import logging
import traceback
import tornado.httpclient
import tornado.web
from tornado.concurrent import run_on_executor
from concurrent.futures import ThreadPoolExecutor
from conf.settings import UPLOAD_PATH ORIGINAL_STATUS = 200
ERROR_UNKOWN = 500
ERROR_FILE_EXISTS = 501
ERROR_FILE_EMPTY = 502 class Media(tornado.web.RequestHandler):
"""所有请求处理入口"""
executor = ThreadPoolExecutor(5) # 启动5个线程处理阻塞请求 def __init__(self, application, request):
tornado.web.RequestHandler.__init__(self, application, request) @run_on_executor
def post(self, *args, **kwargs):
result = self.execute('POST', *args)
self.write(result) @run_on_executor
def delete(self, *args, **kwargs):
logging.info(self.request.arguments)
result = self.execute('DELETE', *args)
self.write(result) def execute(self, method, *args):
'''
上传，删除文件
:param method:
:param args:
:return:
'''
STATUS = ORIGINAL_STATUS
REASON = 'OK' # 上传
if method == 'POST':
try:
for direction in self.request.files:
for content in self.request.files[direction]:
filename = content['filename']
logging.info('direction %s filename %s', direction, filename)
abspath = os.path.join(UPLOAD_PATH, direction, filename)
abspath.replace('\\', '/')
if os.path.exists(abspath):
STATUS = ERROR_FILE_EXISTS
REASON = 'file %s already exists' % abspath
logging.error(REASON)
continue parent_direction = os.path.dirname(abspath)
if not os.path.exists(parent_direction):
logging.info('make direction %s', parent_direction)
os.makedirs(parent_direction) logging.info('start recv file %s', abspath)
with open(abspath, 'wb') as file:
file.write(content['body'])
logging.info('finish recv file %s', abspath) if not self.request.files:
STATUS = ERROR_FILE_EMPTY
REASON = 'the upload file is empty'
logging.error(REASON) except Exception as e:
exec = traceback.format_exc()
logging.error(exec)
STATUS = ERROR_UNKOWN
REASON = 'internal error！ fail to upload' elif method == 'DELETE':
pass
response = self.create_response(code=STATUS, reason=REASON)
return response def create_response(self, code, reason):
'''
发送回复信息
:param code:
:param reason:
:param callid:
:return:
'''
data = {
'code': code,
'reason': reason,
}
return data
另一种表单方式 application/x-www-form-urlencoded
#!/usr/bin/env python
# -*- coding:utf-8 -*- import os
import uuid
import logging
import traceback
import tornado.httpclient
import tornado.web
from tornado.concurrent import run_on_executor
from concurrent.futures import ThreadPoolExecutor
from conf.settings import UPLOAD_PATH ORIGINAL_STATUS = 200
ERROR_UNKOWN = 500
ERROR_FILE_EXISTS = 501
ERROR_FILE_EMPTY = 502
ERROR_MISS_ARGUMENT = 503
ERROR_FILE_FORMAT_NOT_SUPPORTED = 504 class Media(tornado.web.RequestHandler):
"""所有请求处理入口"""
executor = ThreadPoolExecutor(5) # 启动5个线程处理阻塞请求 def __init__(self, application, request):
tornado.web.RequestHandler.__init__(self, application, request) @run_on_executor
def post(self, *args, **kwargs):
result = self.execute('POST', *args)
self.write(result) @run_on_executor
def delete(self, *args, **kwargs):
logging.info(self.request.arguments)
result = self.execute('DELETE', *args)
self.write(result) def execute(self, method, *args):
'''
上传，删除文件
:param method:
:param args:
:return:
'''
STATUS = ORIGINAL_STATUS
REASON = 'OK'
try:
# 上传
if method == 'POST':
directory = self.get_argument('directory',None)
filename = self.get_argument('filename',None)
body = self.request.arguments['body']
logging.info('post directory %s filename %s',directory,filename)
if not directory or not filename:
STATUS = ERROR_MISS_ARGUMENT
REASON = 'miss argument directory %s filename %s' %\
(directory,filename)
logging.error(REASON)
return self.create_response(code=STATUS, reason=REASON) if not body:
STATUS = ERROR_MISS_ARGUMENT
REASON = 'miss argument body %s' %\
(body[0])
logging.error(REASON)
return self.create_response(code=STATUS, reason=REASON) abspath = os.path.join(UPLOAD_PATH, directory, filename)
abspath = abspath.replace('\\', '/')
if os.path.exists(abspath):
STATUS = ERROR_FILE_EXISTS
REASON = 'file %s already exists' % abspath
logging.error(REASON)
return self.create_response(code=STATUS, reason=REASON) _,extension = os.path.splitext(filename)
if extension not in ('.mp3','.wav'):
STATUS = ERROR_FILE_FORMAT_NOT_SUPPORTED
REASON = 'file fromat %s not supported' % extension
logging.error(REASON)
return self.create_response(code=STATUS, reason=REASON) parent_direction = os.path.dirname(abspath)
if not os.path.exists(parent_direction):
logging.info('make direction %s', parent_direction)
os.makedirs(parent_direction)
logging.info('start recv file %s', abspath)
with open(abspath, 'wb') as file:
file.write(body[0])
logging.info('finish recv file %s', abspath) elif method == 'DELETE':
directory = self.get_argument('directory', None)
filename = self.get_argument('filename', None)
logging.info('delete directory %s filename %s', directory, filename)
if not directory or not filename:
STATUS = ERROR_MISS_ARGUMENT
REASON = 'miss argument directory %s filename %s' % \
(directory, filename)
logging.error(REASON)
return self.create_response(code=STATUS, reason=REASON) abspath = os.path.join(UPLOAD_PATH, directory, filename)
abspath = abspath.replace('\\', '/')
if os.path.exists(abspath):
logging.info('remove file %s',abspath)
os.remove(abspath) except Exception as e:
exec = traceback.format_exc()
logging.error(exec)
STATUS = ERROR_UNKOWN
REASON = 'internal error!' return self.create_response(code=STATUS, reason=REASON) def create_response(self, code, reason):
'''
发送回复信息
:param code:
:param reason:
:param callid:
:return:
'''
data = {
'code': code,
'reason': reason,
}
return data
Python开发【Tornado】：异步Web服务（一）的更多相关文章
- 《Introduction to Tornado》中文翻译计划——第五章：异步Web服务
http://www.pythoner.com/294.html 本文为<Introduction to Tornado>中文翻译,将在https://github.com/alioth3 ...
- 第五章：异步Web服务
到目前为止,我们已经看到了许多使Tornado成为一个Web应用强有力框架的功能.它的简单性.易用性和便捷性使其有足够的理由成为许多Web项目的不错的选择.然而,Tornado受到最多关注的功能是其异 ...
- tornado异步web请求
1.为什么要使用异步web服务使用异步非阻塞请求,并发处理更高效. 2.同步与异步请求比较同步请求时,web服务器进程是阻塞的,也就是说当一个请求被处理时,服务器进程会被挂起直至请求完成. 异步请求时 ...
- 如何设计一个异步Web服务——任务调度
接上一篇<如何设计一个异步Web服务——接口部分> Application已经将任务信息发到了Service服务器中,接下来,Service服务器改如何对自身的资源进行合理分配以满足App ...
- 如何设计一个异步Web服务——接口部分
需求比较简单,提供一个异步Web服务供使用者调用.比如说,某应用程序需要批量地给图片加lomo效果.由于加lomo效果这个操作非常消耗CPU资源,所以我们需要把这个加lomo效果的程序逻辑放到一台单独 ...
- Python开发【Tornado】：异步Web服务（二）
真正的 Tornado 异步非阻塞 前言: 其中 Tornado 的定义是 Web 框架和异步网络库,其中他具备有异步非阻塞能力,能解决他两个框架请求阻塞的问题,在需要并发能力时候就应该使用 Torn ...
- asyncio创建协程解析——分析廖雪峰的Python教程之创建WEB服务（转）
第一步,搭建开发环境 所需第三方库: aiohttp,异步 Web 开发框架:jinja2,前端模板引擎:aiomysql,异步 mysql 数据库驱动 所需内置库: logging,系统日志:asy ...
- tornado之异步web服务二
前面介绍了异步方法带来的显著性能提升.我们来看下异步使用的方法. 1 首先对于get方法使用tornado.web.asynchronous进行装饰.当使用tornado.web.asynchrono ...
- tornado之异步web服务一
大部分Web应用(包括我们之前的例子)都是阻塞性质的,也就是说当一个请求被处理时,这个进程就会被挂起直至请求完成.在大多数情况下,Tornado处理的Web请求完成得足够快使得这个问题并不需要被关注. ...
随机推荐
- while(scanf("%d",&n)!=EOF)与while(cin>>n)
我们知道scanf函数是C语言里面的,其返回值是,被输入函数成功赋值的变量个数.针对于int counts = scanf("%d",&n);来说如果赋值成功那么其返回值 ...
- NPOI从数据库中调取数据直接导出到EXCEL中
关于NPOI NPOI是POI项目的.NET版本,是由@Tony Qu(http://tonyqus.cnblogs.com/)等大侠基于POI开发的,可以从http://npoi.codeplex. ...
- Navicat for MySQL使用手记(上）--创建数据库和表
在管理MySQL数据库的图形化工具中,最为熟知的就是phpMyAdmin和Mysql-Front了,今天跟大家分享另外一个管理mysql数据库的另外一个利器---Navicat MySQL. Navi ...
- oracle客户端免安装配置、64位机器PL/SQL和VS自带的IIS连接问题
一.oracle客户端免安装配置 1.到oracle官网下载Oracle InstantClient, 把它解压缩到单独目录,例如C:\OracleClient,2. 添加环境变量 ORACLE_HO ...
- Effective C++ Item 16 Use the same form in corresponding uses of new and delete
1. When you created an array and want to return the memory to system. You need to explicitly add [] ...
- POJ 1321 棋盘问题(状态压缩DP)
不总结的话, 同一个地方会 WA 到死 思路: 状态压缩 DP. 1. s 表示压缩状态, 若第 i 列放了棋子, 那么该列置 1, 否则该列置 0. 假如 s = 3(0x011) 那么表示棋盘的第 ...
- cocos2dx游戏--欢欢英雄传说--添加血条
用一个空血槽图片的Sprite做背景,上面放一个ProgressTimer, 通过设置ProgressTimer的进度来控制血条的长短.建立一个Progress类来实现.Progress.h: #if ...
- Android项目结构介绍
src/存放Java源代码gen/中存放系统自动生成的配置文件Android 4.4.2下包含android.jar文件,这是一个Java归档文件,其中包含构建应用程序所需的所有的Android SD ...
- 集群应用Session一致性实现的三种方案
转自:http://blog.csdn.net/zwx521515/article/details/78679679 https://www.cnblogs.com/study-everyday/p/ ...
- Java中DESKeySpec类
此类位于 javax.crypto.spec 包下.声明如下: public class DESKeySpec extends Object implements KeySpec 此类指定一个 DES ...