Async IO
I was recently reading a series on “Write Sequential Non-Blocking IO Code With Fibers in NodeJS” by Venkatesh.
Venki was essentially trying to emphasize that writing non-blocking code in NodeJS (either via callbacks, or using promises) can get hairy really fast. For example, this code demonstrates that aptly:
var express = require('express');
var app = express();
app.get('/users/:fbId', function(req, res) {
var id = req.params.id;
var key = 'user:' + id;
client.get(key, function(err, reply) {
if (err !== null) {
res.send(500);
return;
}
if (reply === null) {
res.send(404);
return;
}
res.send(200, {id: id, name: reply});
});
});
The exact code is available on GitHub (so is the promises driven version, but I won’t bother inlining it.)
What we actually wanted to write (if it were possible, was):
var express = require('express');
var app = express();
app.get('/users/:fbId', function(req, res) {
var id = req.params.id;
var key = 'user:' + id;
try {
var reply = client.get(key);
if (reply === null) {
res.send(404);
return;
}
res.send(200, {id: id, name: reply});
}
catch(err) {
res.send(500);
}
});
The magic would happen in line number 9 (above.) Instead of having to provide a cascade of callbacks (what if we wanted to do another lookup after we got the value back from the first), we could just write them serially, one after the other.
Well. Apparently we can!
Fibers
A fiber is a particularly lightweight thread of execution. Like threads, fibers share address space. However, fibers use co-operative multitasking while threads use pre-emptive multitasking. Threads often depend on the kernel’s thread scheduler to preempt a busy thread and resume another thread; fibers yield themselves to run another fiber while executing.
Fibers allow exactly this kind of black magic in NodeJS. It is still callbacks internally, but we are exposed to none of it in our application code. Sure you will end up writing a bunch of wrappers (or have some tool generate them for us), but we would have the sweet sweet pleasure of writing async IO code without having to jump through all the hoops. This is how the wrapper code for redis client looks like:
var Fiber = require('fibers');
var client = require('./redis-client');
exports.get = function(key) {
var err, reply;
var fiber = Fiber.current;
client.get(key, function(_err, _reply) {
err = _err;
reply = _reply;
fiber.run();
});
Fiber.yield();
if (err != null) {
throw err;
}
return reply;
};
(the real code is here in case you are curious)
I liked how the code looked. Having survided a ‘promising’ node.js project, I was definitely curious about this new style. Maybe this can be the saving grace (before generators and yield take over the JS world) for real world server side JavaScript.
Fibers you say
But the code (and the underlying technique which makes it tick) sounded very familiar, and reminded me of a similar technique which is used in Go to allow writing beautiful async IO code. For example, the same function from above in Go:
m.Get("/users/:id", func(db *DB, params martini.Params) (int, []byte) {
str := params["id"]
id, err := strconv.Atoi(str)
if err != nil {
return http.StatusBadRequest, []byte{}
}
u, err := db.LoadUser(id)
if err != nil {
return http.StatusNotFound, []byte{}
}
return http.StatusOK, encoder.Must(enc.Encode(u))
})
Sure, there is a little more happening in here (Go is statically typed), buts its the exact same thing as the fibers example, without all the manual wrapping. Any call which does IO (like line 8) blocks the currently executing goroutine (just like a fiber, a lightweight thread.) The natural question to ask is, if the goroutine gets blocked, how do other requests get processed? Its quite simple actually. The Go runtime automatically schedules any other goroutine which is ready to run (their IO call is done) on the thread on which the current goroutine was running.
Since goroutines are light weight (stack size is just 4 KB in Go 1.3beta1 compared to the much larger ~2 MB thread stacks), it is not unusual to have hundreds of thousands of goroutines actively running in a single process, all humming along together. The best part, since the threads have to do less context switching (the same physical thread can continue running on the processor core, just the instruction pointer keeps changing as the goroutines shuffle in and out, just as in method calls), we are able to extract a lot more efficiency from the same unit of hardware than otherwise. Otherwise IO calls, which would otherwise cause the thread to block and wait, could cripple the system and bring it down to its knees. Read this article for more context on this.
Performance
A fellow ThoughtWorker asked me, “Does performance matter when choosing a framework?”
I know where he was coming from, and how we shouldn’t make decisions purely based on performance (we would all be doing assembly if that was the case.) While it is true that as a startup (or even in the case of a well established player), building the MVP and getting it to the users is paramount, you really dont want to face the situation where you suddenly have a huge influx of users (say it goes viral) and you are caught between a ROCK (scale horizontally by throwing compute units at the problem) and a HARD PLACE (have to rewrite the solution in a technology more amenable to scaling.) Both of these options are expensive, and can potentially be a deal breaker.
Therefore, provided everything else is more or less equal, choosing the more performant one is never a bad thing.
With this context, I decided to compare the two solutions for their performance, given that they more or less looked the same. I decided to allow the system under test to use as many cores as they wanted, and then hit them with 100 concurrent users, each of which is going full tilk for around 20 seconds (used the awesome wrktool for benchmarking.)
The results:
Golang | |
---|---|
Stdlib | 134566 (3.81ms) |
Gorilla | 125092 (4.28ms) |
Martini | 51330 (9.51ms) |
node.js | |
---|---|
Stdlib | 54510 (7.78ms) |
Callbacks* | 36107 (10.84ms) |
Fibers* | 27372 (18.76ms) |
Promises* | 22665 (17.15ms) |
* The Callbacks, Fibers and Promises versions are created using Express. The Stdlib versions use the http support in the corresponding standard libraries.
All the numbers are in req/s as given by wrk (higher is better.) The latency details are in brackets (lower is better.) Clicking the numbers will take you to the corresponding code in the GitHub repo (the README has the detailed numbers.)
The tests were done on an updated Ubuntu 14.04 box with a Intel i7 4770 processor, 16 GB of RAM and a SSD.
As you can see, the fibers method of doing async IO in node.js comes with a perceivable loss in throughput compared to the pure callbacks based approach, but looks relatively better than the promises version for this micro-benchmark.
At the same time, the default way of doing IO in Golang does very well for itself. More than 134,000 req/s with a 3.81 ms 99th percentile latency. All this without having to go through crazy callbacks/promises hoops. How cool is that?
How the tests were run?
Software versions
- Go 1.3beta1
- node.js 0.10.28
- wrk 3.1.0
Command used to run
A more detailed description is available in the README but I will explain a simple version here:
- Start the program (by say running ./start_martini.sh)
- Run the benchmark (by running ./bench.sh)
- Record the result
- Rince and repeat 3 times and take the best run
Notes
- All cores on the Intel i7 4770 were set to the performance governor
- Redis was not tweaked
- ulimit was not raised
Summary
This is part 1 in a multipart series looking at how async IO (and programming in general) is done in various languages/platforms. We will be going indepth into one language/platform with the every new article in the series. Future parts will look at Scala, Clojure, Java, C#, Python and Ruby based frameworks and try and present a holistic view of the async world.
But one thing is very clear, async IO is here to stay. Not embrassing it would be foolhardy given the need to stay lean. Hope these articles help you understand gravity of the decision.
While some might argue that what we did in Golang was not really async, as the call was blocking in nature. But the net result achieved, and the reason why Go is still able to provide an awesome throughput despite blocking IO calls, is because the Go runtime essentially does the heavy lifting for you. When one goroutine is busy waiting for the results of a IO call to come back, other goroutines can take their place and not waste CPU cycles. The fact that this mechanism allows us to get away with fewer threads that would be required otherwise, is the icing on top.
Async IO的更多相关文章
- 一款DMA性能优化记录:异步传输和指定实时信号做async IO
关键词:DMA.sync.async.SIGIO.F_SETSIG. DMA本身用于减轻CPU负担,进行CPU off-load搬运工作. 在DMA驱动内部实现有同步和异步模式,异步模式使用dma_a ...
- [Functional Programming] Async IO Functor
We will see a peculiar example of a pure function. This function contained a side-effect, but we dub ...
- 关于Blocking IO,non-Blokcing IO,async IO的区别和理解
来源:http://shmilyaw-hotmail-com.iteye.com/blog/1896683 概括来说,一个IO操作可以分为两个部分:发出请求.结果完成.如果从发出请求到结果返回,一直B ...
- ORACLE数据库异步IO介绍
异步IO概念 Linux 异步 I/O (AIO)是 Linux 内核中提供的一个增强的功能.它是Linux 2.6 版本内核的一个标准特性,当然我们在2.4 版本内核的补丁中也可以找到它.AIO 背 ...
- 为什么我们要使用Async、Await关键字
前不久,在工作中由于默认(xihuan)使用Async.Await关键字受到了很多质问,所以由此引发这篇博文“为什么我们要用Async/Await关键字”,请听下面分解: Async/Await关键字 ...
- [翻译]各个类型的IO - 阻塞, 非阻塞,多路复用和异步
同事推荐,感觉写的不错就试着翻译了下. 原文链接: https://www.rubberducking.com/2018/05/the-various-kinds-of-io-blocking-non ...
- 《Linux/UNIX系统编程手册》第63章 IO多路复用、信号驱动IO以及epoll
关键词:fasync_helper.kill_async.sigsuspend.sigaction.fcntl.F_SETOWN_EX.F_SETSIG.select().poll().poll_wa ...
- linux io的cfq代码理解
内核版本: 3.10内核. CFQ,即Completely Fair Queueing绝对公平调度器,原理是基于时间片的角度去保证公平,其实如果一台设备既有单队列,又有多队列,既有快速的NVME,又有 ...
- Haskell语言学习笔记(85)Async
安装 async $ cabal install async async-2.2.1 installed async / wait / concurrently async :: IO a -> ...
随机推荐
- myeclipse maven 安装
myeclipse 上安装 Maven3 环境准备: JDK 1.6.45 Maven 3.0.5 myeclipse 8.5 安装 Maven 之前要求先确定你的 JDK 已经安装配置完成.Ma ...
- python之面向对象
首先我们应该知道,python在设计之初就已经是一门面向对象的语言,所以说python中创建一个类和对象是很容易的. 面向对象的技术简介 类(class):用来描述具有相同的属性和方法的对象的集合.它 ...
- CentOS 7.2安装docker-compose运行gitlib
服务器已经安装好docker 启动: service start docker 安装docker-compose: 安装pip:sudo yum search pip / sudo yum inst ...
- log4net配置
1.configuration配置 <configSections> <section name="log4net" type="log4net.Con ...
- Eclipse下配置javaweb项目快速部署到tomcat
用惯了VS,再用Eclipse,完全有一种从自动挡到手动挡的感觉啊. 很多同学在Eclipse下开发web项目,每一次修改代码,看效果的时候都有右键项目->Run as -> Run on ...
- Oracle over函数
Oracle over函数 SQL code: sql over的作用及用法RANK ( ) OVER ( [query_partition_clause] order_by_clause )DE ...
- web适配问题
bootstrap模板为使IE6.7.8版本(IE9以下版本)浏览器兼容html5新增的标签,引入下面代码文件即可. <script src="https://oss.maxcdn.c ...
- java io流 图片和字符串之间的转换
package com.yundongsports.arena.controller.basketballsite;import com.yundongsports.arena.ResponseVo. ...
- dos下对mysql的简单操作(linux类似)
>>>>>>>>>>>>>>>>>>>> 基础入门语句10条 1. 连接服务器 ...
- Json格式示意图
json视图工具:http://www.bejson.com/jsonviewernew/ 一.Json格式化,(看到数组里面又有数组一下子疑问不是合格json):尾门地址查询: =>=> ...