Thread pools & Executors

Run your concurrent code in a performant way

All about thread pools #

How do I use the thread pools? #

Wangle provides two concrete thread pools (IOThreadPoolExecutor, CPUThreadPoolExecutor) as well as building them in as part of a complete async framework. Generally you might want to grab the global executor, and use it with a future, like this:

auto f = someFutureFunction().via(getCPUExecutor()).then(...)

Or maybe you need to construct a thrift/memcache client, and need an event base:

auto f = getClient(getIOExecutor()->getEventBase())->callSomeFunction(args...)
.via(getCPUExecutor())
.then([](Result r){ .... do something with result});

vs. C++11's std::launch #

The current C++11 std::launch only has two modes: async or deferred. In a production system, neither is what you want: async will launch a new thread for every launch without limit, while deferred will defer the work until it is needed lazily, but then do the work in the current thread synchronously when it is needed.

Wangle's thread pools always launch work as soon as possible, have limits to the maximum number of tasks / threads allowed, so we will never use more threads than absolutely needed. See implementation details below about each type of executor.

Why do we need yet another set of thread pools? #

Unfortunately none of the existing thread pools had every feature needed - things based on pipes are too slow. Several older ones didn't support std::function.

Why do we need several different types of thread pools? #

If you want epoll support, you need an fd - event_fd is the latest notification hotness. Unfortunately, an active fd triggers all the epoll loops it is in, leading to thundering herd - so if you want a fair queue (one queue total vs. one queue per worker thread), you need to use some kind of semaphore. Unfortunately semaphores can't be put in epoll loops, so they are incompatible with IO. Fortunately, you usually want to separate the IO and CPU bound work anyway to give stronger tail latency guarantees on IO.

IOThreadPoolExecutor #

  • Uses event_fd for notification, and waking an epoll loop.
  • There is one queue (NotificationQueue specifically) per thread/epoll.
  • If the thread is already running and not waiting on epoll, we don't make any additional syscalls to wake up the loop, just put the new task in the queue.
  • If any thread has been waiting for more than a few seconds, its stack is madvised away. Currently however tasks are scheduled round robin on the queues, so unless there is no work going on, this isn't very effective.
  • ::getEventBase() will return an EventBase you can schedule IO work on directly, chosen round-robin.
  • Since there is one queue per thread, there is hardly any contention on the queues - so a simple spinlock around an std::deque is used for the tasks. There is no max queue size.
  • By default, there is one thread per core - it usually doesn't make sense to have more IO threads than this, assuming they don't block.

CPUThreadPoolExecutor #

  • A single queue backed by folly/LifoSem and folly/MPMC queue. Since there is only a single queue, contention can be quite high, since all the worker threads and all the producer threads hit the same queue. MPMC queue excels in this situation. MPMC queue dictates a max queue size.
  • LifoSem wakes up threads in Lifo order - i.e. there are only few threads as necessary running, and we always try to reuse the same few threads for better cache locality.
  • Inactive threads have their stack madvised away. This works quite well in combination with Lifosem - it almost doesn't matter if more threads than are necessary are specified at startup.
  • stop() will finish all outstanding tasks at exit
  • Supports priorities - priorities are implemented as multiple queues - each worker thread checks the highest priority queue first. Threads themselves don't have priorities set, so a series of long running low priority tasks could still hog all the threads. (at last check pthreads thread priorities didn't work very well)

ThreadPoolExecutor #

Base class that contains the thread startup/shutdown/stats logic, since this is pretty disjoint from how tasks are actually run

Observers #

An observer interface is provided to listen for thread start/stop events. This is useful to create objects that should be one-per-thread, but also have them work correctly if threads are added/removed from the thread pool.

Stats #

PoolStats are provided to get task count, running time, waiting time, etc.

Thread pools & Executors的更多相关文章

  1. Java theory and practice: Thread pools and work queues--reference

    Why thread pools? Many server applications, such as Web servers, database servers, file servers, or ...

  2. java 多线程--- Thread Runnable Executors

    java 实现多线程的整理: Thread实现多线程的两种方式: (1)继承 Thread类,同时重载 run 方法: class PrimeThread extends Thread { long ...

  3. 小规模的流处理框架.Part 1: thread pools

    原文链接:http://ifeve.com/part-1-thread-pools/ 很不错的一篇文章

  4. Thread Pools

    许多程序会动态创建数十个设置上百个线程.举个例子,一个web服务器可能在每一个新到来的请求时创建一个新线程,然后在请求完成后将其终止. 然而,创建一个新线程将会带来一定的耗费:它需要在内核中创建自身必 ...

  5. java thread reuse(good)

    I have always read that creating threads is expensive. I also know that you cannot rerun a thread. I ...

  6. 【转】Java 并发:Executors 和线程池

    原文地址: http://baptiste-wicht.com/posts/2010/09/java-concurrency-part-7-executors-and-thread-pools.htm ...

  7. The threads in the thread pool will process the requests on the connections concurrently.

    https://docs.oracle.com/javase/tutorial/essential/concurrency/pools.html Most of the executor implem ...

  8. The CLR's Thread Pool

    We were unable to locate this content in zh-cn. Here is the same content in en-us. .NET The CLR's Th ...

  9. Effective Java 73 Avoid thread groups

    Thread groups were originally envisioned as a mechanism for isolating applets for security purposes. ...

随机推荐

  1. C#调用C++编译的DLL--

    引用 C#调用C++编译的DLL详解 可以单步跟踪进入dll的源码!每种编程语言调用DLL的方法都不尽相同,在此只对用C#调用DLL的方法进行介绍.首先,您需要了解什么是托管,什么是非托管.一般可以认 ...

  2. SpringMVC札集(07)——JSON数据

    自定义View系列教程00–推翻自己和过往,重学自定义View 自定义View系列教程01–常用工具介绍 自定义View系列教程02–onMeasure源码详尽分析 自定义View系列教程03–onL ...

  3. 文件处理工具 gif合成工具 文件后缀批量添加工具 文件夹搜索工具 重复文件查找工具 网页图片解析下载工具等

    以下都是一些简单的免费分享的工具,技术支持群:592132877,提供定制化服务开发. Gif动图合成工具 主要功能是扫描指定的文件夹里的所有zip文件,然后提取Zip文件中的图片,并合成一张gif图 ...

  4. C# ListBox 子项数据更新

    今天在倒腾ListBox控件的数据编辑时,遇到了一个小小的问题,现在就把解决方法记录下来,如果各位道友有更好的方法,一定要留言赐教. 问题还原: 有一个界面,有这么一个ListBox用来显示所有的角色 ...

  5. iOS 网络编程(HTTP协议)

    HTTP协议的概念HTTP协议,Hyper Text Transfer Protocol (超文本传输协议)是用于从万维网服务器传送超文本到本地浏览器的传输协议,HTTP是一个应用层协议,由请求和响应 ...

  6. SPU - SKU - ARPU

    商品和单品: 单品:汉语中的“个”  例如,iphone是一个单品,但是在淘宝上当很多商家同时出售这个产品的时候,iphone就是一个商品了 商品:淘宝叫item,京东叫product,商品特指与商家 ...

  7. [转]MongoDB c++驱动安装与使用

    安装 获取源码:git clone https://github.com/mongodb/mongo-cxx-driver.git,解压 安装编译工具scons:yum install -y scon ...

  8. Jmeter-BeanShell Sampler调用java代码

    1.添加BeanShell Sampler 2.编写BeanShell Sampler代码 3.引用参数 ${id}

  9. tensorflow图像基本处理

    tensorflow库提供的专门的图片处理库,以下只是部分示例,更多函数请参照源码'\tensorflow_api\v1\image__init__.py' 加载图像 方式1: 使用tf.gfile. ...

  10. tableau-交互筛选器

    欢迎观看这段关于“交互筛选器”的视频.   将字段拖到筛选器功能区就可以轻松执行筛选,但是如果我们希望能够直接从视图中修改筛选器 选择项,那该怎么办?如果我们在可看到“子类别”字段的任何位置右键单击, ...