Background on http frontends

-civetweb frontend

--thread per connection, requires a lot of threads

---qos or priority queuing would block frontend threads

-beast frontend

--boost::beast for http parsing

--boost::asio for async networking/io

--async for accepting connections and reading headers

---good model for qos - can queue requests without blocking threads

--synchronous call to process_request()

---thread per request, still need lots of threads

goal: scale requests independently of threads

-why boost::asio

--doesn't impose a threading model. io_service object is a reactor, call run() from any thread

--mature library, basis for C++ std::net library in Networking TS [1]

--the Extensible Asynchronous Model [2] provides several options for async primitives (callbacks, futures, coroutines)

--boost::asio::spawn() stackful coroutines: "enables programs to implement asynchronous logic in a synchronous manner" [3 http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/spawn.html]

proposed librados interfaces for asio [4 https://github.com/ceph/ceph/pull/19054]

--header-only wrapper over librados c++ api

--conform to the Extensible Asynchronous Model, so support the same primitives - see unit tests for examples

--deeper Objecter integration work in progress by Adam Emerson [5]

--gives radosgw a unified interface for async operations over http and rados

async process_request() [6]

-add optional yield_context* argument to process_request()

-beast frontend passes one, civetweb passes nullptr

-any librados calls use new interface when given a yield_context

-requires the yield_context* to be passed everywhere in between

--but we can stash it in req_state to make it available to all ops

-getting started with the easy stuff

--rgw_get_system_obj()

--reading user objects for authentication

--reading bucket/bucket instance objects (common to most s3/swift ops)

-this process leaves a lot of gaps. for example, rgw_get_system_obj() is in tons of call paths without access to a yield_context

--(either outside process_request(), or just aren't hooked up yet)

--just passing 'nullptr' makes it impossible to differentiate the yield_context argument from its 4 other arguments that default to nullptr!

--that makes it impossible to reason about which call paths could run asynchronously

-measurable progress towards full asynchrony

--new vocabulary type 'optional_yield_context' with 'null_yield' for empty value

--null_yield designates a call site that is definitely synchronous

--makes it easy to audit the code and find the pieces that still need conversion

-fighting regression once we're close

--have librados calls log warnings when called synchronously from a beast frontend thread (using a thread_local flag)

--scan those logs in teuthology runs to flag failures

and then?

-vastly reduce the number of frontend threads for beast

-consolidate other background threads

remaining work:

- RGWGetObj waits on AioCompletions - use AioThrottle from PutObj instead

- replace librados IoCtx::operate() calls with rgw_rados_operate() and optional_yield_context

- thread optional_yield_context all the way from beast frontend to rgw_rados_operate() calls
- some cls client calls use IoCtx::operate() directly

- block_while_resharding() sleeps on a condition variable

- no async interface for pool object listings with IoCtx.nobjects_begin()

- libcurl http requests for auth (Keystone and OPA)

[1] "C++ Technical Specification - Extensions for Networking"

http://cplusplus.github.io/networking-ts/draft.pdf

[2] "Library Foundations for Asynchronous Operations"

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n3896.pdf

[3] Reference: boost::asio::spawn

http://www.boost.org/doc/libs/1_65_1/doc/html/boost_asio/reference/spawn.html

[4] "librados: add async interfaces for use with boost::asio"

https://github.com/ceph/ceph/pull/19054

[5] "osdc/Objecter: Boost.Asio (I object!)"

https://github.com/ceph/ceph/pull/16715

[6] work in progress branch:

https://github.com/cbodley/ceph/commits/wip-rgw-async-process-171120

Ceph 之 Background on http frontends的更多相关文章

  1. 虚拟机使用ceph-deploy安装ceph

    参考: ceph官网介绍 使用ceph-deploy安装Ceph 12.x 安装虚拟机 首先安装虚拟机环境,虚拟机安装这里不做介绍,本实验使用的镜像为CentOS-7-x86_64-Everythin ...

  2. Ceph常用命令

    目录 [1.环境准备] [2.部署管理] [3.集群扩容] [4.用户管理] [5.密钥环管理] [6.块设备管理] [7.快照管理] [8.参考链接] 简要说明: 最近心血来潮,对分布式存储感兴趣, ...

  3. ceph mimic版本 部署安装

    ceph 寻址过程 1. file --- object映射, 把file分割成N个相同的对象 2. object - PG 映射, 利用静态hash得到objectID的伪随机值,在 "位 ...

  4. Ceph常规操作及常见问题梳理

    Ceph集群管理 每次用命令启动.重启.停止Ceph守护进程(或整个集群)时,必须指定至少一个选项和一个命令,还可能要指定守护进程类型或具体例程. **命令格式如 {commandline} [opt ...

  5. ceph kubernetes中使用

    1.在管理节点上,进入刚创建的放置配置文件的目录,用 ceph-deploy 执行如下步骤 mkdir /opt/cluster-ceph cd /opt/cluster-ceph ceph-depl ...

  6. 1、ceph-deploy之部署ceph集群

    环境说明 server:3台虚拟机,挂载卷/dev/vdb 10G 系统:centos7.2 ceph版本:luminous repo: 公网-http://download.ceph.com,htt ...

  7. CEPH RGW多 ZONE的配置

    相关的名称解释 Region :可以理解为区域,是基于地理位置的逻辑划分:如:华南,华北之类,包含多个region的Ceph集群必须指定一个master region,一个region可以包含一个或者 ...

  8. Ceph 管理和使用

    ceph 管理 上次介绍了Ceph集群架构并且搭建了ceph集群,本节介绍ceph用户认证流程和挂载.cephFS.ceph RBD以及ceph mds高可用 1. ceph 授权流程和用户权限管理 ...

  9. 安装ceph (快速) 步骤二:存储集群

    用 ceph-deploy 从管理节点建立一个 Ceph 存储集群,该集群包含三个节点,以此探索 Ceph 的功能. 创建一个 Ceph 存储集群,它有一个 Monitor 和两个 OSD 守护进程. ...

随机推荐

  1. [转]js模块化——AMD及require.js

    由CommonJS组织提出了许多新的JavaScript架构方案和标准,希望能为前端开发提供统一的指引.AMD规范就是其中比较著名一个,全称是Asynchronous Module Definitio ...

  2. numpy使用中的疑惑

    numpy中有一些常用的用来产生随机数的函数,randn就是其中一个,randn函数位于numpy.random中,函数原型如下: numpy.random.randn(d0, d1, ..., dn ...

  3. 嘴巴题2 UVA10779 收集者的难题

    UVA10779 收集者的难题 题目: 有\(T(T\leq 20)\)组数据.\(Bob\)在与他的\(n−1(2\leq n\leq 10)个\)朋友交换糖纸,一共有\(m(5\leq m\leq ...

  4. Luogu P1712 [NOI2016]区间(线段树)

    P1712 [NOI2016]区间 题意 题目描述 在数轴上有 \(N\) 个闭区间 \([l_1,r_1],[l_2,r_2],...,[l_n,r_n]\) .现在要从中选出 \(M\) 个区间, ...

  5. Redis源码解析:18Hiredis同步API和回复解析API代码解析

    Redis的sentinel模式使用了Hiredis代码,Hiredis是redis数据库一个轻量级的C语言客户端库.它实现的向Redis发送命令的API函数redisCommand,使用方法类似于p ...

  6. [Array] 566. Reshape the Matrix

    In MATLAB, there is a very useful function called 'reshape', which can reshape a matrix into a new o ...

  7. Codeforces 467D

    题目链接 D. Fedor and Essay time limit per test 2 seconds memory limit per test 256 megabytes input stan ...

  8. ACdream 1108(莫队)

    题目链接 The kth number Time Limit: 12000/6000MS (Java/Others)Memory Limit: 128000/64000KB (Java/Others) ...

  9. spring cloud深入学习(三)-----服务消费

    在上一篇博文中简单实现了eureka-server以及eureka-provider,后面会实现eureka-cosumer,现在针对eureka做进一步的详解. 微服务整体架构 文字再美也没有图片直 ...

  10. mysql优化-数据库设计基本原则

    mysql优化-数据库设计基本原则 一.数据库设计三范式 第一范式:字段具有原子性 原子性是指数据库的所有字段都不可被再次划分,如下表就不满足原子性,起点与终点 字段就可被拆分为起点与终点两个字段. ...