https://docs.celeryproject.org/en/stable/userguide/configuration.html?highlight=control_exchange#new-lowercase-settings

New lowercase settings

Version 4.0 introduced new lower case settings and setting organization.

The major difference between previous versions, apart from the lower case names, are the renaming of some prefixes, like celery_beat_ to beat_celeryd_ to worker_, and most of the top level celery_ settings have been moved into a new task_ prefix.

Note

Celery will still be able to read old configuration files, so there’s no rush in moving to the new settings format. Furthermore, we provide the celery upgrade command that should handle plenty of cases (including Django).

https://github.com/celery/celery/blob/master/docs/userguide/workers.rst#id198

Time Limits

.. versionadded:: 2.0
pool support: prefork/gevent

Soft, or hard?

The time limit is set in two values, soft and hard. The soft time limit allows the task to catch an exception to clean up before it is killed: the hard timeout isn't catch-able and force terminates the task.

A single task can potentially run forever, if you have lots of tasks waiting for some event that'll never happen you'll block the worker from processing new tasks indefinitely. The best way to defend against this scenario happening is enabling time limits.

The time limit (--time-limit) is the maximum number of seconds a task may run before the process executing it is terminated and replaced by a new process. You can also enable a soft time limit (--soft-time-limit), this raises an exception the task can catch to clean up before the hard time limit kills it:

from myapp import app
from celery.exceptions import SoftTimeLimitExceeded @app.task
def mytask():
try:
do_work()
except SoftTimeLimitExceeded:
clean_up_in_a_hurry()

Time limits can also be set using the :setting:`task_time_limit` / :setting:`task_soft_time_limit` settings.

Changing time limits at run-time

.. versionadded:: 2.3
broker support: amqp, redis

There's a remote control command that enables you to change both soft and hard time limits for a task — named time_limit.

Example changing the time limit for the tasks.crawl_the_web task to have a soft time limit of one minute, and a hard time limit of two minutes:

>>> app.control.time_limit('tasks.crawl_the_web',
soft=60, hard=120, reply=True)
[{'worker1.example.com': {'ok': 'time limits set successfully'}}]

Only tasks that starts executing after the time limit change will be affected.

Time Limits

.. versionadded:: 2.0
pool support: prefork/gevent

Soft, or hard?

The time limit is set in two values, soft and hard. The soft time limit allows the task to catch an exception to clean up before it is killed: the hard timeout isn't catch-able and force terminates the task.

A single task can potentially run forever, if you have lots of tasks waiting for some event that'll never happen you'll block the worker from processing new tasks indefinitely. The best way to defend against this scenario happening is enabling time limits.

The time limit (--time-limit) is the maximum number of seconds a task may run before the process executing it is terminated and replaced by a new process. You can also enable a soft time limit (--soft-time-limit), this raises an exception the task can catch to clean up before the hard time limit kills it:

from myapp import app
from celery.exceptions import SoftTimeLimitExceeded @app.task
def mytask():
try:
do_work()
except SoftTimeLimitExceeded:
clean_up_in_a_hurry()

Time limits can also be set using the :setting:`task_time_limit` / :setting:`task_soft_time_limit` settings.

Note

Time limits don't currently work on platforms that don't support the :sig:`SIGUSR1` signal.

:setting:`task_soft_time_limit` celery 异步任务 执行时间限制 内存限制的更多相关文章

  1. Celery异步任务重复执行(Redis as broker)

    之前讲到利用celery异步处理一些耗时或者耗资源的任务,但是近来分析数据的时候发现一个奇怪的现象,即是某些数据重复了,自然想到是异步任务重复执行了. 查阅之后发现,到如果一个任务太耗时,任务完成时间 ...

  2. Django使用Celery异步任务队列

    1  Celery简介 Celery是异步任务队列,可以独立于主进程运行,在主进程退出后,也不影响队列中的任务执行. 任务执行异常退出,重新启动后,会继续执行队列中的其他任务,同时可以缓存停止期间接收 ...

  3. Django --- celery异步任务与RabbitMQ模块

    一 RabbitMQ 和 celery 1 celery Celery 是一个 基于python开发的分布式异步消息任务队列,通过它可以轻松的实现任务的异步处理, 如果你的业务场景中需要用到异步任务, ...

  4. celery异步任务、定时任务

    阅读目录 一 什么是Celery? 二 Celery的使用场景 三 Celery的安装配置 四 Celery异步任务 五Celery定时任务 六在Django中使用Celery   一 什么是Cele ...

  5. celery异步任务框架

    目录 Celery 一.官方 二.Celery异步任务框架 Celery架构图 消息中间件 任务执行单元 任务结果存储 三.使用场景 四.Celery的安装配置 五.两种celery任务结构:提倡用包 ...

  6. [WP8.1UI控件编程]Windows Phone大数据量网络图片列表的异步加载和内存优化

    11.2.4 大数据量网络图片列表的异步加载和内存优化 虚拟化技术可以让Windows Phone上的大数据量列表不必担心会一次性加载所有的数据,保证了UI的流程性.对于虚拟化的技术,我们不仅仅只是依 ...

  7. Celery 异步任务 , 定时任务 , 周期任务 的芹菜

    1.什么是Celery?Celery 是芹菜Celery 是基于Python实现的模块, 用于执行异步定时周期任务的其结构的组成是由    1.用户任务 app    2.管道 broker 用于存储 ...

  8. Django商城项目笔记No.6用户部分-注册接口-短信验证码实现celery异步

    Django商城项目笔记No.4用户部分-注册接口-短信验证码实现celery异步 接上一篇,如何解决前后端请求跨域问题? 首先想一下,为什么图片验证码请求的也是后端的api.meiduo.site: ...

  9. python—Celery异步分布式

    python—Celery异步分布式 Celery  是一个python开发的异步分布式任务调度模块,是一个消息传输的中间件,可以理解为一个邮箱,每当应用程序调用celery的异步任务时,会向brok ...

随机推荐

  1. 分布式事务MSDTC使用时,需要的配置

    服务器最终配置 DTC服务 组件 防火墙 这里,跟下面的解决方案有点差异,在添加2个规则之后,原本就有分布式相关的规则,也给开启了. 网上的解决办法 在服务里打开 Distributed Transa ...

  2. Excel 快速跳到表格最后一行/第一行

    快速跳到表格的最后一行 首先鼠标选中一个带有数据的单元格,点击shift键,把鼠标放到该单元格底部的边缘地带,出现带四个方向的箭头为止,再连续点击鼠标左键两次,直接跳到表格的最后一行 快速跳到表格的最 ...

  3. linux不同环境变量文件的比较,如/etc/profile和/etc/environment

    /etc/profile 为系统的每个用户设置环境信息和启动程序,当用户第一次登录时,该文件被执行,其配置对所有登录的用户都有效. 当被修改时,必须重启才会生效.英文描述:"System w ...

  4. CyclicBarrier回环屏障深度解析

    1. 前沿 从上一节的CountDownLatch的学习,我们发现其只能使用一次,当state递减为0后,就没有用了,需要重新新建一个计数器.那么我们有没有可以复用的计数器呢?当然,JUC包给我们提供 ...

  5. ribbon源码分析

    对于ribbon的使用我们只需要在RestTemplate的申明上面加上 @LoadBalanced 注解之后那么这个RestTemplate就具有了负载均衡的功能 ribbon是怎么实现这一功能的? ...

  6. 由innodb锁引起的数据库相关

    innodb 锁的问题 1.事务 原子性:要么成功,要么失败 一致性:前后数据保持一致状态 隔离性:多个事务并行,相互不影响 持久性:事务提交之后,对数据的影响是永久性的,即使故障也可以保持. 2.并 ...

  7. 使用 CoreDNS 来应对 DNS 污染

    原文链接:https://fuckcloudnative.io/posts/install-coredns-on-macos/ CoreDNS 是 Golang 编写的一个插件式 DNS 服务器,是 ...

  8. VS使用过程中可能会遇到的问题

    Q:某个类无法引用命名空间 A:可能是类名与文件夹名重复了

  9. 柔性分布式事务关于异步解决方案MQ版

    上述思想本质是 二阶段提交变体 1,2是prepare阶段 4是commit阶段 存在问题 MQ提供半消息支持 生产者提供消息回查功能 发送方多次半消息到MQSERVER  消费方会多次消费消息 生产 ...

  10. 洛谷P4848 崂山白花蛇草水 权值线段树+KDtree

    题目描述 神犇 \(Aleph\) 在 \(SDOI\ Round2\) 前立了一个 \(flag\):如果进了省队,就现场直播喝崂山白花蛇草水.凭借着神犇 \(Aleph\) 的实力,他轻松地进了山 ...