明明我只是简单跑了一个数据清洗28W数据的python脚本,不知道怎么就报错如下:

too many threads running within your python process

The "can't start new thread" error almost certainly due to the fact that you have already have too many threads running within your python process,

and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you're creating; the maximum number you will be able to create will be determined by your environment,

but it should be in the order of hundreds at least. It would probably be a good idea to re-think your architecture here;

seeing as this is running asynchronously anyhow, perhaps you could use a pool of threads to fetch resources from another site instead of always starting up a thread for every request. Another improvement to consider is your use of Thread.join and Thread.stop; this would probably be better accomplished by providing a timeout value to the constructor of HTTPSConnection.

RuntimeError: can't start new thread的更多相关文章

  1. Python 3.5 RuntimeError: can't start new thread

    /*********************************************************************** * Python 3.5 RuntimeError: ...

  2. tcp端口扫描(python多线程)

    1 使用单线程扫描单台主机 首先实现的是对单台主机中0-1024端口的扫描,发现差不多每秒扫描一个端口,很慢. import socket def tcp_scanner(host,port): cl ...

  3. 我的Keras使用总结(3)——利用bottleneck features进行微调预训练模型VGG16

    Keras的预训练模型地址:https://github.com/fchollet/deep-learning-models/releases 一个稍微讲究一点的办法是,利用在大规模数据集上预训练好的 ...

  4. [线程]Thead 中传参数RuntimeError: thread.__init__() not called

    在写一个多线程类的时候调用报错 RuntimeError: thread.__init__() not called class NotifyTread(threading.Thread): def ...

  5. tensorflow_目标识别object_detection_api,RuntimeError: main thread is not in main loop,fig = plt.figure(frameon=False)_tkinter.TclError: no display name and no $DISPLAY environment variable

    最近在使用目标识别api,但是报错了: File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/script_o ...

  6. 解决async 运行多线程时报错RuntimeError: There is no current event loop in thread 'Thread-2'

    原来使用: loop = asyncio.get_event_loop()task = asyncio.ensure_future(do_work(checker))loop.run_until_co ...

  7. Python thread local

    由于GIL的原因,笔者在日常开发中几乎没有用到python的多线程.如果需要并发,一般使用多进程,对于IO Bound这种情况,使用协程也是不错的注意.但是在python很多的网络库中,都支持多线程, ...

  8. [Python]Threading.Thread之Daemon线程

    之前对Daemon线程理解有偏差,特记录说明: 一.什么是Daemon A thread can be flagged as a "daemon thread". The sign ...

  9. python threading.thread

    Thread 是threading模块中最重要的类之一,可以使用它来创建线程.有两种方式来创建线程:一种是通过继承Thread类,重写它的run方法:另一种是创建一个threading.Thread对 ...

随机推荐

  1. 【转】转载一篇优质的讲解epoll模型的文章

    从事服务端开发,少不了要接触网络编程.Epoll 作为 Linux 下高性能网络服务器的必备技术至关重要,Nginx.Redis.Skynet 和大部分游戏服务器都使用到这一多路复用技术. Epoll ...

  2. windows远程重启

    用管理员权限打开win+r,输入cmd后: net use \ip地址 "管理员密码" /user:administrator shutdown /r /t 0 /m ip地址 即 ...

  3. Netty学习篇①

    什么是netty Netty封装了JDK自带的NIO,运用起来更加简单快速,Netty是一个异步事件驱动的网络应用框架,让开发更加简便 Netty相比JDK自带的NIO的优点 Netty的api调用简 ...

  4. c++ 行为型_备忘录模式(Memento)

    行为型_备忘录模式(Memento) 作用场景: 当意图在对象外面保存对象的内部状态,但是又不想破坏对象的封装性,就可以考虑备忘录模式. 解释: 其参与者包括 1.Memnto(备忘录,如下列Coun ...

  5. [Arc102B]All Your Paths are Different Lengths_构造_二进制拆分

    All Your Paths are Different Lengths 题目链接:https://atcoder.jp/contests/arc102/tasks/arc102_b 题解: 构造题有 ...

  6. 性能工具之JMeter+InfluxDB+Grafana打造压测可视化实时监控(centos7环境)

    前提条件,已经安装jmeter并可以运行 1.安装influxdata wget et https://dl.influxdata.com/influxdb/releases/influxdb-1.7 ...

  7. orcale备份语句

    1.创建一个文件夹,比如d盘下创建一个expdp的文件夹 d:\expdp2.使用一个用户,必须具有DBA权限 比如 sqlplus /nolog conn system/password@数据库连接 ...

  8. 【AtCoder】ARC067

    ARC067 C - Factors of Factorial 这个直接套公式就是,先求出来每个质因数的指数幂,然后约数个数就是 \((1 + e_{1})(1 + e_{2})(1 + e_{3}) ...

  9. SHA1签名工具类java

    package com.net.util; import java.security.MessageDigest; import java.util.Iterator; import java.uti ...

  10. Win7原装ISO镜像封装USB3.0&网卡驱动

    Win7原装ISO镜像封装USB3.0&网卡驱动   最新购买的电脑是Windows10系统,想装回Windows7,但是装Windows7发现网络适配器没出现,如果没有USB2.0接口,US ...