明明我只是简单跑了一个数据清洗28W数据的python脚本,不知道怎么就报错如下:

too many threads running within your python process

The "can't start new thread" error almost certainly due to the fact that you have already have too many threads running within your python process,

and due to a resource limit of some kind the request to create a new thread is refused. You should probably look at the number of threads you're creating; the maximum number you will be able to create will be determined by your environment,

but it should be in the order of hundreds at least. It would probably be a good idea to re-think your architecture here;

seeing as this is running asynchronously anyhow, perhaps you could use a pool of threads to fetch resources from another site instead of always starting up a thread for every request. Another improvement to consider is your use of Thread.join and Thread.stop; this would probably be better accomplished by providing a timeout value to the constructor of HTTPSConnection.

RuntimeError: can't start new thread的更多相关文章

  1. Python 3.5 RuntimeError: can't start new thread

    /*********************************************************************** * Python 3.5 RuntimeError: ...

  2. tcp端口扫描(python多线程)

    1 使用单线程扫描单台主机 首先实现的是对单台主机中0-1024端口的扫描,发现差不多每秒扫描一个端口,很慢. import socket def tcp_scanner(host,port): cl ...

  3. 我的Keras使用总结(3)——利用bottleneck features进行微调预训练模型VGG16

    Keras的预训练模型地址:https://github.com/fchollet/deep-learning-models/releases 一个稍微讲究一点的办法是,利用在大规模数据集上预训练好的 ...

  4. [线程]Thead 中传参数RuntimeError: thread.__init__() not called

    在写一个多线程类的时候调用报错 RuntimeError: thread.__init__() not called class NotifyTread(threading.Thread): def ...

  5. tensorflow_目标识别object_detection_api,RuntimeError: main thread is not in main loop,fig = plt.figure(frameon=False)_tkinter.TclError: no display name and no $DISPLAY environment variable

    最近在使用目标识别api,但是报错了: File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/ops/script_o ...

  6. 解决async 运行多线程时报错RuntimeError: There is no current event loop in thread 'Thread-2'

    原来使用: loop = asyncio.get_event_loop()task = asyncio.ensure_future(do_work(checker))loop.run_until_co ...

  7. Python thread local

    由于GIL的原因,笔者在日常开发中几乎没有用到python的多线程.如果需要并发,一般使用多进程,对于IO Bound这种情况,使用协程也是不错的注意.但是在python很多的网络库中,都支持多线程, ...

  8. [Python]Threading.Thread之Daemon线程

    之前对Daemon线程理解有偏差,特记录说明: 一.什么是Daemon A thread can be flagged as a "daemon thread". The sign ...

  9. python threading.thread

    Thread 是threading模块中最重要的类之一,可以使用它来创建线程.有两种方式来创建线程:一种是通过继承Thread类,重写它的run方法:另一种是创建一个threading.Thread对 ...

随机推荐

  1. JavaSE基础(六)--Java流程控制语句之条件语句

    Java 条件语句 - if...else 一个 if 语句包含一个布尔表达式和一条或多条语句. 语法 if 语句的语法如下: if(布尔表达式) { //如果布尔表达式为true将执行的语句 } 如 ...

  2. jenkins持续集成(svn+maven)

    一.初始化环境 1.系统管理->插件管理 安装插件 Maven Integration 2.系统管理->全局工具配置 配置jdk.maven 二.项目配置1.源码配置 (使用之前搭建的sv ...

  3. LeetCode 第 167 场周赛

    1290.二进制链表转整数 1291.顺次数 1292.元素和小于等于阈值的正方形的最大边长 1293.网格中的最短路径 1290.二进制链表转整数 1290.二进制链表转整数 给你一个单链表的引用结 ...

  4. EasyUI datagrid 动态加载表头和数据

    首先返回到页面的需要是JSON数据: 第一步: 遍历表头,插入到array中 for (var i = 0; i < jsonObj.title.length; i++) { //把返回的数据封 ...

  5. cmake 简单build和删除

    mkdir build cd build cmake .. make

  6. 题目12 打印1到最大的n位数

    ///////////////////////////////////////////////////////////////////////////////////// // 2.打印1到最大的n位 ...

  7. spark异常篇-Removing executor 5 with no recent heartbeats: 120504 ms exceeds timeout 120000 ms 可能的解决方案

    问题描述与分析 题目中的问题大致可以描述为: 由于某个 Executor 没有按时向 Driver 发送心跳,而被 Driver 判断该 Executor 已挂掉,此时 Driver 要把 该 Exe ...

  8. C#签名验签帮助类

    using System; using System.IO; using System.Text; using System.Collections.Generic; using System.Sec ...

  9. RedHat7 配置yum源

    今天需要搭建一个测试环境,没办法只能找了个Linux服务器,但是之前的其他同事弄过是其他系统的,不是centos的,所以只能自己搞. 合计直接百度,怎么安装docker,结果一直报错,下载失败之类的 ...

  10. SpringBoot事务隔离等级和传播行为

    一.开启事物管理 //import org.springframework.transaction.annotation.EnableTransactionManagement; @SpringBoo ...