HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法
爬虫多次访问同一个网站一段时间后会出现错误 HTTPConnectionPool(host:XX)Max retries exceeded with url '<requests.packages.urllib3.connection.HTTPConnection object at XXXX>: Failed to establish a new connection: [Errno 99] Cannot assign requested address'
是因为在每次数据传输前客户端要和服务器建立TCP连接,为节省传输消耗,默认为keep-alive,即连接一次,传输多次,然而在多次访问后不能结束并回到连接池中,导致不能产生新的连接
headers中的Connection默认为keep-alive,
将header中的Connection一项置为close
headers = { 'Connection': 'close', } r = requests.get(url, headers=headers)
此时问题解决
HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法的更多相关文章
- HTTPConnectionPool(host='xx.xx.xx.xx', port=xx): Max retries exceeded with url:(Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000015A25025EB8>...))
HTTPConnectionPool(host='xx.xx.xx.xx', port=xx): Max retries exceeded with url:(Caused by ConnectTim ...
- scrapy::Max retries exceeded with url
运行scrapy时出错这个错误:Max retries exceeded with url解决方法: img1=requests.get(url=aa,headers=header1,timeout= ...
- HTTPConnectionPool(host:XX)Max retries exceeded with url
爬虫多次访问同一个网站一段时间后会出现错误 HTTPConnectionPool(host:XX)Max retries exceeded with url '<requests.package ...
- 关于python3.6上传文件时报错:HTTPSConnectionPool(host='***.org', port=443): Max retries exceeded with url: /post (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAIL解决办法
第一个报错: 最近在练习post请求中上传文件时遇到了一个奇葩事情,两台电脑上写了一模一样的代码,一个运行正常,另一个一片红. 最后了解了一下原因以及解决办法.先记录下关键代码: files = {& ...
- windows环境pip安装时一直报错Could not fetch URL https://pypi.org/simple/xrld/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url:
最近项目不忙了~~有开始专研的python大业,上来想用pip安装一个第三方的库,就一直报错: Could not fetch URL https://pypi.org/simple/xrld/: T ...
- HTTPSConnectionPool(host='xxxxx', port=443): Max retries exceeded with url:xxxxxxxx (Caused by NewConnectionError('<urllib3.connect,Max retries exceeded with ,(Caused by NewConnectionError
HTTPSConnectionPool(host='f6ws-sha8re-o88k.s3.ama66zaws.com', port=443): Max retries exceeded with u ...
- Max retries exceeded with url
78 Traceback (most recent call last): File "thread072413.py", line 163, in <module> ...
- 解决Max retries exceeded with url的问题
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='itunes.apple.com', port=443): Max ret ...
- requests.exceptions.SSLError……Max retries exceeded with url错误求助!!!
import requests head = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Appl ...
随机推荐
- 费马大定理以及求解a^2+b^2=c^2的奇偶数列法则
<一>费马大定理:a^n+b^n=c^n 当n大于2时无正整数解. <二>求解a^2+b^=c^2可以使用a值奇偶法则:1.当a=2*n时,b=n^2-1,c=n^2+1 ...
- 《DSP using MATLAB》Problem 5.31
第3小题: 代码: %% ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ %% Out ...
- <------------------字节流--------------------->
字节流: 输入和输出:1.参照物都是java程序来惨遭 2.Input输入,持久化上的数据---->内存 3.Output输出,内存--->硬盘 字节输出流: OutputStream: ...
- laya的UI编辑器
//加载一个图集 Laya.loader.load("res/atlas/comp.json",Handler.create(this,this.onLoaderComp), Ha ...
- keil5配置ST Link v2 for STM32F10x
请务必先安装MDKCM525.EXE,否则会找不到闪存编程算法
- day9大纲
01 作业内容回顾 函数的初识: 封装一个功能. def 函数名(): 函数体 函数的返回值:return 1,结束函数. 2,返回给执行者(函数名())值. return ----> None ...
- jmap获取内存排名靠前的对象
docker部署,jvvm通过jmx访问失败,只能永远是的jdk自带工具查看情况: jmap -histo pid | sort -n -r -k 2 | head -10
- php调用c#的dll(转)
这几天,一直在做DES ecb模式的加解密,刚用.net实现了加解密,完了由于需要又要转型成PHP代码,费了九牛二虎之力单独用PHP没能实现,结构看到一篇php直接调用c#里生成的.dll文件的方法, ...
- Linux crontab配置
crontab配置 1.命令功能 通过crontab 命令,我们可以在固定的间隔时间执行指定的系统指令或 shell script脚本.时间间隔的单位可以是分钟.小时.日.月.周及以上的任意组合.这个 ...
- 多线程中的Lock小结
出处:http://www.cnblogs.com/DarrenChan/p/6528578.html#undefined 1.lock和synchronized的区别 1)Lock不是Java语言内 ...