HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法
爬虫多次访问同一个网站一段时间后会出现错误 HTTPConnectionPool(host:XX)Max retries exceeded with url '<requests.packages.urllib3.connection.HTTPConnection object at XXXX>: Failed to establish a new connection: [Errno 99] Cannot assign requested address'
是因为在每次数据传输前客户端要和服务器建立TCP连接,为节省传输消耗,默认为keep-alive,即连接一次,传输多次,然而在多次访问后不能结束并回到连接池中,导致不能产生新的连接
headers中的Connection默认为keep-alive,
将header中的Connection一项置为close
headers = {
'Connection': 'close',
}
r = requests.get(url, headers=headers)
此时问题解决
HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法的更多相关文章
- HTTPConnectionPool(host='xx.xx.xx.xx', port=xx): Max retries exceeded with url:(Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x0000015A25025EB8>...))
HTTPConnectionPool(host='xx.xx.xx.xx', port=xx): Max retries exceeded with url:(Caused by ConnectTim ...
- scrapy::Max retries exceeded with url
运行scrapy时出错这个错误:Max retries exceeded with url解决方法: img1=requests.get(url=aa,headers=header1,timeout= ...
- HTTPConnectionPool(host:XX)Max retries exceeded with url
爬虫多次访问同一个网站一段时间后会出现错误 HTTPConnectionPool(host:XX)Max retries exceeded with url '<requests.package ...
- 关于python3.6上传文件时报错:HTTPSConnectionPool(host='***.org', port=443): Max retries exceeded with url: /post (Caused by SSLError(SSLError(1, '[SSL: CERTIFICATE_VERIFY_FAIL解决办法
第一个报错: 最近在练习post请求中上传文件时遇到了一个奇葩事情,两台电脑上写了一模一样的代码,一个运行正常,另一个一片红. 最后了解了一下原因以及解决办法.先记录下关键代码: files = {& ...
- windows环境pip安装时一直报错Could not fetch URL https://pypi.org/simple/xrld/: There was a problem confirming the ssl certificate: HTTPSConnectionPool(host='pypi.org', port=443): Max retries exceeded with url:
最近项目不忙了~~有开始专研的python大业,上来想用pip安装一个第三方的库,就一直报错: Could not fetch URL https://pypi.org/simple/xrld/: T ...
- HTTPSConnectionPool(host='xxxxx', port=443): Max retries exceeded with url:xxxxxxxx (Caused by NewConnectionError('<urllib3.connect,Max retries exceeded with ,(Caused by NewConnectionError
HTTPSConnectionPool(host='f6ws-sha8re-o88k.s3.ama66zaws.com', port=443): Max retries exceeded with u ...
- Max retries exceeded with url
78 Traceback (most recent call last): File "thread072413.py", line 163, in <module> ...
- 解决Max retries exceeded with url的问题
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='itunes.apple.com', port=443): Max ret ...
- requests.exceptions.SSLError……Max retries exceeded with url错误求助!!!
import requests head = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) Appl ...
随机推荐
- (11)线程池(最新的concurrent.futures包去开启)
'''concurrent.futures是最新的开启线程池的包'''import timefrom concurrent.futures import ThreadPoolExecutor #开启线 ...
- Spring源码学习(总)
前文: ------------------------------------------------------------------------------------------------ ...
- 【NOI2005】 聪聪可可
树分治劲啊 原题: 聪聪和可可是兄弟俩,他们俩经常为了一些琐事打起来,例如家中只剩下最后一根冰棍而两人都想吃.两个人都想玩儿电脑(可是他们家只有一台电脑)……遇到这种问题,一般情况下石头剪刀布就好了, ...
- linux系统调用的三种方法
通过glibc提供的库函数 [23:02:14] gcc chmodtest.c [23:02:17] ls -l kali //记得先创建这个文件 -rwxrwxrwx. 1 root root 0 ...
- 用flask和长轮询实现对帅哥投票和实时查看票数
flask中的代码 from flask import Flask,request,render_template,redirect,session,jsonify import uuid impor ...
- Singer 学习四 可视化数据操作工具
knots 是一款基于electron 开发的可视化UI界面,我们可以此工具进行方便的数据处理,注意工具使用了 docker 运行,需要安装docker 下载地址 https://github.c ...
- YAML Class ID Reference
Classes Ordered by ID Number ID Class 1 GameObject 2 Component 3 LevelGameManager 4 Transform 5 Time ...
- Hi3518EV200音频相关
1.sample程序可以录音,音频格式为G711A.G711U.G726.ADPCM: 2.ADPCM找不到音频播放器 3.G711格式海思添加了4字节头0x00 0x01 0x0a 0x00,普通播 ...
- webpack报错:Module build failed (from ./node_modules/babel-loader/lib/index.js): Error: Cannot find module 'babel-preset-env' from '...' - Did you mean "@babel/env"?
webpack报错:Module build failed (from ./node_modules/babel-loader/lib/index.js): Error: Cannot find mo ...
- egg 知识点
egg 的约定 约定 使用方法 路由对应controller中的方法 举例:router.get('/', controller.home.index);,此时对应app/controller/hom ...