笔记-pyrhon-lib-requests
笔记-pyrhon-lib-requests
1. 简介
Requests is the only Non-GMO HTTP library for Python, safe for human consumption.
1.1. install/upgrade
pip install requests
pip install –upgrade requests
2. 快速入门
2.1. make a Request
引用
>>> import requests
请求:
>>> r = requests.get('https://api.github.com/events')
post:
>>> r = requests.post('https://httpbin.org/post', data = {'key':'value'})
the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:
>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})
>>> r = requests.delete('https://httpbin.org/delete')
>>> r = requests.head('https://httpbin.org/get')
>>> r = requests.options('https://httpbin.org/get')
2.2. passing parameters in urls
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.get('https://httpbin.org/get', params=payload)
You can see that the URL has been correctly encoded by printing the URL:
>>> print(r.url)
https://httpbin.org/get?key2=value2&key1=value1
2.3. response content
>>> r = requests.get('https://api.github.com/events')
>>> r.text
u'[{"repository":{"open_issues":0,"url":"https://github.com/...
Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded.
>>> r.encoding
'utf-8'
>>> r.encoding = 'ISO-8859-1'
改编码格式后调用r.text时该格式会应用,但一般没必要改编码格式。
2.4. binary reponse content
>>> r.content
b'[{"repository":{"open_issues":0,"url":"https://github.com/...
The gzip and deflate transfer-encodings are automatically decoded for you.
2.5. raw response content
有时需要获取原始套接字内容:
>>> r = requests.get('https://api.github.com/events', stream=True)
>>> r.raw
<urllib3.response.HTTPResponse object at 0x101194810>
>>> r.raw.read(10)
'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'
注意stream设置。
2.6. custom headers
定制请求头部
>>> url = 'https://api.github.com/some/endpoint'
>>> headers = {'user-agent': 'my-app/0.0.1'}
>>> r = requests.get(url, headers=headers)
2.7. more complicated POST requests
>>> payload = {'key1': 'value1', 'key2': 'value2'}
>>> r = requests.post("https://httpbin.org/post", data=payload)
>>> print(r.text)
{
...
"form": {
"key2": "value2",
"key1": "value1"
},
...
}
2.8. response status codes
查看返回状态码
>>> r = requests.get('https://httpbin.org/get')
>>> r.status_code
200
异常信息查看
当返回4xx或5xx状态码时,查看异常信息:
>>> bad_r = requests.get('https://httpbin.org/status/404')
>>> bad_r.status_code
404
>>> bad_r.raise_for_status()
Traceback (most recent call last):
File "requests/models.py", line 832, in raise_for_status
raise http_error
requests.exceptions.HTTPError: 404 Client Error
如果返回状态码是200,该值为空。
>>> r.raise_for_status()
None
2.9. response headers
返回包的头部:
>>> r.headers
{
'content-encoding': 'gzip',
'transfer-encoding': 'chunked',
'connection': 'close',
'server': 'nginx/1.0.4',
'x-runtime': '148ms',
'etag': '"e1ca502697e5c9317743dc078f67693f"',
'content-type': 'application/json'
}
查看指定字段:
>>> r.headers['Content-Type']
'application/json'
>>> r.headers.get('content-type')
'application/json'
2.10. cookies
查看cookies:
>>> url = 'http://example.com/some/cookie/setting/url'
>>> r = requests.get(url)
>>> r.cookies['example_cookie_name']
'example_cookie_value'
在请求中指定cookies:
>>> url = 'https://httpbin.org/cookies'
>>> cookies = dict(cookies_are='working')
>>> r = requests.get(url, cookies=cookies)
>>> r.text
'{"cookies": {"cookies_are": "working"}}'
Cookies are returned in a RequestsCookieJar, which acts like a dict but also offers a more complete interface, suitable for use over multiple domains or paths. Cookie jars can also be passed in to requests:
>>> jar = requests.cookies.RequestsCookieJar()
>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')
>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')
>>> url = 'https://httpbin.org/cookies'
>>> r = requests.get(url, cookies=jar)
>>> r.text
'{"cookies": {"tasty_cookie": "yum"}}'
2.11. timeouts
指定请求超时。
>>> requests.get('https://github.com/', timeout=0.001)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)
2.12. errors and exceptions
网络报错:In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.
http异常:Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code.
超时:If a request times out, a Timeout exception is raised.
重定向:If a request exceeds the configured number of maximum redirections, aTooManyRedirects exception is raised.
基类:All exceptions that Requests explicitly raises inherit fromrequests.exceptions.RequestException.
3. 高级用法
这一部分内容主要讲解定制Requests。
3.1. session objects
session对象允许复用参数,典型的是cookies,它使用urllib3的connecton pooling。
如此可以重用tcp底层。
A Session object has all the methods of the main Requests API.
s = requests.Session()
s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')
r = s.get('https://httpbin.org/cookies')
print(r.text)
# '{"cookies": {"sessioncookie": "123456789"}}'
3.2. Request and Reponse object
简单来说,get()做两件事:
- 构造一个Request对象发给server;
- 将返回的数据生成一个Response对象,该对象包含有Request信息。
>>> r = requests.get('https://en.wikipedia.org/wiki/Monty_Python')
获取头部信息:r.headers
获取请求包头部:r.request.headers
3.3. prepared Requests
from requests import Request, Session
s = Session()
req = Request('POST', url, data=data, headers=headers)
prepped = req.prepare()
# do something with prepped.body
prepped.body = 'No, I want exactly this as the body.'
# do something with prepped.headers
del prepped.headers['Content-Type']
resp = s.send(prepped,
stream=stream,
verify=verify,
proxies=proxies,
cert=cert,
timeout=timeout
)
print(resp.status_code)
笔记-pyrhon-lib-requests的更多相关文章
- python学习笔记(excel+requests)
已经可以对excel简单的操作后 可以开始通过excel写测试用例 读取用例 执行用例 提前写好execl 如图: 下面是代码: #!/usr/bin/env python # -*- coding: ...
- PYTHON 爬虫笔记八:利用Requests+正则表达式爬取猫眼电影top100(实战项目一)
利用Requests+正则表达式爬取猫眼电影top100 目标站点分析 流程框架 爬虫实战 使用requests库获取top100首页: import requests def get_one_pag ...
- python学习笔记(pict+requests+xml)
博主尝试了下更换python版本 之前很多脚本改正运行错误后.还是不能正常运行 忙会了半天还是没有成功 只好还原版本 所以下面的代码还没实际运行成功.先记录下 #!/usr/bin/env pytho ...
- 爬虫1.1-基础知识+requests库
目录 爬虫-基础知识+requests库 1. 状态返回码 2. URL各个字段解释 2. requests库 3. requests库爬虫的基本流程 爬虫-基础知识+requests库 关于html ...
- 笔记-http-header
笔记-http-header 1. Requests部分 Host:请求的web服务器域名地址 User-Agent:HTTP客户端运行的浏览器类型的详细信息.通过该头部信息,web服务器可 ...
- python爬虫笔记Day01
python爬虫笔记第一天 Requests库的安装 先在cmd中pip install requests 再打开Python IDM写入import requests 完成requests在.py文 ...
- 聊聊 virtualenv 和 virtualenvwrapper 实践
各位 Python 的小伙伴肯定多多少少接触过 virtualenv.本文将介绍 virtualenv 以及如何更科学更优雅地使用 virtualenv. virtualenv 首先来聊一下 virt ...
- 零基础Python爬虫实现(爬取最新电影排行)
提示:本学习来自Ehco前辈的文章, 经过实现得出的笔记. 目标网站 http://dianying.2345.com/top/ 网站结构 要爬的部分,在ul标签下(包括li标签), 大致来说迭代li ...
- Request库学习
0x00前言 这库让我爱上了python 碉堡! 开心去学了一些python,然后就来学这个时候神库~~ 资料来源:http://cn.python-requests.org/en/latest/u ...
- Requests:Python HTTP Module学习笔记(一)(转)
Requests:Python HTTP Module学习笔记(一) 在学习用python写爬虫的时候用到了Requests这个Http网络库,这个库简单好用并且功能强大,完全可以代替python的标 ...
随机推荐
- 解决Pandoc wasn't found.pdflatex not found on PATH
解决nbconvert failed: Pandoc wasn't found.解决nbconvert failed: pdflatex not found on PATH 问题1描述 500 : I ...
- fd_set实现原理
fd_set是一个结构 /* The fd_set member is required to be an array of longs. */ typedef long int __fd_mask; ...
- u-boot分析(五)----I/D cache失效|关闭MMU和cache|关闭看门狗
u-boot分析(五) 上篇博文我们按照210的启动流程,对u-boot启动中的设置异常向量表,设置SVC模式进行了分析,今天我们继续按照u-boot的启动流程对以下内容进行分析. 今天我们会用到的文 ...
- 《深入理解Java 7核心技术与最佳实践》读书笔记(2) Java语言动态性引言
Java语言是一种静态类型的编程语言.静态类型的含义是指在编译时进行类型检查.Java源代码中的每个变量的类型都要显式地进行声明.所有变量.方法的参数和方法返回值的类型在程序运行之前就必须是已知的.J ...
- ubuntu & sublime字体设置
ubuntu # yahei http://pan.baidu.com/share/link?shareid=972621198&uk=1243888096&fid=333591974 ...
- BIN文件对象数据库,直接存储对象做数据库,小型项目用它准没错
using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.I ...
- PS:将一个图片变成圆形
1.选择一张正方形图片并放置到PS软件中 2.选择“选框工具”->右击选择"椭圆选框工具” 3.画出你要圆形内的范围: 4.选择两种方式中的一种,实际上效果是一毛一样的 5.右下角就会 ...
- gcc编译流程
gcc的编译流程分为四个步骤,分别为: 预处理(Pre-Processing) 编译(Compiling) 汇编(Assembling) 链接(Linking) 以hello.c为例子,在这四个步骤中 ...
- UE4的蓝图都能做什么
创建关卡脚本 蓝图具有和虚幻3中Kismet一样的功能,每个关卡都由自己的蓝图,他可以: 引用和操控actors 控制使用Matinee的过场 管理关卡流,存档点以及其他关卡相关的系统 和关卡中的类蓝 ...
- iOS开发:小技巧积累
1.获取全局的Delegate对象,这样我们可以调用这个对象里的方法和变量: [(MyAppDelegate*)[[UIApplication sharedApplication] delegate] ...