笔记-pyrhon-lib-requests

1.      简介

Requests is the only Non-GMO HTTP library for Python, safe for human consumption.

1.1.    install/upgrade

pip install requests

pip install –upgrade requests

2.      快速入门

2.1.    make a Request

引用

>>> import requests

请求:

>>> r = requests.get('https://api.github.com/events')

post:

>>> r = requests.post('https://httpbin.org/post', data = {'key':'value'})

the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:

>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})

>>> r = requests.delete('https://httpbin.org/delete')

>>> r = requests.head('https://httpbin.org/get')

>>> r = requests.options('https://httpbin.org/get')

2.2.    passing parameters in urls

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.get('https://httpbin.org/get', params=payload)

You can see that the URL has been correctly encoded by printing the URL:

>>> print(r.url)

https://httpbin.org/get?key2=value2&key1=value1

2.3.    response content

>>> r = requests.get('https://api.github.com/events')

>>> r.text

u'[{"repository":{"open_issues":0,"url":"https://github.com/...

Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded.

>>> r.encoding

'utf-8'

>>> r.encoding = 'ISO-8859-1'

改编码格式后调用r.text时该格式会应用,但一般没必要改编码格式。

2.4.    binary reponse content

>>> r.content

b'[{"repository":{"open_issues":0,"url":"https://github.com/...

The gzip and deflate transfer-encodings are automatically decoded for you.

2.5.    raw response content

有时需要获取原始套接字内容:

>>> r = requests.get('https://api.github.com/events', stream=True)

>>> r.raw

<urllib3.response.HTTPResponse object at 0x101194810>

>>> r.raw.read(10)

'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'

注意stream设置。

2.6.    custom headers

定制请求头部

>>> url = 'https://api.github.com/some/endpoint'

>>> headers = {'user-agent': 'my-app/0.0.1'}

>>> r = requests.get(url, headers=headers)

2.7.    more complicated POST requests

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.post("https://httpbin.org/post", data=payload)

>>> print(r.text)

{

...

"form": {

"key2": "value2",

"key1": "value1"

},

...

}

2.8.    response status codes

查看返回状态码

>>> r = requests.get('https://httpbin.org/get')

>>> r.status_code

200

异常信息查看

当返回4xx或5xx状态码时,查看异常信息:

>>> bad_r = requests.get('https://httpbin.org/status/404')

>>> bad_r.status_code

404

>>> bad_r.raise_for_status()

Traceback (most recent call last):

File "requests/models.py", line 832, in raise_for_status

raise http_error

requests.exceptions.HTTPError: 404 Client Error

如果返回状态码是200,该值为空。

>>> r.raise_for_status()

None

2.9.    response headers

返回包的头部:

>>> r.headers

{

'content-encoding': 'gzip',

'transfer-encoding': 'chunked',

'connection': 'close',

'server': 'nginx/1.0.4',

'x-runtime': '148ms',

'etag': '"e1ca502697e5c9317743dc078f67693f"',

'content-type': 'application/json'

}

查看指定字段:

>>> r.headers['Content-Type']

'application/json'

>>> r.headers.get('content-type')

'application/json'

2.10.        cookies

查看cookies:

>>> url = 'http://example.com/some/cookie/setting/url'

>>> r = requests.get(url)

>>> r.cookies['example_cookie_name']

'example_cookie_value'

在请求中指定cookies:

>>> url = 'https://httpbin.org/cookies'

>>> cookies = dict(cookies_are='working')

>>> r = requests.get(url, cookies=cookies)

>>> r.text

'{"cookies": {"cookies_are": "working"}}'

Cookies are returned in a RequestsCookieJar, which acts like a dict but also offers a more complete interface, suitable for use over multiple domains or paths. Cookie jars can also be passed in to requests:

>>> jar = requests.cookies.RequestsCookieJar()

>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')

>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')

>>> url = 'https://httpbin.org/cookies'

>>> r = requests.get(url, cookies=jar)

>>> r.text

'{"cookies": {"tasty_cookie": "yum"}}'

2.11.        timeouts

指定请求超时。

>>> requests.get('https://github.com/', timeout=0.001)

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

2.12.        errors and exceptions

网络报错:In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.

http异常:Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code.

超时:If a request times out, a Timeout exception is raised.

重定向:If a request exceeds the configured number of maximum redirections, aTooManyRedirects exception is raised.

基类:All exceptions that Requests explicitly raises inherit fromrequests.exceptions.RequestException.

3.      高级用法

这一部分内容主要讲解定制Requests。

3.1.    session objects

session对象允许复用参数,典型的是cookies,它使用urllib3的connecton pooling。

如此可以重用tcp底层。

A Session object has all the methods of the main Requests API.

s = requests.Session()

s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')

r = s.get('https://httpbin.org/cookies')

print(r.text)

# '{"cookies": {"sessioncookie": "123456789"}}'

3.2.    Request and Reponse object

简单来说,get()做两件事:

  1. 构造一个Request对象发给server;
  2. 将返回的数据生成一个Response对象,该对象包含有Request信息。

>>> r = requests.get('https://en.wikipedia.org/wiki/Monty_Python')

获取头部信息:r.headers

获取请求包头部:r.request.headers

3.3.    prepared Requests

from requests import Request, Session

s = Session()

req = Request('POST', url, data=data, headers=headers)

prepped = req.prepare()

# do something with prepped.body

prepped.body = 'No, I want exactly this as the body.'

# do something with prepped.headers

del prepped.headers['Content-Type']

resp = s.send(prepped,

stream=stream,

verify=verify,

proxies=proxies,

cert=cert,

timeout=timeout

)

print(resp.status_code)

笔记-pyrhon-lib-requests的更多相关文章

  1. python学习笔记(excel+requests)

    已经可以对excel简单的操作后 可以开始通过excel写测试用例 读取用例 执行用例 提前写好execl 如图: 下面是代码: #!/usr/bin/env python # -*- coding: ...

  2. PYTHON 爬虫笔记八:利用Requests+正则表达式爬取猫眼电影top100(实战项目一)

    利用Requests+正则表达式爬取猫眼电影top100 目标站点分析 流程框架 爬虫实战 使用requests库获取top100首页: import requests def get_one_pag ...

  3. python学习笔记(pict+requests+xml)

    博主尝试了下更换python版本 之前很多脚本改正运行错误后.还是不能正常运行 忙会了半天还是没有成功 只好还原版本 所以下面的代码还没实际运行成功.先记录下 #!/usr/bin/env pytho ...

  4. 爬虫1.1-基础知识+requests库

    目录 爬虫-基础知识+requests库 1. 状态返回码 2. URL各个字段解释 2. requests库 3. requests库爬虫的基本流程 爬虫-基础知识+requests库 关于html ...

  5. 笔记-http-header

    笔记-http-header 1.      Requests部分 Host:请求的web服务器域名地址 User-Agent:HTTP客户端运行的浏览器类型的详细信息.通过该头部信息,web服务器可 ...

  6. python爬虫笔记Day01

    python爬虫笔记第一天 Requests库的安装 先在cmd中pip install requests 再打开Python IDM写入import requests 完成requests在.py文 ...

  7. 聊聊 virtualenv 和 virtualenvwrapper 实践

    各位 Python 的小伙伴肯定多多少少接触过 virtualenv.本文将介绍 virtualenv 以及如何更科学更优雅地使用 virtualenv. virtualenv 首先来聊一下 virt ...

  8. 零基础Python爬虫实现(爬取最新电影排行)

    提示:本学习来自Ehco前辈的文章, 经过实现得出的笔记. 目标网站 http://dianying.2345.com/top/ 网站结构 要爬的部分,在ul标签下(包括li标签), 大致来说迭代li ...

  9. Request库学习

    0x00前言 这库让我爱上了python  碉堡! 开心去学了一些python,然后就来学这个时候神库~~ 资料来源:http://cn.python-requests.org/en/latest/u ...

  10. Requests:Python HTTP Module学习笔记(一)(转)

    Requests:Python HTTP Module学习笔记(一) 在学习用python写爬虫的时候用到了Requests这个Http网络库,这个库简单好用并且功能强大,完全可以代替python的标 ...

随机推荐

  1. 解决Pandoc wasn't found.pdflatex not found on PATH

    解决nbconvert failed: Pandoc wasn't found.解决nbconvert failed: pdflatex not found on PATH 问题1描述 500 : I ...

  2. fd_set实现原理

    fd_set是一个结构 /* The fd_set member is required to be an array of longs. */ typedef long int __fd_mask; ...

  3. u-boot分析(五)----I/D cache失效|关闭MMU和cache|关闭看门狗

    u-boot分析(五) 上篇博文我们按照210的启动流程,对u-boot启动中的设置异常向量表,设置SVC模式进行了分析,今天我们继续按照u-boot的启动流程对以下内容进行分析. 今天我们会用到的文 ...

  4. 《深入理解Java 7核心技术与最佳实践》读书笔记(2) Java语言动态性引言

    Java语言是一种静态类型的编程语言.静态类型的含义是指在编译时进行类型检查.Java源代码中的每个变量的类型都要显式地进行声明.所有变量.方法的参数和方法返回值的类型在程序运行之前就必须是已知的.J ...

  5. ubuntu & sublime字体设置

    ubuntu # yahei http://pan.baidu.com/share/link?shareid=972621198&uk=1243888096&fid=333591974 ...

  6. BIN文件对象数据库,直接存储对象做数据库,小型项目用它准没错

    using System; using System.Collections.Generic; using System.Linq; using System.Text; using System.I ...

  7. PS:将一个图片变成圆形

    1.选择一张正方形图片并放置到PS软件中 2.选择“选框工具”->右击选择"椭圆选框工具” 3.画出你要圆形内的范围: 4.选择两种方式中的一种,实际上效果是一毛一样的 5.右下角就会 ...

  8. gcc编译流程

    gcc的编译流程分为四个步骤,分别为: 预处理(Pre-Processing) 编译(Compiling) 汇编(Assembling) 链接(Linking) 以hello.c为例子,在这四个步骤中 ...

  9. UE4的蓝图都能做什么

    创建关卡脚本 蓝图具有和虚幻3中Kismet一样的功能,每个关卡都由自己的蓝图,他可以: 引用和操控actors 控制使用Matinee的过场 管理关卡流,存档点以及其他关卡相关的系统 和关卡中的类蓝 ...

  10. iOS开发:小技巧积累

    1.获取全局的Delegate对象,这样我们可以调用这个对象里的方法和变量: [(MyAppDelegate*)[[UIApplication sharedApplication] delegate] ...