笔记-pyrhon-lib-requests

1. 简介

Requests is the only Non-GMO HTTP library for Python, safe for human consumption.

1.1. install/upgrade

pip install requests

pip install –upgrade requests

2. 快速入门

2.1. make a Request

引用

>>> import requests

请求：

>>> r = requests.get('https://api.github.com/events')

post:

>>> r = requests.post('https://httpbin.org/post', data = {'key':'value'})

the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:

>>> r = requests.put('https://httpbin.org/put', data = {'key':'value'})

>>> r = requests.delete('https://httpbin.org/delete')

>>> r = requests.head('https://httpbin.org/get')

>>> r = requests.options('https://httpbin.org/get')

2.2. passing parameters in urls

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.get('https://httpbin.org/get', params=payload)

You can see that the URL has been correctly encoded by printing the URL:

>>> print(r.url)

https://httpbin.org/get?key2=value2&key1=value1

2.3. response content

>>> r = requests.get('https://api.github.com/events')

>>> r.text

u'[{"repository":{"open_issues":0,"url":"https://github.com/...

Requests will automatically decode content from the server. Most unicode charsets are seamlessly decoded.

>>> r.encoding

'utf-8'

>>> r.encoding = 'ISO-8859-1'

改编码格式后调用r.text时该格式会应用，但一般没必要改编码格式。

2.4. binary reponse content

>>> r.content

b'[{"repository":{"open_issues":0,"url":"https://github.com/...

The gzip and deflate transfer-encodings are automatically decoded for you.

2.5. raw response content

有时需要获取原始套接字内容：

>>> r = requests.get('https://api.github.com/events', stream=True)

>>> r.raw

<urllib3.response.HTTPResponse object at 0x101194810>

>>> r.raw.read(10)

'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'

注意stream设置。

2.6. custom headers

定制请求头部

>>> url = 'https://api.github.com/some/endpoint'

>>> headers = {'user-agent': 'my-app/0.0.1'}

>>> r = requests.get(url, headers=headers)

2.7. more complicated POST requests

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.post("https://httpbin.org/post", data=payload)

>>> print(r.text)

{

...

"form": {

"key2": "value2",

"key1": "value1"

...

}

2.8. response status codes

查看返回状态码

>>> r = requests.get('https://httpbin.org/get')

>>> r.status_code

200

异常信息查看

当返回4xx或5xx状态码时，查看异常信息：

>>> bad_r = requests.get('https://httpbin.org/status/404')

>>> bad_r.status_code

404

>>> bad_r.raise_for_status()

Traceback (most recent call last):

File "requests/models.py", line 832, in raise_for_status

raise http_error

requests.exceptions.HTTPError: 404 Client Error

如果返回状态码是200，该值为空。

>>> r.raise_for_status()

None

2.9. response headers

返回包的头部：

>>> r.headers

{

'content-encoding': 'gzip',

'transfer-encoding': 'chunked',

'connection': 'close',

'server': 'nginx/1.0.4',

'x-runtime': '148ms',

'etag': '"e1ca502697e5c9317743dc078f67693f"',

'content-type': 'application/json'

}

查看指定字段：

>>> r.headers['Content-Type']

'application/json'

>>> r.headers.get('content-type')

'application/json'

2.10. cookies

查看cookies：

>>> url = 'http://example.com/some/cookie/setting/url'

>>> r = requests.get(url)

>>> r.cookies['example_cookie_name']

'example_cookie_value'

在请求中指定cookies:

>>> url = 'https://httpbin.org/cookies'

>>> cookies = dict(cookies_are='working')

>>> r = requests.get(url, cookies=cookies)

>>> r.text

'{"cookies": {"cookies_are": "working"}}'

Cookies are returned in a RequestsCookieJar, which acts like a dict but also offers a more complete interface, suitable for use over multiple domains or paths. Cookie jars can also be passed in to requests:

>>> jar = requests.cookies.RequestsCookieJar()

>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')

>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')

>>> url = 'https://httpbin.org/cookies'

>>> r = requests.get(url, cookies=jar)

>>> r.text

'{"cookies": {"tasty_cookie": "yum"}}'

2.11. timeouts

指定请求超时。

>>> requests.get('https://github.com/', timeout=0.001)

Traceback (most recent call last):

File "<stdin>", line 1, in <module>

requests.exceptions.Timeout: HTTPConnectionPool(host='github.com', port=80): Request timed out. (timeout=0.001)

2.12. errors and exceptions

网络报错：In the event of a network problem (e.g. DNS failure, refused connection, etc), Requests will raise a ConnectionError exception.

http异常：Response.raise_for_status() will raise an HTTPError if the HTTP request returned an unsuccessful status code.

超时：If a request times out, a Timeout exception is raised.

重定向：If a request exceeds the configured number of maximum redirections, aTooManyRedirects exception is raised.

基类：All exceptions that Requests explicitly raises inherit fromrequests.exceptions.RequestException.

3. 高级用法

这一部分内容主要讲解定制Requests。

3.1. session objects

session对象允许复用参数，典型的是cookies，它使用urllib3的connecton pooling。

如此可以重用tcp底层。

A Session object has all the methods of the main Requests API.

s = requests.Session()

s.get('https://httpbin.org/cookies/set/sessioncookie/123456789')

r = s.get('https://httpbin.org/cookies')

print(r.text)

# '{"cookies": {"sessioncookie": "123456789"}}'

3.2. Request and Reponse object

简单来说，get()做两件事：

构造一个Request对象发给server；
将返回的数据生成一个Response对象，该对象包含有Request信息。

>>> r = requests.get('https://en.wikipedia.org/wiki/Monty_Python')

获取头部信息：r.headers

获取请求包头部：r.request.headers

3.3. prepared Requests

from requests import Request, Session

s = Session()

req = Request('POST', url, data=data, headers=headers)

prepped = req.prepare()

# do something with prepped.body

prepped.body = 'No, I want exactly this as the body.'

# do something with prepped.headers

del prepped.headers['Content-Type']

resp = s.send(prepped,

stream=stream,

verify=verify,

proxies=proxies,

cert=cert,

timeout=timeout

)

print(resp.status_code)