python基础===requests学习笔记

这里有一个新的学习requests网站：http://docs.python-requests.org/zh_CN/latest/user/quickstart.html
2017/11/30

Requests is the only Non-GMO HTTP library for Python, safe for human consumption.

Note

The use of Python 3 is highly preferred over Python 2. Consider upgrading your applications and infrastructure if you find yourself still using Python 2 in production today. If you are using Python 3, congratulations — you are indeed a person of excellent taste. ———

>>> import requests

>>> r = requests.get('http://www.baidu.com')

Now, we have a response object called r. We can get all the information we need from this object.

For example, this is how you make an HTTP POST request:

>>> r = requests.post('http://xxxxx.com/post', data = {'key':'value'})

What about the other HTTP request types: PUT, DELETE, HEAD and OPTIONS? These are all just as simple:

>>> r = requests.put('http://xxxxxx.org/put', data = {'key':'value'})

>>> r = requests.delete('http://xxxxxxxx.org/delete')

>>> r = requests.head('http://xxxxxxx.org/get')

>>> r = requests.options('http://xxxxxxxxxx.org/get')

Passing Parameters In URLs

We often want to send some sort of data in the URL's query string.

As an example, if you wanted to pass key1=value1 and key2=value2 to httpbin.org/get, you would use the following code:

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.get('http://xxxxx.org/get', params=payload)

You can see that the URL has been correctly encoded；

>>> print(r.url)

http://httpbin.org/get?key2=value2&key1=value1

You can also pass a list of items as a value:

>>> payload = {'key1': 'value1', 'key2': ['value2', 'value3']}

>>> r = requests.get('http://xxxx.org/get', params=payload)

>>> print(r.url)

http://xxxx.org/get?key1=value1&key2=value2&key2=value3

Response Content

>>> import requests

>>> r = requests.get("http://www.baidu.com")

>>> r.text

'<!DOCTYPE html>\r\n<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge>

When you make a request, Requests makes educated guesses about the encoding of the response based on the HTTP headers. The text encoding guessed by Requests is used when you access r.text. You can find out what encoding Requests is using, and change it, using the r.encoding property:

>>> r.encoding

'ISO-8859-1'

>>> r.encoding = 'utf-8'

>>> r.text

'<!DOCTYPE html>\r\n<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=alway

If you change the encoding, Requests will use the new value of r.encoding whenever you call r.text.

If you have created your own encoding and registered it with the codecs module, you can simply use the codec name as the value of r.encoding and Requests will handle the decoding for you.

Binary Response Content

>>> r.content

b'<!DOCTYPE html>\r\n<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always

You can also access the response body as bytes, for non-text requests.

The gzip and deflate transfer-encodings are automatically decoded for you.

For example, to create an image from binary data returned by a request, you can use the following code:

>>> from PIL import Image

>>> from io import BytesIO

>>> i = Image.open(BytesIO(r.content))

JSON Response Content

>>> r = requests.get('https://api.github.com/events')

>>> r.json()

[{'id': '6736180556', 'type': 'PushEvent', 'actor': {'id': 14111893, 'login': 'rvitorgomes', 'display_login': 'rvitorgomes', 'gravatar_id': '', 'url': 'https://api.github.com/users/rvitorgomes', 'avatar_url': 'https://avatars.g

In case the JSON decoding fails, r.json() raises an exception. For example, if the response gets a 204 (No Content), or if the response contains invalid JSON, attempting r.json() raises ValueError: No JSON object could be decoded.

It should be noted that the success of the call to r.json() does not indicate the success of the response. Some servers may return a JSON object in a failed response (e.g. error details with HTTP 500). Such JSON will be decoded and returned. To check that a request is successful, use r.raise_for_status() or check r.status_code is what you expect.

Raw Response Content

In the rare case that you'd like to get the raw socket response from the server, you can access r.raw. If you want to do this, make sure you set stream=True in your initial request. Once you do, you can do this:

>>> r = requests.get('https://api.github.com/events', stream=True)

>>> r.raw

<requests.packages.urllib3.response.HTTPResponse object at 0x101194810>

>>> r.raw.read(10)

'\x1f\x8b\x08\x00\x00\x00\x00\x00\x00\x03'

Custom Headers

>>> url = 'https://api.github.com/some/endpoint'

>>> headers = {'user-agent': 'my-app/0.0.1'}

>>> r = requests.get(url, headers=headers)

Note: Custom headers are given less precedence than more specific sources of information. For instance:

Authorization headers set with headers= will be overridden if credentials are specified in .netrc, which in turn will be overridden by the auth= parameter.
Authorization headers will be removed if you get redirected off-host.
Proxy-Authorization headers will be overridden by proxy credentials provided in the URL.
Content-Length headers will be overridden when we can determine the length of the content.

Furthermore, Requests does not change its behavior at all based on which custom headers are specified. The headers are simply passed on into the final request.

Note: All header values must be a string, bytestring, or unicode. While permitted, it's advised to avoid passing unicode header values.

More complicated POST requests

Typically, you want to send some form-encoded data — much like an HTML form. To do this, simply pass a dictionary to the data argument. Your dictionary of data will automatically be form-encoded when the request is made:

>>> payload = {'key1': 'value1', 'key2': 'value2'}

>>> r = requests.post("http://httpbin.org/post", data=payload)

>>> print(r.text)

{

  ...

  "form": {

    "key2": "value2",

    "key1": "value1"

  },

  ...

}

You can also pass a list of tuples to the data argument. This is particularly useful when the form has multiple elements that use the same key:

>>> payload = (('key1', 'value1'), ('key1', 'value2'))

>>> r = requests.post('http://httpbin.org/post', data=payload)

>>> print(r.text)

{

  ...

  "form": {

    "key1": [

      "value1",

      "value2"

    ]

  },

  ...

}

There are times that you may want to send data that is not form-encoded. If you pass in a string instead of a dict, that data will be posted directly.

For example, the GitHub API v3 accepts JSON-Encoded POST/PATCH data:

>>> import json

>>>import requests

>>> url = 'https://api.github.com/some/endpoint'

>>> payload = {'some': 'data'}

>>> r = requests.post(url, data=json.dumps(payload))

Instead of encoding the dict yourself, you can also pass it directly using the json parameter (added in version 2.4.2) and it will be encoded automatically:

>>> url = 'https://api.github.com/some/endpoint'

>>> payload = {'some': 'data'}

>>> r = requests.post(url, json=payload)
>>> print(r.text)
{
  "args": {}, 
  "data": "{\"key1\": \"value1\", \"key2\": \"value2\"}", 
  "files": {}, 
  "form": {},

POST a Multipart-Encoded File(多重编码文件)

Requests makes it simple to upload Multipart-encoded files:

>>> url = 'http://httpbin.org/post'

>>> files = {'file': open('report.xls', 'rb')}

>>> r = requests.post(url, files=files)

>>> r.text

{

  ...

  "files": {

    "file": "<censored...binary...data>"

  },

  ...

}

You can set the filename, content_type and headers explicitly:

>>> url = 'http://httpbin.org/post'

>>> files = {'file': ('report.xls', open('report.xls', 'rb'), 'application/vnd.ms-excel', {'Expires': '0'})}

>>> r = requests.post(url, files=files)

>>> r.text

{

  ...

  "files": {

    "file": "<censored...binary...data>"

  },

  ...

}

Response Status Codes

*详见之前博客，http响应码

>>> r = requests.get('http://www.baidu.com')

>>> r.status_code

200

Requests also comes with a built-in status code lookup object for easy reference:

>>> r.status_code == requests.codes.ok

True

If we made a bad request (a 4XX client error or 5XX server error response), we can raise it with Response.raise_for_status():

>>> bad_r = requests.get('http://httpbin.org/status/404')

>>> bad_r.status_code

404

>>> bad_r.raise_for_status()

Traceback (most recent call last):

  File "requests/models.py", line 832, in raise_for_status

    raise http_error

requests.exceptions.HTTPError: 404 Client Error

But, since our status_code for r was 200, when we call raise_for_status() we get:

>>> r.raise_for_status()

None

Response Headers

We can view the server's response headers using a Python dictionary:

>>> r.headers

{'Connection': 'keep-alive', 'Server': 'meinheld/0.6.1', 'Date': 'Thu, 19 Oct 2017 02:34:53 GMT', 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*', 'Access-Control-Allow-Credentials': 'true', 'X-Powered-By': 'Flask', 'X-Processed-Time': '0.000764846801758', 'Content-Length': '488', 'Via': '1.1 vegur'}

or,we can access the headers using any capitalization we want:

>>> r.headers["Date"]

'Thu, 19 Oct 2017 02:34:53 GMT'
>>> r.headers["Connection"]
'keep-alive'

If a response contains some Cookies, you can quickly access them:

>>> url = 'http://example.com/some/cookie/setting/url'

>>> r = requests.get(url)

>>> r.cookies['example_cookie_name']

'example_cookie_value'

我这里执行失败的，先记录下来吧

To send your own cookies to the server, you can use the cookies parameter:

>>> url = 'http://httpbin.org/cookies'

>>> cookies = dict(cookies_are = 'working')

>>> r = requests.get(url,cookies = cookies)

>>> r.text

'{\n  "cookies": {\n    "cookies_are": "working"\n  }\n}\n'

Cookies are returned in a RequestsCookieJar, which acts like a dict but also offers a more complete interface, suitable for use over multiple domains or paths. Cookie jars can also be passed in to requests:

>>> jar = requests.cookies.RequestsCookieJar()

>>> jar.set('tasty_cookie', 'yum', domain='httpbin.org', path='/cookies')

Cookie(version=0, name='tasty_cookie', value='yum', port=None, port_specified=False, domain='httpbin.org', domain_specified=True, domain_initial_dot=False, path='/cookies', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False)

>>> jar.set('gross_cookie', 'blech', domain='httpbin.org', path='/elsewhere')

Cookie(version=0, name='gross_cookie', value='blech', port=None, port_specified=False, domain='httpbin.org', domain_specified=True, domain_initial_dot=False, path='/elsewhere', path_specified=True, secure=False, expires=None, discard=True, comment=None, comment_url=None, rest={'HttpOnly': None}, rfc2109=False)

>>> url = 'http://httpbin.org/cookies'

>>> r = requests.get(url, cookies=jar)

>>> r.text

'{\n  "cookies": {\n    "tasty_cookie": "yum"\n  }\n}\n'

Timeouts

>>> requests.get("http://www.baidu.com",timeout = 0.001)

Traceback (most recent call last):

  File "C:\Users\yangbo\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3\connection.py", line 141, in _new_conn

    (self.host, self.port), self.timeout, **extra_kw)

  File "C:\Users\yangbo\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3\util\connection.py", line 83, in create_connection

    raise err

  File "C:\Users\yangbo\AppData\Local\Programs\Python\Python36-32\lib\site-packages\urllib3\util\connection.py", line 73, in create_connection

    sock.connect(sa)

socket.timeout: timed out

python基础===requests学习笔记的更多相关文章

Python基础教程学习笔记：第一章基础知识
Python基础教程第二版学习笔记 1.python的每一个语句的后面可以添加分号也可以不添加分号:在一行有多条语句的时候,必须使用分号加以区分 2.查看Python版本号,在Dos窗口中输入“p ...
Python基础班学习笔记
本博客采用思维导图式笔记,所有思维导图均为本人亲手所画.因为本人也是初次学习Python语言所以有些知识点可能不太全. 基础班第一天学习笔记:链接基础班第二天学习笔记:链接基础班第三天学习笔记:链 ...
【学习笔记】Python基础教程学习笔记
教程视频网盘共享:http://pan.baidu.com/s/1hrTrR5E 03-python基础.if判断 print 输出数据 print("hahahah")----- ...
Python 基础语法学习笔记
以下运行结果均通过Python3.5版本实测! 1.列表转换为字典 a = ['a', 'b'] b = [1, 2] c = ['c','d'] print (dict([a,b,c])) 输出结果 ...
Python基础教程学习笔记：第二章列表和元组
1.序列中元素的索引: 第一个元素索引是0,第二个是1,依次递增最后一个元素索引是-1,倒数第二个是-2,依次递减 2.序列(Sequence)的种类: 列表(list).元组(tuple).字符串 ...
Requests:Python HTTP Module学习笔记（一）（转）
Requests:Python HTTP Module学习笔记(一) 在学习用python写爬虫的时候用到了Requests这个Http网络库,这个库简单好用并且功能强大,完全可以代替python的标 ...
Python基础知识总结笔记（四）函数
Python基础知识总结笔记(四)函数python中的函数函数中的参数变量作用域偏函数PFA递归函数高阶函数BIFs中的高阶函数匿名函数lambda闭包Closure装饰器Decorator函数式编程 ...
python网络爬虫学习笔记
python网络爬虫学习笔记 By 钟桓 9月 4 2014 更新日期:9月 4 2014 文章文件夹 1. 介绍: 2. 从简单语句中開始: 3. 传送数据给server 4. HTTP头-描写叙述 ...
linux基础命令学习笔记（二）
linux基础命令学习笔记(二) 1.kill :终止进程 kill pid (唯一标示一个进程) kill -9 强制终止 kill -15 命令未结束不能终止 # ps aux 查看所有进程 ...

随机推荐

【bzoj3732】Network 最小生成树+倍增LCA
题目描述给你N个点的无向图 (1 <= N <= 15,000),记为:1…N. 图中有M条边 (1 <= M <= 30,000) ,第j条边的长度为: d_j ( 1 & ...
【bzoj1858】[Scoi2010]序列操作线段树区间合并
题目描述 lxhgww最近收到了一个01序列,序列里面包含了n个数,这些数要么是0,要么是1,现在对于这个序列有五种变换操作和询问操作: 0 a b 把[a, b]区间内的所有数全变成0 1 a b ...
容器化RDS｜计算存储分离 or 本地存储？
随着交流机会的增多(集中在金融行业,规模都在各自领域数一数二),发现大家对 Docker + Kubernetes 的接受程度超乎想象, 并极有兴趣将这套架构应用到 RDS 领域.数据库服务的需求可以 ...
BZOJ4103 [Thu Summer Camp 2015]异或运算【可持久化trie树】
题目链接 BZOJ4103 题解一眼看过去是二维结构,实则未然需要树套树之类的数据结构区域异或和,就一定是可持久化\(trie\)树观察数据,\(m\)非常大,而\(n\)和\(p\)比较小,甚 ...
HDOJ.1113 Word Amalgamation（map）
Word Amalgamation 点我挑战题目点我一起学习STL-MAP 题意分析给出字典.之后给出一系列======乱序======单词,要求你查字典,如过这个乱序单词对用有多个有序单词可以输 ...
浅谈Hibernate框架(一)——.hbm.xml中的配置
Hibernate一枚“全自动”的ORM框架: 用IDE工具集成Hibernate会自动生成: 以.hbm.xml为后缀结尾的配置文件+ POJO类 + Dao类主键查询: Session.load ...
关于notepad++如何自动补全标签的问题
转自:https://blog.csdn.net/Panda_Eyes1/article/details/81486331 关于notepad++如何自动补全标签的问题 2018年08月07日 18: ...
Eclipse中 properties 文件中中文乱码
在.properties文件写注释时,发现中文乱码了,由于之前在idea中有见设置.properties文件的编码类型,便找了找乱码原因在中文操作系统中,Eclipse中的Java类型文件的编码的默 ...
BZOJ1880: [Sdoi2009]Elaxia的路线（最短路）
1880: [Sdoi2009]Elaxia的路线 Time Limit: 4 Sec Memory Limit: 64 MBSubmit: 2049 Solved: 805 题目链接:https ...
Rectangle 暴力枚举大法
frog has a piece of paper divided into n rows and m columns. Today, she would like to draw a rectang ...

python基础===requests学习笔记

Passing Parameters In URLs

Response Content

Binary Response Content

JSON Response Content

Raw Response Content

Custom Headers

More complicated POST requests

POST a Multipart-Encoded File(多重编码文件)

Response Status Codes

Response Headers

Cookies

Timeouts

python基础===requests学习笔记的更多相关文章

随机推荐

热门专题