Requests是用python语言基于urllib编写的,采用的是Apache2 Licensed开源协议的HTTP库

各种请求方式:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
print(requests.get(url))
print(requests.post(url))
print(requests.put(url))
print(requests.head(url))
print(requests.options(url))
print(requests.delete(url))
#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
response = requests.get(url)
print(type(requests))
print(response.status_code)
print(response.text)
print(response.cookies)

基本用法:

get:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
response = requests.get(url)
print(response.text)

get(参数):

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://httpbin.org/get?name=cc&age=24"
response = requests.get(url)
print(response.text)

URL查询字符串传递数据,Requests模块允许使用params关键字传递参数,以一个字典来传递这些参数。

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
data = {
"name": "cc",
"age": ""
}
response = requests.get(url, params=data)
print(response.text)

json:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
import json
url = "http://httpbin.org/get"
response = requests.get(url)
print(type(response.text))
print(response.json())
print(type(response.json()))
print(json.loads(response.text))

获取二进制数据:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://github.com/favicon.ico"
response = requests.get(url)
print(type(response.text), type(response.content))
print(response.text)
print(response.content)
with open("favicon.ico", "wb") as f:
f.write(response.content)
f.close()

添加headers:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
headers = {
"user-agent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Mobile Safari/537.36"
}
response = requests.get(url, headers=headers)
print(response.text)

POST请求:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://httpbin.org/post"
data = {
"name": "cc",
"age": 24
}
response = requests.post(url, data)
print(response.text)
print(help(requests.post)

状态判断:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.jianshu.com"
headers = {
"user-agent":"Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Mobile Safari/537.36"
}
response = requests.get(url, headers=headers)
# if response.status_code == 200:
# 判断状态
if response.status_code == requests.codes.ok:
# 判断状态
print(response.text)
else:
print("Request Successfully")
#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests response = requests.get("http://www.baidu.com")
print(type(response.status_code), response.status_code)
print(type(response.headers), response.headers)
print(type(response.cookies), response.cookies)
print(type(response.url), response.url)
print(type(response.history), response.history)

requests高级操作:

cookie:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
response = requests.get(url)
print(response.cookies)
for key, value in response.cookies.items():
print(key + "=" + value)

会话维持:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://httpbin.org/cookies"
req = requests.Session()
req.get("http://httpbin.org/cookies/set/number/1234567890")
response = req.get(url)
print(response.text)

证书验证:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "https://www.12306.cn"
response = requests.get(url)
print(response.status_code) Traceback (most recent call last):
File "E:\python3.6\lib\site-packages\urllib3\connectionpool.py", line 601, in urlopen
chunked=chunked)
File "E:\python3.6\lib\site-packages\urllib3\connectionpool.py", line 346, in _make_request
self._validate_conn(conn)
File "E:\python3.6\lib\site-packages\urllib3\connectionpool.py", line 850, in _validate_conn
conn.connect()
File "E:\python3.6\lib\site-packages\urllib3\connection.py", line 346, in connect
_match_hostname(cert, self.assert_hostname or hostname)
 requests.exceptions.SSLError: HTTPSConnectionPool(host='www.12306.cn', port=443): Max retries exceeded with url: / (Caused by SSLError(CertificateError("hostname 'www.12306.cn' doesn't match either of 'webssl.chinanetcenter.com', 'i.l.inmobicdn.net', '*.fn-mart.com', 'www.1zhe.com', 'dl.jphbpk.gxpan.cn', 'dl.givingtales.gxpan.cn', 'dl.toyblast.gxpan.cn', 'dl.sds.gxpan.cn', 'download.ctrip.com', 'mh.tiancity.com', 'cdn.hxjyios.iwan4399.com', 'ios.hxjy.iwan4399.com', 'gjzx.

避免这种情况的发生可以通过verify=False来处理:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "https://www.12306.cn"
response = requests.get(url, verify=False)
print(response.text)

还是会提示:InsecureRequestWarning: Unverified HTTPS request is being made. Adding certificate verification is strongly advised. See: https://urllib3.readthedocs.io/en/latest/advanced-usage.html#ssl-warningsInsecureRequestWarning)

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
from requests.packages import urllib3
url = "https://www.12306.cn"
urllib3.disable_warnings()
response = requests.get(url, verify=False)
print(response.status_code)
print(response.text)

方法二:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "https://www.12306.cn"
response = requests.get(url, cert=("/path/server.crt", "/path/key")) # cert:需要把12306证书下载来
print(response.status_code)
print(response.text)

文件上传:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://httpbin.org/post"
files = {"file": open("favicon.ico", "rb")}
response = requests.post(url, files=files)
print(response.text)

IP代理:

import requests
url = "https://www.baidu.com"
proxies = {
"http": "http://127.0.0.1:3456",
"https": "http://127.0.0.1:2123"
}
response = requests.get(url, proxies=proxies)
print(response.status_code)
#!/urs/bin/evn python
# -*- coding:utr-8 -*-
import requests
url = "https://www.baidu.com"
proxies = {
"http": "http://user:password@127.0.0.1:1234",
}
# 代理需要设置账户名和密码
response = requests.get(url, proxies=proxies)
print(response.status_code)

如果你的代理是通过sokces这种方式则需要pip install "requests[socks]":

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "https://www.baidu.com"
proxies = {
"http": "socks5://user:password@127.0.0.1:1234",
}
# 代理需要设置账户名和密码
response = requests.get(url, proxies=proxies)
print(response.status_code)

超时设置:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
url = "http://www.baidu.com"
response = requests.get(url, timeout=0.1)
print(response.status_code)

认证设置:

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
from requests.auth import HTTPBasicAuth
url = "http://www.baidu.com"
response = requests.get(url, auth=HTTPBasicAuth("user", ""))
print(response.status_code)

异常处理:(http://www.python-requests.org/en/master/api/#exceptions

#!/urs/bin/evn python
# -*- coding:utf-8 -*-
import requests
from requests.exceptions import ReadTimeout, HTTPError, RequestException
try:
url = "http://httpbin.org/get"
response = requests.get(url, timeout=0.1)
except ReadTimeout:
print("Timeout")
except HTTPError:
print("Error")
except RequestException:
print("error")

Requset模块的更多相关文章

  1. urllib,request 设置代理

     通常防止爬虫被反主要有以下几个策略: 1.动态设置User-Agent(随机切换User-Agent,模拟不同用户的浏览器信息) 2.使用IP地址池:VPN和代理IP,现在大部分网站都是根据IP来b ...

  2. Flask 框架中 上下文基础理念,包括cookie,session存储方法,requset属性,current_app模块和g模块

    Flask中上下文,分为请求上下文和应用上下文.既状态留存 ,就是把变量存在某一个地方可以调用 请求上下文:实际就是request和session用法理念,既都是可以存储东西. 应用上下文:既变量共享 ...

  3. nodejs原生模块简介

    一.Express框架 前面的章节已经介绍过了,可以使用npm来安装node.js模块.具体操作请参照以前写的nodejs概论. Express是一个nodejs的web开源框架,用于快速的搭建web ...

  4. python3之模块urllib

    urllib是python内置的HTTP请求库,无需安装即可使用,它包含了4个模块: request:它是最基本的http请求模块,用来模拟发送请求 error:异常处理模块,如果出现错误可以捕获这些 ...

  5. Node学习HTTP模块(HTTP 服务器与客户端)

    Node学习HTTP模块(HTTP 服务器与客户端) Node.js 标准库提供了 http 模块,其中封装了一个高效的 HTTP 服务器和一个简易的HTTP 客户端.http.Server 是一个基 ...

  6. python 网络编程之socket开发和socketserver模块

    一 客户端/服务器架构 1.硬件C/S架构(打印机) 2.软件C/S架构 互联网中处处是C/S架构 如黄色网站是服务端,你的浏览器是客户端(B/S架构也是C/S架构的一种) 腾讯作为服务端为你提供视频 ...

  7. urllib 模块 https://www.cnblogs.com/guishou/articles/7089496.html

    1.基本方法 urllib.request.urlopen(url, data=None, [timeout, ]*, cafile=None, capath=None, cadefault=Fals ...

  8. Django之用户认证—auth模块

    用户认知———auth模块 目录: auth模块 User对象 实例 扩展默认的auth_user表 - 创建超级用户 - python3 manager.py createsuperuser - 认 ...

  9. 用户认证系统 django.contrib.auth模块

    一 导入auth模块 from django.contrib.auth.models import User from django.contrib import auth auth模块的操作针对的就 ...

随机推荐

  1. 关于Jupyter Notebook快捷操作

    Jupyter Notebook 的快捷键 Jupyter Notebook 有两种键盘输入模式.编辑模式,允许你往单元中键入代码或文本:这时的单元框线是绿色的.命令模式,键盘输入运行程序命令:这时的 ...

  2. php中一个"异类"语法: $a && $b = $c; 【转载】

    $a = 1;$b = 2;$c = 3;$a && $b = $c;echo "a:$a";echo "b:$b";echo "c: ...

  3. PHP7.2中安装mcrypt扩展的问题

    当前使用的是宝塔工具,PHP版本是7.2,据说只有7.1以下的版本才会默认安装了mcrypt功能. 所以的7.2的安装如下: yum install libmcrypt libmcrypt-devel ...

  4. NumPy基础

    一.NumPy ndarray (一)生成ndarray 表 数组生成函数 函数 描述(默认数据类型是float64) array 将输入数据(序列型对象)转换为ndarray,若不显示的指定数据类型 ...

  5. django2.0再写一行代码

    基础 @LTS长期支持 @django-admin startproject xxx python manage.py runserver python manage.py startapp xxx ...

  6. 05 JS基础DOM

    JS的window对象定时器: window下一些方法: <script> 弹出 window.alert('hello') 返回布尔值 var ret = window.confirm( ...

  7. MPC学习笔记1:基于状态空间模型的预测控制(1)

    MPC调节器 1.给定一个由状态空间法描述的离散系统: MPC控制器与其他线性二次调节器(LQR)的区别就在于其可以很好的将系统动态约束纳入考虑. 采样周期Ts控制了算法的效率,太大会错过很多系统运行 ...

  8. 两将军问题、拜占庭将军问题、TCP三路握手过程的联系

    2015年初时产生了一个疑问:基于不可靠的通信链路,为什么在两将军问题中永远无法达到共识,而在TCP三路握手中可以? 今天抽出了一些时间进行研究发现,实际上TCP三路握手也不是完全可靠的,只是一个近似 ...

  9. Druid中配置双数据库

    配置如下 <?xml version="1.0" encoding="UTF-8"?> <beans xmlns="http://w ...

  10. conda国内源的设置 by dwSun

    conda国内源的设置 by dwSun anaconda是一个balabalabala... 知道这个软件的人肯定不用介绍,不知道的也不必介绍. conda是anaconda的包管理器,通过cond ...