笔记－urllib-parse

笔记－urllib-parse

1. 简介
模块官方解释
This module defines a standard interface to break Uniform Resource Locator (URL) strings up in components (addressing scheme, network location, path etc.), to combine the components back into a URL string, and to convert a “relative URL” to an absolute URL given a “base URL.”
简单来说就是处理/解析URL的。
2. 解析函数
2.1. urllib.parse
urllib.parse.urlparse(urlstring, scheme='', allow_fragments=True)
函数用于将一个URL解析成六个部分，返回一个元组，URL的格式为：scheme://netloc/path;parameters?query#fragment；包含六个部分，元组中每一个元素都是一个字符串，可以为空，这六个部分均不能再被分割成更小的部分；
以下为返回的元组元素：

元素 编号 值 值不存在时默认值
scheme 0 请求 一定存在
netloc 1 网址 空字符串
path 2 分层路径 空字符串
params 3 参数 空字符串
query 4 查询组件 空字符串
fragment 5 标识符 空字符串
username 用户名 None
password 密码 None
hostname 主机名 None
port 端口号 None

典型使用：
import urllib.parse
o = urllib.parse.urlparse('https://blog.csdn.net/qq_41769259/article/details/79434494')
print(o)

2.2. parse_qs
urllib.parse.parse_qs(qs, keep_blank_values=False, strict_parsing=False, encoding='utf-8', errors='replace')
这个函数主要用于分析URL中query组件的参数，返回一个key-value对应的字典格式。
简单应用：
print(urllib.parse.parse_qs("FuncNo=9009001&username=1"))
输出：
{'FuncNo': ['9009001'], 'username': ['1']}

urllib.parse.parse_qsl(qs, keep_blank_values=False, strict_parsing=False, encoding=’utf-8’, errors=’replace’)
这个函数和urllib.parse.parse_qs（）作用一样，唯一的区别就是这个函数返回值是list形式；

2.3. urlunparse()
urllib.parse.urlunparse(parts)
Construct a URL from a tuple as returned by urlparse(). The parts argument can be any six-item iterable. This may result in a slightly different, but equivalent URL, if the URL that was parsed originally had unnecessary delimiters (for example, a ? with an empty query; the RFC states that these are equivalent).
这个函数可以将urlparse（）分解出来的元组组装成URL；
o = urllib.parse.urlparse('https://www.zhihu.com/question/50056807/answer/2235669')
print(o)
print(urllib.parse.urlunparse(o))
output:
ParseResult(scheme='https', netloc='www.zhihu.com', path='/question/50056807/answer/2235669', params='', query='', fragment='')
https://www.zhihu.com/question/50056807/answer/2235669
2.4. urlsplit()
urllib.parse.urlsplit(urlstring, scheme='', allow_fragments=True)
This is similar to urlparse(), but does not split the params from the URL. This should generally be used instead of urlparse() if the more recent URL syntax allowing parameters to be applied to each segment of the path portion of the URL (see RFC 2396) is wanted. A separate function is needed to separate the path segments and parameters. This function returns a 5-tuple: (addressing scheme, network location, path, query, fragment identifier).
这个函数和urlparse()功能类似，但是不会将param分离出来；相比urlparse()少一个param元素，返回的元组元素参照urlparse()的元组表。
2.5. urljoin()
urllib.parse.urljoin(base, url, allow_fragments=True)
将一个基址和一个相对地址组合成新的URL。
>>> urljoin('http://www.cwi.nl/%7Eguido/Python.html', 'FAQ.html')
'http://www.cwi.nl/%7Eguido/FAQ.html'
值得注意的是如果被组合的地址是绝对地址，那么结果会不一样，在编程中应通过预处理尽量避免这种情况：
>>> urljoin('http://www.cwi.nl/%7Eguido/Python.html',
... '//www.python.org/%7Eguido')
'http://www.python.org/%7Eguido'

3. quote
The URL quoting functions focus on taking program data and making it safe for use as URL components by quoting special characters and appropriately encoding non-ASCII text. They also support reversing these operations to recreate the original data from the contents of a URL component if that task isn’t already covered by the URL parsing functions above.
这个模块的主要作用就是通过引入合适编码和特殊字符对URL进行安全重构，并且可以反向解析
3.1. quote()
urllib.parse.quote(string, safe='/', encoding=None, errors=None)
Replace special characters in string using the %xx escape. Letters, digits, and the characters '_.-' are never quoted. By default, this function is intended for quoting the path section of URL. The optional safe parameter specifies additional ASCII characters that should not be quoted — its default value is '/'.
案例：
url="https://www.zhihu.com/question/50056807/answer/223566912"
print(urllib.parse.quote(url))
print(urllib.parse.quote(url,safe=":"))
输出：
https%3A//www.zhihu.com/question/50056807/answer/223566912
https:%2F%2Fwww.zhihu.com%2Fquestion%2F50056807%2Fanswer%2F223566912

3.2. quote_plus()
urllib.parse.quote_plus(string, safe='', encoding=None, errors=None)
Like quote(), but also replace spaces by plus signs, as required for quoting HTML form values when building up a query string to go into a URL. Plus signs in the original string are escaped unless they are included in safe. It also does not have safe default to '/'.
Example: quote_plus('/El Niño/') yields '%2FEl+Ni%C3%B1o%2F'.
这个函数和quote()相似，但是这个函数能把空格转成加号，并且safe的默认值为空
3.3. quote_from_bytes()
urllib.parse.quote_from_bytes(bytes, safe=’/’)
和quote()相似，但是接收的是字节而不是字符；