urllib设置debuglevel打开调试开关

1. 参考

Turning on debug output for python 3 urllib

https://bugs.python.org/issue26892

Python爬虫入门四之Urllib库的高级用法  5.使用DebugLog

2. 代码

(1) python2

 # python2
 import httplib
 import urllib
 httplib.HTTPConnection.debuglevel = 1
 response = urllib.urlopen('http://www.baidu.com').read() 

(2) 兼容python2和python3

 import sys
 try:
     import urllib.request as urllib_request  #python3
 except:
     import urllib2 as urllib_request  #python2
 print(sys.version)
 httpHandler = urllib_request.HTTPHandler(debuglevel=1)
 httpsHandler = urllib_request.HTTPSHandler(debuglevel=1)
 opener = urllib_request.build_opener(httpHandler, httpsHandler)
 print(opener.open('https://httpbin.org/user-agent').read().decode('utf-8'))
 #install_opener之后才能全局
 urllib_request.install_opener(opener)
 response = urllib_request.urlopen('http://www.baidu.com')