python 爬虫资料

API

1. Requests
2. PyQuery
   
   http://www.tuicool.com/articles/UZrmUb2
   
   http://blog.csdn.net/cnmilan/article/details/8727308
3. scrapy
   
   http://www.cnblogs.com/voidsky/p/5490798.html

用法

1. 循环处理

import requests
from pyquery import PyQuery as pq
from lxml import etree
//
try:
    response = requests.get("http://www.d0cs.org/", timeout=5).text
    doc = pq(etree.fromstring(response))
    for title in doc('div#post-title>h1>a').items():
        print title.text() + '\n'
except Exception, e:
    print e