PyQuery 是 Python 仿照 jQuery 的严格实现。语法与 jQuery 几乎完全相同。

官方文档：http://pyquery.readthedocs.io/

安装

1
pip install pyquery

初始化

字符串初始化

1
2
3
4
5
6
7
8
9
10
11
12
13
14
html = '''
<div>
    <ul>
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
print(doc('li'))
 
 

URL初始化

1
2
3
from pyquery import PyQuery as pq
doc = pq(url='http://www.baidu.com')
print(doc('head'))

文件初始化

1
2
3
from pyquery import PyQuery as pq
doc = pq(filename='demo.html')
print(doc('li'))

基本CSS选择器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
html = '''
<div id="container">
    <ul class="list">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
print(doc('#container .list li'))

查找元素

子元素

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
html = '''
<div id="container">
    <ul class="list">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
items = doc('.list')
print(type(items))
print(items)
lis = items.find('li')
print(type(lis))
print(lis)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
html = '''
<div id="container">
    <ul class="list">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
items = doc('.list')
lis = items.children()
print(type(lis))
print(lis)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
html = '''
<div id="container">
    <ul class="list">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
items = doc('.list')
lis = items.children('.active')
print(lis)

父元素

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
html = '''
<div id="container">
    <ul class="list">
         <li class="item-0">first item</li>
         <li class="item-1"><a href="link2.html">second item</a></li>
         <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
         <li class="item-1 active"><a href="link4.html">fourth item</a></li>
         <li class="item-0"><a href="link5.html">fifth item</a></li>
     </ul>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
items = doc('.list')
container = items.parent()
print(type(container))
print(container)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
items = doc('.list')
parents = items.parents()
print(type(parents))
print(parents)
1
2
parent = items.parents('.wrap')
print(parent)

兄弟元素

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('.list .item-0.active')
print(li.siblings())
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('.list .item-0.active')
print(li.siblings('.active'))

遍历

单个元素

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('.item-0.active')
print(li)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
lis = doc('li').items()
print(type(lis))
for li in lis:
    print(li)

获取信息

获取属性

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
a = doc('.item-0.active a')
print(a)
print(a.attr('href'))
print(a.attr.href)

获取文本

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
a = doc('.item-0.active a')
print(a)
print(a.text())

获取HTML

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('.item-0.active')
print(li)
print(li.html())

DOM操作

addClass、removeClass

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('.item-0.active')
print(li)
li.removeClass('active')
print(li)
li.addClass('active')
print(li)

attr、css

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('.item-0.active')
print(li)
li.attr('name', 'link')
print(li)
li.css('font-size', '14px')
print(li)

remove

1
2
3
4
5
6
7
8
9
10
11
12
html = '''
<div class="wrap">
    Hello, World
    <p>This is a paragraph.</p>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
wrap = doc('.wrap')
print(wrap.text())
wrap.find('p').remove()
print(wrap.text())

其他DOM方法 http://pyquery.readthedocs.io/en/latest/api.html

伪类选择器

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
html = '''
<div class="wrap">
    <div id="container">
        <ul class="list">
             <li class="item-0">first item</li>
             <li class="item-1"><a href="link2.html">second item</a></li>
             <li class="item-0 active"><a href="link3.html"><span class="bold">third item</span></a></li>
             <li class="item-1 active"><a href="link4.html">fourth item</a></li>
             <li class="item-0"><a href="link5.html">fifth item</a></li>
         </ul>
     </div>
 </div>
'''
from pyquery import PyQuery as pq
doc = pq(html)
li = doc('li:first-child')
print(li)
li = doc('li:last-child')
print(li)
li = doc('li:nth-child(2)')
print(li)
li = doc('li:gt(2)')
print(li)
li = doc('li:nth-child(2n)')
print(li)
li = doc('li:contains(second)')
print(li)

常见的爬虫分析库（4）-爬虫之PyQuery的更多相关文章

  1. 常见的爬虫分析库（3）-Python正则表达式与re模块

    在线正则表达式测试 http://tool.oschina.net/regex/ 常见匹配模式 模式 描述 \w 匹配字母数字及下划线 \W 匹配非字母数字下划线 \s 匹配任意空白字符,等价于 [\ ...

  2. 常见的爬虫分析库（2）-xpath语法

    xpath简介 1.xpath使用路径表达式在xml和html中进行导航 2.xpath包含标准函数库 3.xpath是一个w3c的标准 xpath节点关系 1.父节点 2.子节点 3.同胞节点 4. ...

  3. 常见的爬虫分析库（1）-Python3中Urllib库基本使用

    原文来自:https://www.cnblogs.com/0bug/p/8893677.html 什么是Urllib? Python内置的HTTP请求库 urllib.request          ...

  4. 爬虫requests库 之爬虫贴吧

    首先要观察爬虫的URL规律,爬取一个贴吧所有页的数据,观察点击下一页时URL是如何变化的. 思路: 定义一个类,初始化方法什么都不用管 定义一个run方法,用来实现主要逻辑 3 class Tieba ...

  5. 2.爬虫 urlib库讲解 异常处理、URL解析、分析Robots协议

    1.异常处理 URLError类来自urllib库的error模块,它继承自OSError类,是error异常模块的基类,由request模块产生的异常都可以通过这个类来处理. from urllib ...

  6. Python爬虫Urllib库的基本使用

    Python爬虫Urllib库的基本使用 深入理解urllib.urllib2及requests  请访问: http://www.mamicode.com/info-detail-1224080.h ...

  7. Python爬虫—requests库get和post方法使用

    目录 Python爬虫-requests库get和post方法使用 1. 安装requests库 2.requests.get()方法使用 3.requests.post()方法使用-构造formda ...

  8. [python爬虫]Requests-BeautifulSoup-Re库方案--Requests库介绍

    [根据北京理工大学嵩天老师“Python网络爬虫与信息提取”慕课课程编写  文章中部分图片来自老师PPT 慕课链接:https://www.icourse163.org/learn/BIT-10018 ...

  9. Python爬虫与数据分析之爬虫技能：urlib库、xpath选择器、正则表达式

    专栏目录: Python爬虫与数据分析之python教学视频.python源码分享,python Python爬虫与数据分析之基础教程:Python的语法.字典.元组.列表 Python爬虫与数据分析 ...

随机推荐

  1. “指定的参数已超出有效值的范围”在【 parameterUpdate.Add(new OracleParameter("STATUS", 0));】报错

    改成:parameterUpdate.Add()); 就不报错,并不能知道为什么,有知道为什么的,评论告诉我. /// <summary> /// 插入数据 /// </summar ...

  2. 【VMware vSphere】再谈VMware vSphere

    写在前面 在进行操作vSphere产品之前,就曾经对它进行过一个简单了解:[运维]VMware vSphere简单了解,现在再回头看,发现了解的真的是太简单了.经过前一段时间学习之后,对它又有了新的感 ...

  3. jquery获取当前按钮、截取字符串、字符串拼接、动态循环添加元素

    截取字符串:字符串拼接:动态循环添加元素:获取当前按钮: {data : null, render: function(data, type, row ) { var loginName = $(&q ...

  4. centos如何安装Python3

    Linux下默认系统自带python2.6的版本,这个版本被系统很多程序所依赖,所以不建议删除,如果使用最新的Python3那么我们知道编译安装源码包和系统默认包之间是没有任何影响的,所以可以安装py ...

  5. 关于在Fragment中设置toolbar及菜单的方法

    在NoActionBar的主题中onCreateOptionsMenu方法不会运行,这里就需要将toolbar强制转换为ActionBar 在加入toolbar的监听之类后需要在onCreateVie ...

  6. REST风格接口测试利器Wisdom rest-client

    前言 偶然间接触到Wisdom rest-client这款测试工具,后来经过尝试体验,感觉还不错,现在分享给大家,如何使用这款测试利器 Wisdom rest-client是什么? Wisdom re ...

  7. JMeter 中_time 函数的使用（时间戳、当前时间）

    有时在接口测试时,其参数需要为时间戳或者是日期格式的数据.针对此问题,可以使用JMeter自带的_time 函数来解决以上问题 操作步骤: 1.通过函数助手,生成一个_time 函数: 2.如果参数为 ...

  8. [Linux]流媒体服务器概述

    二.何为流媒体与流式传输? 2.1 流媒体 「流媒体」是指采用「流式传输」的方式在Internet播放的媒体格式,流媒体最大的特点就是「边下边播」,常用的流媒体格式有FLV(使用FLASH作为视频播放 ...

  9. Python 队列

    import multiprocessing import time if __name__ == '__main__': # 创建消息队列 # 3: 表示消息队列最大个数 queue = multi ...

  10. windows 系统纯净版官网下载地址

    http://www.imsdn.cn/operating-systems/windows-7/