解析库

解析器 使用方法 优势 劣势
Python标准库 BeautifulSoup(markup, "html.parser") Python的内置标准库、执行速度适中 、文档容错能力强 Python 2.7.3 or 3.2.2)前的版本中文容错能力差
lxml HTML 解析器 BeautifulSoup(markup, "lxml") 速度快、文档容错能力强 需要安装C语言库
lxml XML 解析器 BeautifulSoup(markup, "xml") 速度快、唯一支持XML的解析器 需要安装C语言库
html5lib BeautifulSoup(markup, "html5lib") 最好的容错性、以浏览器的方式解析文档、生成HTML5格式的文档 速度慢、不依赖外部扩展

基本使用

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.prettify())

print(soup.title.string)

# 输出

<html>

 <head>

  <title>

   The Dormouse's story

  </title>

 </head>

 <body>

  <p class="title" name="dromouse">

   <b>

    The Dormouse's story

   </b>

  </p>

  <p class="story">

   Once upon a time there were three little sisters; and their names were

   <a class="sister" href="http://example.com/elsie" id="link1">

    <!-- Elsie -->

   </a>

   ,

   <a class="sister" href="http://example.com/lacie" id="link2">

    Lacie

   </a>

   and

   <a class="sister" href="http://example.com/tillie" id="link3">

    Tillie

   </a>

   ;

and they lived at the bottom of a well.

  </p>

  <p class="story">

   ...

  </p>

 </body>

</html>

The Dormouse's story

标签选择器

 

选择元素

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.title)

print(type(soup.title))

print(soup.head)

print(soup.p)

# 输出

<title>The Dormouse's story</title>

<class 'bs4.element.Tag'>

<head><title>The Dormouse's story</title></head>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

获取名称

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.title.name)

# 输出

title

获取属性

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.attrs['name'])

print(soup.p['name'])

# 输出

dromouse

dromouse

获取内容

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p clss="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.string)

# 输出

The Dormouse's story

嵌套选择

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.head.title.string)

# 输出

The Dormouse's story

子节点和子孙节点

html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.contents)

# 输出

['\n            Once upon a time there were three little sisters; and their names were\n            ', <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>, '\n', <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>, ' \n            and\n            ', <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>, '\n            and they lived at the bottom of a well.\n        ']
html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.children)

for i, child in enumerate(soup.p.children):

    print(i, child)

# 输出

<list_iterator object at 0x1064f7dd8>

0

            Once upon a time there were three little sisters; and their names were

1 <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

2 

3 <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

4

            and

5 <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

6

            and they lived at the bottom of a well.
html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.descendants)

for i, child in enumerate(soup.p.descendants):

    print(i, child)

# 输出

<generator object descendants at 0x10650e678>

0

            Once upon a time there were three little sisters; and their names were

1 <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

2 

3 <span>Elsie</span>

4 Elsie

5 

6 

7 <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

8 Lacie

9

            and

10 <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

11 Tillie

12

            and they lived at the bottom of a well.

父节点和祖先节点

html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.a.parent)

# 输出

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>
html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(list(enumerate(soup.a.parents)))

# 输出

[(0, <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>), (1, <body>

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

<p class="story">...</p>

</body>), (2, <html>

<head>

<title>The Dormouse's story</title>

</head>

<body>

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

<p class="story">...</p>

</body></html>), (3, <html>

<head>

<title>The Dormouse's story</title>

</head>

<body>

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

<p class="story">...</p>

</body></html>)]

兄弟节点

html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(list(enumerate(soup.a.next_siblings)))

print(list(enumerate(soup.a.previous_siblings)))

# 输出

[(0, '\n'), (1, <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>), (2, ' \n            and\n            '), (3, <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>), (4, '\n            and they lived at the bottom of a well.\n        ')]

[(0, '\n            Once upon a time there were three little sisters; and their names were\n            ')]

标准选择器

find_all( name , attrs , recursive , text , **kwargs )

可根据标签名、属性、内容查找文档

 

name

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all('ul'))

print(type(soup.find_all('ul')[0]))

# 输出

[<ul class="list" id="list-1">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>, <ul class="list list-small" id="list-2">

<li class="element">Foo</li>

<li class="element">Bar</li>

</ul>]

<class 'bs4.element.Tag'>
html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for ul in soup.find_all('ul'):

    print(ul.find_all('li'))

# 输出

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>]

[<li class="element">Foo</li>, <li class="element">Bar</li>]

attrs

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1" name="elements">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all(attrs={'id': 'list-1'}))

print(soup.find_all(attrs={'name': 'elements'}))

# 输出

[<ul class="list" id="list-1" name="elements">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>]

[<ul class="list" id="list-1" name="elements">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>]
html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all(id='list-1'))

print(soup.find_all(class_='element'))

# 输出

[<ul class="list" id="list-1">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>]

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>, <li class="element">Foo</li>, <li class="element">Bar</li>]

text

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all(text='Foo'))

# 输出

['Foo', 'Foo']

find( name , attrs , recursive , text , **kwargs )

 

find返回单个元素，find_all返回所有元素

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find('ul'))

print(type(soup.find('ul')))

print(soup.find('page'))

# 输出

<ul class="list" id="list-1">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>

<class 'bs4.element.Tag'>

None

find_parents() find_parent()

 

find_parents()返回所有祖先节点，find_parent()返回直接父节点。

 

find_next_siblings() find_next_sibling()

 

find_next_siblings()返回后面所有兄弟节点，find_next_sibling()返回后面第一个兄弟节点。

 

find_previous_siblings() find_previous_sibling()

 

find_previous_siblings()返回前面所有兄弟节点，find_previous_sibling()返回前面第一个兄弟节点。

 

find_all_next() find_next()

 

find_all_next()返回节点后所有符合条件的节点, find_next()返回第一个符合条件的节点

 

find_all_previous() 和 find_previous()

 

find_all_previous()返回节点后所有符合条件的节点, find_previous()返回第一个符合条件的节点

 

CSS选择器

 

通过select()直接传入CSS选择器即可完成选择：

. 代表类名        # 代表id       啥也不写是 标签名

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.select('.panel .panel-heading'))

print(soup.select('ul li'))

print(soup.select('#list-2 .element'))

print(type(soup.select('ul')[0]))

# 输出

[<div class="panel-heading">

<h4>Hello</h4>

</div>]

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>, <li class="element">Foo</li>, <li class="element">Bar</li>]

[<li class="element">Foo</li>, <li class="element">Bar</li>]

<class 'bs4.element.Tag'>
html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for ul in soup.select('ul'):

    print(ul.select('li'))

# 输出

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>]

[<li class="element">Foo</li>, <li class="element">Bar</li>]

获取属性

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for ul in soup.select('ul'):

    print(ul['id'])

    print(ul.attrs['id'])

# 输出

list-1

list-1

list-2

list-2

获取内容

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for li in soup.select('li'):

    print(li.get_text())

# 输出

Foo

Bar

Jay

Foo

Bar

总结

 
  • 推荐使用lxml解析库，必要时使用html.parser
  • 标签选择筛选功能弱但是速度快
  • 建议使用find()、find_all() 查询匹配单个结果或者多个结果
  • 如果对CSS选择器熟悉建议使用select()
  • 记住常用的获取属性和文本值的方法

爬虫--BeautifulSoup使用的更多相关文章

  1. 爬虫beautifulsoup实践

    爬虫beautifulsoup实践: 目的:在https://unsplash.com/上爬取图片并保存到本地文件夹里.   一.观察response.首先,在Chrome浏览器里观察一下该网页的re ...

  2. 爬虫——BeautifulSoup和Xpath

    爬虫我们大概可以分为三部分:爬取——>解析——>存储 一 Beautiful Soup: Beautiful Soup提供一些简单的.python式的函数用来处理导航.搜索.修改分析树等功 ...

  3. Python爬虫-- BeautifulSoup库

    BeautifulSoup库 beautifulsoup就是一个非常强大的工具,爬虫利器.一个灵活又方便的网页解析库,处理高效,支持多种解析器.利用它就不用编写正则表达式也能方便的实现网页信息的抓取 ...

  4. Python爬虫 | Beautifulsoup解析html页面

    引入 大多数情况下的需求,我们都会指定去使用聚焦爬虫,也就是爬取页面中指定部分的数据值,而不是整个页面的数据.因此,在聚焦爬虫中使用数据解析.所以,我们的数据爬取的流程为: 指定url 基于reque ...

  5. 网络爬虫BeautifulSoup库的使用

    使用BeautifulSoup库提取HTML页面信息 #!/usr/bin/python3 import requests from bs4 import BeautifulSoup url='htt ...

  6. 爬虫----beautifulsoup的简单使用

    beautifulSoup使用: 简单来说,Beautiful Soup是python的一个库,最主要的功能是从网页抓取数据. pip3 install beautifulsoup4 解析器 Beau ...

  7. 爬虫----BeautifulSoup模块

    一.介绍 Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你 ...

  8. python爬虫---BeautifulSoup的用法

    BeautifulSoup是一个灵活的网页解析库,不需要编写正则表达式即可提取有效信息. 推荐使用lxml作为解析器,因为效率更高. 在Python2.7.3之前的版本和Python3中3.2.2之前 ...

  9. Python 爬虫-BeautifulSoup

    2017-07-26 10:10:11 Beautiful Soup可以解析html 和 xml 格式的文件. Beautiful Soup库是解析.遍历.维护“标签树”的功能库.使用Beautifu ...

  10. 爬虫--BeautifulSoup

    什么是BeautifulSoup? BeautifulSoup支持的一些解析库 基本使用 from bs4 import BeautifulSoup html =""" ...

随机推荐

  1. ora-01157怎么解决

    在数据库startup时,出现以下两个错误:ora-01157:cannot identify/lock data file 8 -see DBWR trace fileora-01110:data ...

  2. Eclipse里Tomcat报错：Document base ……does not exist or is not a readable directory（图文详解）

    问题描述: 严重: Error starting static Resourcesjava.lang.IllegalArgumentException: Document base D:\Code\M ...

  3. python 3.x 学习笔记12 (反射 and 异常)

    1.反射通过字符串映射或修改程序运行时的状态.属性.方法 getattr(obj,name_str):  根据字符串name_str去获取obj对象里的对应的方法的内存地址 hasttr(obj,na ...

  4. http请求post，返回excel文件，并接收

    1.post的方法里要加responseType: 'arraybuffer'参数,不然下载的excel会乱码 2.使用{type: "application/vnd.ms-excel&qu ...

  5. （转载）PopuWindow和软键盘共存时的设置

    PopuWindow和软键盘共存时的设置 收藏 artshell 发表于 2年前 阅读 1499 收藏 10 点赞 2 评论 0 腾讯云上实验室 1小时搭建人工智能应用 让技术更容易入门>> ...

  6. jqGrid系列知识

    1.获取选中到行的ID var rowKey = jQuery(grid_selector).getGridParam("selrow"); 2.获取选中行除ID之外的数据 var ...

  7. proxy in java

    [Static] IFeature.java ImpicateF.java Runport.java StaticProxy.java IFeature.java package UProxy.sta ...

  8. 前端路由的两种模式:hash(#)模式和history模式(转)

    随着 ajax 的使用越来越广泛,前端的页面逻辑开始变得越来越复杂,特别是spa的兴起,前端路由系统随之开始流行. 从用户的角度看,前端路由主要实现了两个功能(使用ajax更新页面状态的情况下): 记 ...

  9. [arc076e]connected?

    题意: 给出一个$R\times C$的棋盘,其中$1$到$N$之间的每个正整数都会在棋盘上出现两次,第$i$个数出现的位置是$(X_{i,1},Y_{i,1})$和$(X_{i,2},Y_{i,2} ...

  10. [arc082e]ConvexScore

    题意: 给出直角坐标系中的$N$个点$(X_i,Y_i)$,定义由其中部分点构成的点集为“凸点集”当且仅当这些点恰好能构成一个凸多边形(内部没有其他点). 如图,点集$\{A,C,E\}$和$\{B, ...