解析库

解析器 使用方法 优势 劣势
Python标准库 BeautifulSoup(markup, "html.parser") Python的内置标准库、执行速度适中 、文档容错能力强 Python 2.7.3 or 3.2.2)前的版本中文容错能力差
lxml HTML 解析器 BeautifulSoup(markup, "lxml") 速度快、文档容错能力强 需要安装C语言库
lxml XML 解析器 BeautifulSoup(markup, "xml") 速度快、唯一支持XML的解析器 需要安装C语言库
html5lib BeautifulSoup(markup, "html5lib") 最好的容错性、以浏览器的方式解析文档、生成HTML5格式的文档 速度慢、不依赖外部扩展

基本使用

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.prettify())

print(soup.title.string)

# 输出

<html>

 <head>

  <title>

   The Dormouse's story

  </title>

 </head>

 <body>

  <p class="title" name="dromouse">

   <b>

    The Dormouse's story

   </b>

  </p>

  <p class="story">

   Once upon a time there were three little sisters; and their names were

   <a class="sister" href="http://example.com/elsie" id="link1">

    <!-- Elsie -->

   </a>

   ,

   <a class="sister" href="http://example.com/lacie" id="link2">

    Lacie

   </a>

   and

   <a class="sister" href="http://example.com/tillie" id="link3">

    Tillie

   </a>

   ;

and they lived at the bottom of a well.

  </p>

  <p class="story">

   ...

  </p>

 </body>

</html>

The Dormouse's story

标签选择器

 

选择元素

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.title)

print(type(soup.title))

print(soup.head)

print(soup.p)

# 输出

<title>The Dormouse's story</title>

<class 'bs4.element.Tag'>

<head><title>The Dormouse's story</title></head>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

获取名称

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.title.name)

# 输出

title

获取属性

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.attrs['name'])

print(soup.p['name'])

# 输出

dromouse

dromouse

获取内容

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p clss="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.string)

# 输出

The Dormouse's story

嵌套选择

html = """

<html><head><title>The Dormouse's story</title></head>

<body>

<p class="title" name="dromouse"><b>The Dormouse's story</b></p>

<p class="story">Once upon a time there were three little sisters; and their names were

<a href="http://example.com/elsie" class="sister" id="link1"><!-- Elsie --></a>,

<a href="http://example.com/lacie" class="sister" id="link2">Lacie</a> and

<a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>;

and they lived at the bottom of a well.</p>

<p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.head.title.string)

# 输出

The Dormouse's story

子节点和子孙节点

html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.contents)

# 输出

['\n            Once upon a time there were three little sisters; and their names were\n            ', <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>, '\n', <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>, ' \n            and\n            ', <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>, '\n            and they lived at the bottom of a well.\n        ']
html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.children)

for i, child in enumerate(soup.p.children):

    print(i, child)

# 输出

<list_iterator object at 0x1064f7dd8>

0

            Once upon a time there were three little sisters; and their names were

1 <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

2 

3 <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

4

            and

5 <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

6

            and they lived at the bottom of a well.
html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.p.descendants)

for i, child in enumerate(soup.p.descendants):

    print(i, child)

# 输出

<generator object descendants at 0x10650e678>

0

            Once upon a time there were three little sisters; and their names were

1 <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

2 

3 <span>Elsie</span>

4 Elsie

5 

6 

7 <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

8 Lacie

9

            and

10 <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

11 Tillie

12

            and they lived at the bottom of a well.

父节点和祖先节点

html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.a.parent)

# 输出

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>
html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(list(enumerate(soup.a.parents)))

# 输出

[(0, <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>), (1, <body>

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

<p class="story">...</p>

</body>), (2, <html>

<head>

<title>The Dormouse's story</title>

</head>

<body>

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

<p class="story">...</p>

</body></html>), (3, <html>

<head>

<title>The Dormouse's story</title>

</head>

<body>

<p class="story">

            Once upon a time there were three little sisters; and their names were

            <a class="sister" href="http://example.com/elsie" id="link1">

<span>Elsie</span>

</a>

<a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>

            and

            <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

<p class="story">...</p>

</body></html>)]

兄弟节点

html = """

<html>

    <head>

        <title>The Dormouse's story</title>

    </head>

    <body>

        <p class="story">

            Once upon a time there were three little sisters; and their names were

            <a href="http://example.com/elsie" class="sister" id="link1">

                <span>Elsie</span>

            </a>

            <a href="http://example.com/lacie" class="sister" id="link2">Lacie</a>

            and

            <a href="http://example.com/tillie" class="sister" id="link3">Tillie</a>

            and they lived at the bottom of a well.

        </p>

        <p class="story">...</p>

"""

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(list(enumerate(soup.a.next_siblings)))

print(list(enumerate(soup.a.previous_siblings)))

# 输出

[(0, '\n'), (1, <a class="sister" href="http://example.com/lacie" id="link2">Lacie</a>), (2, ' \n            and\n            '), (3, <a class="sister" href="http://example.com/tillie" id="link3">Tillie</a>), (4, '\n            and they lived at the bottom of a well.\n        ')]

[(0, '\n            Once upon a time there were three little sisters; and their names were\n            ')]

标准选择器

find_all( name , attrs , recursive , text , **kwargs )

可根据标签名、属性、内容查找文档

 

name

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all('ul'))

print(type(soup.find_all('ul')[0]))

# 输出

[<ul class="list" id="list-1">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>, <ul class="list list-small" id="list-2">

<li class="element">Foo</li>

<li class="element">Bar</li>

</ul>]

<class 'bs4.element.Tag'>
html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for ul in soup.find_all('ul'):

    print(ul.find_all('li'))

# 输出

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>]

[<li class="element">Foo</li>, <li class="element">Bar</li>]

attrs

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1" name="elements">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all(attrs={'id': 'list-1'}))

print(soup.find_all(attrs={'name': 'elements'}))

# 输出

[<ul class="list" id="list-1" name="elements">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>]

[<ul class="list" id="list-1" name="elements">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>]
html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all(id='list-1'))

print(soup.find_all(class_='element'))

# 输出

[<ul class="list" id="list-1">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>]

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>, <li class="element">Foo</li>, <li class="element">Bar</li>]

text

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find_all(text='Foo'))

# 输出

['Foo', 'Foo']

find( name , attrs , recursive , text , **kwargs )

 

find返回单个元素，find_all返回所有元素

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.find('ul'))

print(type(soup.find('ul')))

print(soup.find('page'))

# 输出

<ul class="list" id="list-1">

<li class="element">Foo</li>

<li class="element">Bar</li>

<li class="element">Jay</li>

</ul>

<class 'bs4.element.Tag'>

None

find_parents() find_parent()

 

find_parents()返回所有祖先节点，find_parent()返回直接父节点。

 

find_next_siblings() find_next_sibling()

 

find_next_siblings()返回后面所有兄弟节点，find_next_sibling()返回后面第一个兄弟节点。

 

find_previous_siblings() find_previous_sibling()

 

find_previous_siblings()返回前面所有兄弟节点，find_previous_sibling()返回前面第一个兄弟节点。

 

find_all_next() find_next()

 

find_all_next()返回节点后所有符合条件的节点, find_next()返回第一个符合条件的节点

 

find_all_previous() 和 find_previous()

 

find_all_previous()返回节点后所有符合条件的节点, find_previous()返回第一个符合条件的节点

 

CSS选择器

 

通过select()直接传入CSS选择器即可完成选择：

. 代表类名        # 代表id       啥也不写是 标签名

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

print(soup.select('.panel .panel-heading'))

print(soup.select('ul li'))

print(soup.select('#list-2 .element'))

print(type(soup.select('ul')[0]))

# 输出

[<div class="panel-heading">

<h4>Hello</h4>

</div>]

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>, <li class="element">Foo</li>, <li class="element">Bar</li>]

[<li class="element">Foo</li>, <li class="element">Bar</li>]

<class 'bs4.element.Tag'>
html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for ul in soup.select('ul'):

    print(ul.select('li'))

# 输出

[<li class="element">Foo</li>, <li class="element">Bar</li>, <li class="element">Jay</li>]

[<li class="element">Foo</li>, <li class="element">Bar</li>]

获取属性

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for ul in soup.select('ul'):

    print(ul['id'])

    print(ul.attrs['id'])

# 输出

list-1

list-1

list-2

list-2

获取内容

html='''

<div class="panel">

    <div class="panel-heading">

        <h4>Hello</h4>

    </div>

    <div class="panel-body">

        <ul class="list" id="list-1">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

            <li class="element">Jay</li>

        </ul>

        <ul class="list list-small" id="list-2">

            <li class="element">Foo</li>

            <li class="element">Bar</li>

        </ul>

    </div>

</div>

'''

from bs4 import BeautifulSoup

soup = BeautifulSoup(html, 'lxml')

for li in soup.select('li'):

    print(li.get_text())

# 输出

Foo

Bar

Jay

Foo

Bar

总结

 
  • 推荐使用lxml解析库，必要时使用html.parser
  • 标签选择筛选功能弱但是速度快
  • 建议使用find()、find_all() 查询匹配单个结果或者多个结果
  • 如果对CSS选择器熟悉建议使用select()
  • 记住常用的获取属性和文本值的方法

爬虫--BeautifulSoup使用的更多相关文章

  1. 爬虫beautifulsoup实践

    爬虫beautifulsoup实践: 目的:在https://unsplash.com/上爬取图片并保存到本地文件夹里.   一.观察response.首先,在Chrome浏览器里观察一下该网页的re ...

  2. 爬虫——BeautifulSoup和Xpath

    爬虫我们大概可以分为三部分:爬取——>解析——>存储 一 Beautiful Soup: Beautiful Soup提供一些简单的.python式的函数用来处理导航.搜索.修改分析树等功 ...

  3. Python爬虫-- BeautifulSoup库

    BeautifulSoup库 beautifulsoup就是一个非常强大的工具,爬虫利器.一个灵活又方便的网页解析库,处理高效,支持多种解析器.利用它就不用编写正则表达式也能方便的实现网页信息的抓取 ...

  4. Python爬虫 | Beautifulsoup解析html页面

    引入 大多数情况下的需求,我们都会指定去使用聚焦爬虫,也就是爬取页面中指定部分的数据值,而不是整个页面的数据.因此,在聚焦爬虫中使用数据解析.所以,我们的数据爬取的流程为: 指定url 基于reque ...

  5. 网络爬虫BeautifulSoup库的使用

    使用BeautifulSoup库提取HTML页面信息 #!/usr/bin/python3 import requests from bs4 import BeautifulSoup url='htt ...

  6. 爬虫----beautifulsoup的简单使用

    beautifulSoup使用: 简单来说,Beautiful Soup是python的一个库,最主要的功能是从网页抓取数据. pip3 install beautifulsoup4 解析器 Beau ...

  7. 爬虫----BeautifulSoup模块

    一.介绍 Beautiful Soup 是一个可以从HTML或XML文件中提取数据的Python库.它能够通过你喜欢的转换器实现惯用的文档导航,查找,修改文档的方式.Beautiful Soup会帮你 ...

  8. python爬虫---BeautifulSoup的用法

    BeautifulSoup是一个灵活的网页解析库,不需要编写正则表达式即可提取有效信息. 推荐使用lxml作为解析器,因为效率更高. 在Python2.7.3之前的版本和Python3中3.2.2之前 ...

  9. Python 爬虫-BeautifulSoup

    2017-07-26 10:10:11 Beautiful Soup可以解析html 和 xml 格式的文件. Beautiful Soup库是解析.遍历.维护“标签树”的功能库.使用Beautifu ...

  10. 爬虫--BeautifulSoup

    什么是BeautifulSoup? BeautifulSoup支持的一些解析库 基本使用 from bs4 import BeautifulSoup html =""" ...

随机推荐

  1. python部分

    读取骨骼数据相关的多个json,拼接到一起 # -- coding: utf-8 -- import os path = "./test" #文件夹目录 files= os.lis ...

  2. JS的数据类型（包含:7种数据类型的介绍、数据类型的转换、数据类型的判断）

    前言 最新的 ECMAScript 标准定义了JS的 7 种数据类型,其中包括: 6 种基本类型:Boolean.Null.Undefined.Number.String.Symbol (ECMASc ...

  3. 爬取xml数据之R

    生物信息很多时候要爬数据.最近也看了一些这些方面的. url<-"要爬取的网址" url.html<-htmlParse(url,encoding="UTF- ...

  4. POJ 1979 Red and Black (简单dfs)

    题目: 简单dfs,没什么好说的 代码: #include <iostream> using namespace std; typedef long long ll; #define IN ...

  5. android之handler机制深入解析

    一.android中需要另开线程处理耗时.网络的任务,但是有必须要在UI线程中修改组件.这样做是为了: ①只能在UI线程中修改组件,避免了多线程造成组件显示混乱 ②不使用加锁策略是为了提高性能,因为a ...

  6. kindEditor编写插件遇到的问题

    kindEditor是一个功能强大的在线文本编辑器,而且提供了插件扩展功能,更好的满足用户各方面的需求.在项目中,我们就有如此的需求:在kindEditor编辑器中,添加一条下划线,并且在下划线的中间 ...

  7. 【原创】Apache和基于虚拟主机的Tomcat集群方案

    最近建设了北京某政府机构的网站,网站前段使用Apache做负载均衡,后端使用Tomcat做的集群,基于虚拟主机的方式访问,并且实现了静态资源和动态资源的分离. 开始的建设方案有两种,一种是使用apac ...

  8. A. Amr and Music

    解题思路:给出n种乐器学习所需要的时间,以及总共的天数, 问最多能够学多少门乐器,并且输出这几门乐器在原序列中的序号(不唯一) 按照升序排序,为了学到最多的乐器,肯定要选择花费时间最少的来学习 然后用 ...

  9. Codeforces Round #289 Div 2

    A. Maximum in Table 题意:给定一个表格,它的第一行全为1,第一列全为1,另外的数满足a[i][j]=a[i-1][j]+a[i][j-1],求这个表格中的最大的数 a[n][n]即 ...

  10. 手工备份恢复oracle数据库

     手工备份恢复oracle数据库: 虽然已经有了rman工具 但是手工恢复oracle能够让你对oracle数据库有更加深入的了解 数据库一致性开机条件: 数据文件 scn,控制文件 scn,redo ...