python使用selenium

首先安装

pip install selenium

测试抓取baidu，其中的chromedriver.exe需要自己下载，百度有很多的

import time
from selenium import webdriver

browser = webdriver.Chrome('C:\Program Files (x86)\Google\ChromeDriver\chromedriver.exe') # 可选参数，如果不指定将搜索环境变量
browser.get('http://www.baidu.com/')

time.sleep(5)

执行结果如下

进阶

import time
from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Chrome('C:\Program Files (x86)\Google\ChromeDriver\chromedriver.exe')
driver.get('https://www.baidu.com/')
elem = driver.find_element_by_id('kw')
elem.send_keys("test")
elem.send_keys(Keys.RETURN)
print(driver.page_source)

#<input type="text" name="passwd" id="passwd-id" />

element = driver.find_element_by_id("passwd-id")
element = driver.find_element_by_name("passwd")
element = driver.find_elements_by_tag_name("input")
element = driver.find_element_by_xpath("//input[@id='passwd-id']")

如果遇到错误

C:\Users\bin\AppData\Local\Programs\Python\Python36\python.exe C:/Users/bin/PycharmProjects/Test/Test/Crawler/TestSelenium.py
Traceback (most recent call last):
  File "C:\Users\bin\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\common\service.py", line 74, in start
    stdout=self.log_file, stderr=self.log_file)
  File "C:\Users\bin\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 707, in __init__
    restore_signals, start_new_session)
  File "C:\Users\bin\AppData\Local\Programs\Python\Python36\lib\subprocess.py", line 990, in _execute_child
    startupinfo)
FileNotFoundError: [WinError 2] 系统找不到指定的文件。

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:/Users/bin/PycharmProjects/Test/Test/Crawler/TestSelenium.py", line 3, in <module>
    browser = webdriver.Chrome()
  File "C:\Users\bin\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 62, in __init__
    self.service.start()
  File "C:\Users\bin\AppData\Local\Programs\Python\Python36\lib\site-packages\selenium\webdriver\common\service.py", line 81, in start
    os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home

原因是没有安装chrome driver，到 https://sites.google.com/a/chromium.org/chromedriver/downloads 下载，

如果是windows就将chrome driver添加到Path环境变量中，Linux直接放到 usr/bin目录下