scrapy常用命令（持续） | Commonly used Scrapy command list (con't)

以下命令都是在CMD中运行，首先把路径定位到项目文件夹

------------------------------------------------------------------------------------------

1. 创建一个scrapy project

scrapy startproject project_name

--------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------

2. 自动生成spider代码命令

首先进入项目目录，然后敲入如下命令

scrapy genspider spider_name start_url

------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------

3. 列出项目中所有spider

scrapy list

返回项目中所有爬虫的名字

-------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------

4. 运行一个scrapy project,并且保存结果到csv文件

scrapy crawl spider_name -o file.csv

会通过yield命令生成iteration generator，把数据保存到file.csv文件中

---------------------------------------------------------------------------------------------

English Version

The Below commands will run at CMD which included python path in system env path.

Before run the commands, navigate to the directory where you want to create Scrapy project.

------------------------------------------------------------------------------------------

1. Createa a Scrapy project

scrapy startproject project_name

--------------------------------------------------------------------------------------------

------------------------------------------------------------------------------------------

2. Create a spider. It will create a spider sub-class in your source code based on template.

Navigete to project directory, then key in below command

scrapy genspider spider_name start_url

------------------------------------------------------------------------------------------

--------------------------------------------------------------------------------------

3. List all spiders in your project

scrapy list

It will return all spiders with spider name

-------------------------------------------------------------------------------------------

---------------------------------------------------------------------------------------

4. Run a Scrapy project and save the result set into a CSV file.

scrapy crawl spider_name -o file.csv -t csv

Scrapy framework will make a iteration generator thru yield command in code and save your data into file.csv

---------------------------------------------------------------------------------------------