python爬虫: 豆瓣电影top250数据分析

转载博客 https://segmentfault.com/a/1190000005920679

根据自己的环境修改并配置mysql数据库

系统：Mac OS X 10.11

python 2.7

mysql安装

• 使用brew安装mysql，启动mysql服务

  ☁  ~  brew install mysql
  Warning: mysql-5.7.18 already installed
  ☁  ~  which mysql
  /usr/local/bin/mysql
  ☁  ~  ls /usr/local/bin/mysql*
  /usr/local/bin/mysql                      /usr/local/bin/mysql_ssl_rsa_setup        /usr/local/bin/mysqldumpslow
  /usr/local/bin/mysql.server               /usr/local/bin/mysql_tzinfo_to_sql        /usr/local/bin/mysqlimport
  /usr/local/bin/mysql_client_test          /usr/local/bin/mysql_upgrade              /usr/local/bin/mysqlpump
  /usr/local/bin/mysql_client_test_embedded /usr/local/bin/mysqladmin                 /usr/local/bin/mysqlshow
  /usr/local/bin/mysql_config               /usr/local/bin/mysqlbinlog                /usr/local/bin/mysqlslap
  /usr/local/bin/mysql_config_editor        /usr/local/bin/mysqlcheck                 /usr/local/bin/mysqltest
  /usr/local/bin/mysql_embedded             /usr/local/bin/mysqld                     /usr/local/bin/mysqltest_embedded
  /usr/local/bin/mysql_install_db           /usr/local/bin/mysqld_multi               /usr/local/bin/mysqlxtest
  /usr/local/bin/mysql_plugin               /usr/local/bin/mysqld_safe
  /usr/local/bin/mysql_secure_installation  /usr/local/bin/mysqldump
  ☁  ~  mysql.server start
  Starting MySQL
  . SUCCESS!
  

• 数据库创建，表创建

  ☁  ~  mysql -u root -p
  Enter password:
  Welcome to the MySQL monitor.  Commands end with ; or \g.
  Your MySQL connection id is 6
  Server version: 5.7.18 Homebrew
  
  Copyright (c) 2000, 2017, Oracle and/or its affiliates. All rights reserved.
  
  Oracle is a registered trademark of Oracle Corporation and/or its
  affiliates. Other names may be trademarks of their respective
  owners.
  
  Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
  
  mysql> show databases;
  +--------------------+
  | Database           |
  +--------------------+
  | information_schema |
  | alt_web            |
  | db_name            |
  | db_tendenci        |
  | douban             |
  | mysql              |
  | performance_schema |
  | sys                |
  | testdb1            |
  +--------------------+
  9 rows in set (0.00 sec)
  
  mysql>create douban; #创建douban数据库
  mysql>use douban; #更换当前数据库
      #在douban数据库中，创建表douban
  mysql>create table douban(
      id INT NOT NULL AUTO_INCREMENT,
      class VARCHAR(100) NOT NULL,
      count INT NOT NULL,
      PRIMARY KEY( id )
      )ENGINE=InnoDB DEFAULT CHARSET=utf8;
  mysql> desc douban; #显示表结构
  +-------+--------------+------+-----+---------+----------------+
  | Field | Type         | Null | Key | Default | Extra          |
  +-------+--------------+------+-----+---------+----------------+
  | id    | int(11)      | NO   | PRI | NULL    | auto_increment |
  | class | varchar(100) | NO   |     | NULL    |                |
  | count | int(11)      | NO   |     | NULL    |                |
  +-------+--------------+------+-----+---------+----------------+
  3 rows in set (0.01 sec)
  

• mysql教程

  • MySQL之终端(Terminal)管理MySQL
  • MySQL之终端（Terminal）管理数据库、数据表、数据的基本操作

python包安装

requests，lxml.etree，pymysql，matplotlib

• 使用pip安装
  
  pip install packagename

下载源代码并运行

$ python demo.py

我的源码

https://github.com/MingjaLee/spiders_douban_movies