猫眼电影网址：www.maoyan.com

前言：网上一些大神已经对猫眼电影进行过爬取，所用的方法也是各有其优，最终目的是把影片排名、图片、名称、主要演员、上映时间与评分提取出来并保存到文件或者数据库，本人也是巩固知识进行练习，整理笔记方便阅读

获取单个网页

 # 获取单个网页html

    def get_one_page(self,url):

        try:

            response =requests.get(url)

            if response.status_code==200:

                return response.text

            return None

        except RequestException:

            return None

利用正则表达式解析网页

def pare_one_page(self,html):

    pattern = re.compile('<dd>.*?board-index.*?>(\d+)</i>.*?data-src="(.*?)"'

                         '.*?name"><a.*?>(.*?)</a>.*?star">(.*?)</p>.*?releasetime">(.*?)'

                         '</p>.*?integer">(.*?)</i>.*?fraction">(.*?)</i>.*?</dd>',re.S)

    items = re.findall(pattern,html)

    for item in items:

        yield {

            # 索引

            'index':item[0],

            # 封面

            'image':item[1],

            # 电影名称

            'name': item[2],

            # 演员

            'actor': item[3].strip()[3:],

            # 上映时间

            'time': item[4].strip()[5:],

            # 评分

            'score': item[5]+item[6]

        }

写入文件

def write_to_file(self,content):

    with open('maoyan.txt','a',encoding='utf-8') as f:

        f.write(json.dumps(content,ensure_ascii=False)+'\n')

        f.close()

执行写入，打印到控制台

# 猫眼电影TOP榜url拼接

url = 'https://maoyan.com/board/4?offset='+str(offset)

html = self.get_one_page(url)

for item in self.pare_one_page(html):

    self.write_to_file(item)

    print(item)

循环10页

if __name__ == '__main__':

   m = maoyan()

   for i in range(10):

       m.main(i*10)

猫眼电影7-22日的数据

{"index": "", "image": "https://p1.meituan.net/movie/20803f59291c47e1e116c11963ce019e68711.jpg@160w_220h_1e_1c", "name": "霸王别姬", "actor": "张国荣,张丰毅,巩俐", "time": "1993-01-01", "score": "9.5"}

{"index": "", "image": "https://p0.meituan.net/movie/283292171619cdfd5b240c8fd093f1eb255670.jpg@160w_220h_1e_1c", "name": "肖申克的救赎", "actor": "蒂姆·罗宾斯,摩根·弗里曼,鲍勃·冈顿", "time": "1994-09-10(加拿大)", "score": "9.5"}

{"index": "", "image": "https://p0.meituan.net/movie/289f98ceaa8a0ae737d3dc01cd05ab052213631.jpg@160w_220h_1e_1c", "name": "罗马假日", "actor": "格利高里·派克,奥黛丽·赫本,埃迪·艾伯特", "time": "1953-09-02(美国)", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/6bea9af4524dfbd0b668eaa7e187c3df767253.jpg@160w_220h_1e_1c", "name": "这个杀手不太冷", "actor": "让·雷诺,加里·奥德曼,娜塔莉·波特曼", "time": "1994-09-14(法国)", "score": "9.5"}

{"index": "", "image": "https://p1.meituan.net/movie/b607fba7513e7f15eab170aac1e1400d878112.jpg@160w_220h_1e_1c", "name": "泰坦尼克号", "actor": "莱昂纳多·迪卡普里奥,凯特·温丝莱特,比利·赞恩", "time": "1998-04-03", "score": "9.5"}

{"index": "", "image": "https://p0.meituan.net/movie/da64660f82b98cdc1b8a3804e69609e041108.jpg@160w_220h_1e_1c", "name": "唐伯虎点秋香", "actor": "周星驰,巩俐,郑佩佩", "time": "1993-07-01(中国香港)", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/46c29a8b8d8424bdda7715e6fd779c66235684.jpg@160w_220h_1e_1c", "name": "魂断蓝桥", "actor": "费雯·丽,罗伯特·泰勒,露塞尔·沃特森", "time": "1940-05-17(美国)", "score": "9.2"}

{"index": "", "image": "https://p0.meituan.net/movie/223c3e186db3ab4ea3bb14508c709400427933.jpg@160w_220h_1e_1c", "name": "乱世佳人", "actor": "费雯·丽,克拉克·盖博,奥利维娅·德哈维兰", "time": "1939-12-15(美国)", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/ba1ed511668402605ed369350ab779d6319397.jpg@160w_220h_1e_1c", "name": "天空之城", "actor": "寺田农,鹫尾真知子,龟山助清", "time": "", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/b0d986a8bf89278afbb19f6abaef70f31206570.jpg@160w_220h_1e_1c", "name": "辛德勒的名单", "actor": "连姆·尼森,拉尔夫·费因斯,本·金斯利", "time": "1993-12-15(美国)", "score": "9.2"}

{"index": "", "image": "https://p1.meituan.net/movie/18e3191039d5e71562477659301f04aa61905.jpg@160w_220h_1e_1c", "name": "喜剧之王", "actor": "周星驰,莫文蔚,张柏芝", "time": "1999-02-13(中国香港)", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/14a7b337e8063e3ce05a5993ed80176b74208.jpg@160w_220h_1e_1c", "name": "大闹天宫", "actor": "邱岳峰,毕克,富润生", "time": "1965-12-31", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/6bc004d57358ee6875faa5e9a1239140128550.jpg@160w_220h_1e_1c", "name": "音乐之声", "actor": "朱莉·安德鲁斯,克里斯托弗·普卢默,埃琳诺·帕克", "time": "1965-03-02(美国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/ae7245920d95c03765fe1615f3a1fe3865785.jpg@160w_220h_1e_1c", "name": "春光乍泄", "actor": "张国荣,梁朝伟,张震", "time": "1997-05-30(中国香港)", "score": "9.2"}

{"index": "", "image": "https://p1.meituan.net/movie/6a964e9cee699267053bd6a4bf6f2671195394.jpg@160w_220h_1e_1c", "name": "剪刀手爱德华", "actor": "约翰尼·德普,薇诺娜·瑞德,黛安·韦斯特", "time": "1990-12-06(美国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/43d259ecbcd53e8bbe902632772281d6327525.jpg@160w_220h_1e_1c", "name": "美丽人生", "actor": "罗伯托·贝尼尼,尼可莱塔·布拉斯基,乔治·坎塔里尼", "time": "1997-12-20(意大利)", "score": "9.3"}

{"index": "", "image": "https://p0.meituan.net/movie/2bbaa395c825920fac397706369746e4300686.jpg@160w_220h_1e_1c", "name": "海上钢琴师", "actor": "蒂姆·罗斯,普路特·泰勒·文斯,比尔·努恩", "time": "1998-10-28(意大利)", "score": "9.2"}

{"index": "", "image": "https://p1.meituan.net/movie/ad974d3527879f00be2eec29135118163728582.jpg@160w_220h_1e_1c", "name": "黑客帝国", "actor": "基努·里维斯,凯瑞-安·莫斯,劳伦斯·菲什伯恩", "time": "2000-01-14", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/932bdfbef5be3543e6b136246aeb99b8123736.jpg@160w_220h_1e_1c", "name": "指环王3：王者无敌", "actor": "伊莱贾·伍德,伊恩·麦克莱恩,丽芙·泰勒", "time": "2004-03-15", "score": "9.2"}

{"index": "", "image": "https://p1.meituan.net/movie/aacb9ed2a6601bfe515ef0970add1715623792.jpg@160w_220h_1e_1c", "name": "哈利·波特与魔法石", "actor": "丹尼尔·雷德克里夫,鲁伯特·格林特,艾玛·沃特森", "time": "2002-01-26", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/09658109acfea0e248a63932337d8e6a4268980.jpg@160w_220h_1e_1c", "name": "蝙蝠侠：黑暗骑士", "actor": "克里斯蒂安·贝尔,希斯·莱杰,阿伦·伊克哈特", "time": "2008-07-18(美国)", "score": "9.3"}

{"index": "", "image": "https://p0.meituan.net/movie/8959888ee0c399b0fe53a714bc8a5a17460048.jpg@160w_220h_1e_1c", "name": "楚门的世界", "actor": "金·凯瑞,劳拉·琳妮,诺亚·艾默里奇", "time": "1998-06-01(美国)", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/b05b94b28eca53f325ae8d807fcd4ce01798036.jpg@160w_220h_1e_1c", "name": "加勒比海盗", "actor": "约翰尼·德普,凯拉·奈特莉,奥兰多·布鲁姆", "time": "2003-11-21", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/53b6f0b66882a53b08896c92076515a8236400.jpg@160w_220h_1e_1c", "name": "射雕英雄传之东成西就", "actor": "张国荣,梁朝伟,张学友", "time": "1993-02-05(中国香港)", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/606de8f394d40dbcbb9b87943fec71a2130408.jpg@160w_220h_1e_1c", "name": "无间道", "actor": "刘德华,梁朝伟,黄秋生", "time": "2003-09-05", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/bb1dee5e0b25889a2410211c1d5010ae190824.jpg@160w_220h_1e_1c", "name": "教父2", "actor": "阿尔·帕西诺,罗伯特·德尼罗,黛安·基顿", "time": "1974-12-12(美国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/267dd2483f0fb57081474c00fbea38451415571.jpg@160w_220h_1e_1c", "name": "机器人总动员", "actor": "本·贝尔特,艾丽莎·奈特,杰夫·格尔林", "time": "2008-06-27(美国)", "score": "9.3"}

{"index": "", "image": "https://p1.meituan.net/movie/dd08154878aac7c8c649fe3eeb8ccd0a2498277.jpg@160w_220h_1e_1c", "name": "指环王1：护戒使者", "actor": "伊莱贾·伍德,伊恩·麦克莱恩,丽芙·泰勒", "time": "2002-04-04", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/02bb9fd161c05bad6089133098efcdb5546589.jpg@160w_220h_1e_1c", "name": "指环王2：双塔奇兵", "actor": "伊莱贾·伍德,伊恩·麦克莱恩,丽芙·泰勒", "time": "2003-04-25", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/27b76fe6cf3903f3d74963f70786001e1438406.jpg@160w_220h_1e_1c", "name": "狮子王", "actor": "马修·布罗德里克,尼基塔·卡兰姆,詹姆斯·厄尔·琼斯", "time": "1995-07-15", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/76fc92cfa6c8f2959431b8aa604ef7ae126414.jpg@160w_220h_1e_1c", "name": "天堂电影院", "actor": "菲利浦·诺瓦雷,赛尔乔·卡斯特利托,蒂兹亚娜·罗达托", "time": "1988-11-17(意大利)", "score": "9.2"}

{"index": "", "image": "https://p0.meituan.net/movie/4c41068ef7608c1d4fbfbe6016e589f7204391.jpg@160w_220h_1e_1c", "name": "活着", "actor": "葛优,巩俐,牛犇", "time": "1994-05-18(法国)", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/a2a287c77415dc1f85b04d288f7d63ab1089754.jpg@160w_220h_1e_1c", "name": "拯救大兵瑞恩", "actor": "汤姆·汉克斯,马特·达蒙,汤姆·塞兹摩尔", "time": "1998-11-13", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/0127b451d5b8f0679c6f81c8ed414bb2432442.jpg@160w_220h_1e_1c", "name": "哈尔的移动城堡", "actor": "倍赏千惠子,木村拓哉,美轮明宏", "time": "2004-11-20(日本)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/7787c10ad5e95b03cf83ef9473500d8e282796.jpg@160w_220h_1e_1c", "name": "忠犬八公的故事", "actor": "Forest,理查·基尔,琼·艾伦", "time": "2010-03-12(英国)", "score": "9.3"}

{"index": "", "image": "https://p1.meituan.net/movie/bc022b86345c643ca21d759166f77a553679589.jpg@160w_220h_1e_1c", "name": "疯狂原始人", "actor": "尼古拉斯·凯奇,艾玛·斯通,瑞安·雷诺兹", "time": "2013-04-20", "score": "9.5"}

{"index": "", "image": "https://p1.meituan.net/movie/2f344a9f9575edbcae9f0abe0578bc90339773.jpg@160w_220h_1e_1c", "name": "盗梦空间", "actor": "莱昂纳多·迪卡普里奥,渡边谦,约瑟夫·高登-莱维特", "time": "2010-09-01", "score": "9.2"}

{"index": "", "image": "https://p0.meituan.net/movie/a08f65e6cb50fab32df5da69ff116f593095363.jpg@160w_220h_1e_1c", "name": "幽灵公主", "actor": "松田洋治,石田百合子,田中裕子", "time": "1998-05-01", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/7e471a9171a410ebc9413b2f1de67afc130067.jpg@160w_220h_1e_1c", "name": "东邪西毒", "actor": "张国荣,梁朝伟,刘嘉玲", "time": "1994-09-17", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/91f575ec93f019f428d1f33e3ceca7c5115495.jpg@160w_220h_1e_1c", "name": "阿凡达", "actor": "萨姆·沃辛顿,佐伊·索尔达娜,米歇尔·罗德里格兹", "time": "2010-01-04", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/b3defc07dfaa1b6f5b74852ce38a3f8f242792.jpg@160w_220h_1e_1c", "name": "搏击俱乐部", "actor": "爱德华·哈里森·诺顿,布拉德·皮特,海伦娜·伯翰·卡特", "time": "1999-10-15(美国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/4f9638ba234c3fb673f23a09968db875371576.jpg@160w_220h_1e_1c", "name": "风之谷", "actor": "岛本须美,永井一郎,坂本千夏", "time": "", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/06ec3c1c647942b1e40bca84036014e9490863.jpg@160w_220h_1e_1c", "name": "V字仇杀队", "actor": "娜塔莉·波特曼,雨果·维文,斯蒂芬·瑞", "time": "2006-03-17(美国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/df15efd261060d3094a73ef679888d4f238149.jpg@160w_220h_1e_1c", "name": "十二怒汉", "actor": "亨利·方达,李·科布,马丁·鲍尔萨姆", "time": "1957-04-13(美国)", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/5896de3c1474277730e321c9b1db04a9205644.jpg@160w_220h_1e_1c", "name": "当幸福来敲门", "actor": "威尔·史密斯,贾登·史密斯,坦迪·牛顿", "time": "2008-01-17", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/b0d97e4158b47d653d7a81d66f7dd3092146907.jpg@160w_220h_1e_1c", "name": "驯龙高手", "actor": "杰伊·巴鲁切尔,杰拉德·巴特勒,亚美莉卡·费雷拉", "time": "2010-05-14", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/845ce32778a1b3f258de089f91a3979b5766154.jpg@160w_220h_1e_1c", "name": "速度与激情5", "actor": "范·迪塞尔,保罗·沃克,道恩·强森", "time": "2011-05-12", "score": "9.2"}

{"index": "", "image": "https://p0.meituan.net/movie/70de97ebb6b5251ecb7c3f6d7a782a7f189340.jpg@160w_220h_1e_1c", "name": "放牛班的春天", "actor": "热拉尔·朱尼奥,让-巴蒂斯特·莫尼耶,玛丽·布奈尔", "time": "2004-10-16", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/f8e9d5a90224746d15dfdbd53d4fae3d209420.jpg@160w_220h_1e_1c", "name": "勇敢的心", "actor": "梅尔·吉布森,苏菲·玛索,帕特里克·麦高汉", "time": "1995-05-24(美国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/85c2bfba6025bfbfb53291ae5924c215308805.jpg@160w_220h_1e_1c", "name": "神偷奶爸", "actor": "史蒂夫·卡瑞尔,杰森·席格尔,拉塞尔·布兰德", "time": "2010-07-09(美国)", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/8d7b0b902afd4ec1a3dd7a9c6149463c187734.jpg@160w_220h_1e_1c", "name": "闻香识女人", "actor": "阿尔·帕西诺,克里斯·奥唐纳,加布里埃尔·安瓦尔", "time": "1992-12-23(美国)", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/ca4a128a5a54d5b5e35ceba622636c831810197.jpg@160w_220h_1e_1c", "name": "三傻大闹宝莱坞", "actor": "阿米尔·汗,黄渤,卡琳娜·卡普", "time": "2011-12-08", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/e71affe126eeb4f8bfcc738cbddeebc8288766.jpg@160w_220h_1e_1c", "name": "断背山", "actor": "希斯·莱杰,杰克·吉伦哈尔,米歇尔·威廉姆斯", "time": "2006-01-13(美国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/2e383b5f5f306f10f9f26d9f1c28cf1d825537.jpg@160w_220h_1e_1c", "name": "黑客帝国3：矩阵革命", "actor": "基努·里维斯,雨果·维文,凯瑞-安·莫斯", "time": "2003-11-05", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/34998e31c6d07475f1add6b8b16fd21d192579.jpg@160w_220h_1e_1c", "name": "少年派的奇幻漂流", "actor": "苏拉·沙玛,伊尔凡·可汗,塔布", "time": "2012-11-22", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/47dd790e19dad72b50580641de5608c5199014.jpg@160w_220h_1e_1c", "name": "飞屋环游记", "actor": "爱德华·阿斯纳,乔丹·长井,鲍勃·彼德森", "time": "2009-08-04", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/21b9211eb1094af360842472018db634286646.jpg@160w_220h_1e_1c", "name": "末代皇帝", "actor": "尊龙,陈冲,彼得·奥图尔", "time": "1987-10-23", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/0b507aa44c4dfbbcc91949b69b1b39a168922.jpg@160w_220h_1e_1c", "name": "鬼子来了", "actor": "姜文,姜宏波,陈强", "time": "2000-05-12(法国戛纳)", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/92eb862c42c49f8e41e459c369c4512b226610.jpg@160w_220h_1e_1c", "name": "大话西游之月光宝盒", "actor": "周星驰,莫文蔚,吴孟达", "time": "2014-10-24", "score": "9.6"}

{"index": "", "image": "https://p0.meituan.net/movie/15f1ac49b6d1ff7b71207672993ed6901536456.jpg@160w_220h_1e_1c", "name": "怦然心动", "actor": "玛德琳·卡罗尔,卡兰·麦克奥利菲,艾丹·奎因", "time": "2010-08-06(美国)", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/75c0d3eb584be030a01f2e26741a8f41251454.jpg@160w_220h_1e_1c", "name": "致命魔术", "actor": "休·杰克曼,克里斯蒂安·贝尔,迈克尔·凯恩", "time": "2006-10-20(美国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/7b7d1f8aa36d7a15463ce6942708a1a7265296.jpg@160w_220h_1e_1c", "name": "美丽心灵", "actor": "罗素·克洛,詹妮弗·康纳利,艾德·哈里斯", "time": "2001-12-21(美国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/fcc17667b8343131101eeb4c67d90bf9150883.jpg@160w_220h_1e_1c", "name": "无敌破坏王", "actor": "约翰·C·赖利,萨拉·西尔弗曼,简·林奇", "time": "2012-11-06", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/96d98200d2afb4b87ff189f9c15b6545568339.jpg@160w_220h_1e_1c", "name": "倩女幽魂", "actor": "张国荣,王祖贤,午马", "time": "2011-04-30", "score": "9.2"}

{"index": "", "image": "https://p0.meituan.net/movie/7ec873ba943f13e3c63789d899bd0e23256871.jpg@160w_220h_1e_1c", "name": "夜访吸血鬼", "actor": "汤姆·克鲁斯,布拉德·皮特,克尔斯滕·邓斯特", "time": "1994-11-11(美国)", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/96bb58f3e9d213fb0438987d16d27561379209.jpg@160w_220h_1e_1c", "name": "蝙蝠侠：黑暗骑士崛起", "actor": "克里斯蒂安·贝尔,迈克尔·凯恩,加里·奥德曼", "time": "2012-08-27", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/484171372de45945e8bbbcc97db57e09136701.jpg@160w_220h_1e_1c", "name": "钢琴家", "actor": "艾德里安·布洛迪,艾米莉娅·福克斯,米哈乌·热布罗夫斯基", "time": "2002-09-25(法国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/2526f77c650bf7cf3d5ee2dccdeac332244951.jpg@160w_220h_1e_1c", "name": "本杰明·巴顿奇事", "actor": "布拉德·皮特,凯特·布兰切特,塔拉吉·P·汉森", "time": "2008-12-25(美国)", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/bb0eca029cd25329776a4549b3fbe262924727.jpg@160w_220h_1e_1c", "name": "哈利·波特与死亡圣器（下）", "actor": "丹尼尔·雷德克里夫,鲁伯特·格林特,艾玛·沃特森", "time": "2011-08-04", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/0b0d45b58946078dd24d4945dd6be3b51329411.jpg@160w_220h_1e_1c", "name": "甜蜜蜜", "actor": "黎明,张曼玉,曾志伟", "time": "2015-02-13", "score": "9.2"}

{"index": "", "image": "https://p1.meituan.net/movie/7ed07b8ea8c0e0d0c7b685d20e3ec64e232004.jpg@160w_220h_1e_1c", "name": "初恋这件小事", "actor": "马里奥·毛瑞尔,平采娜·乐维瑟派布恩,阿查拉那·阿瑞亚卫考", "time": "2012-06-05", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/1e700e53e4fe29dd5942381bb353c8532239179.jpg@160w_220h_1e_1c", "name": "触不可及", "actor": "弗朗索瓦·克鲁塞,奥玛·希,安娜·勒尼", "time": "2011-11-02(法国)", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/7833126c8c21a11571bb52fbdece0acb811449.jpg@160w_220h_1e_1c", "name": "新龙门客栈", "actor": "张曼玉,梁家辉,甄子丹", "time": "2012-02-24", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/2a0783b4fd95566568f24adfad2181bb5392280.jpg@160w_220h_1e_1c", "name": "熔炉", "actor": "孔刘,郑有美,金智英", "time": "2011-09-22(韩国)", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/dc2246233a6f5ac1e34c7176b602c8ca174557.jpg@160w_220h_1e_1c", "name": "大话西游之大圣娶亲", "actor": "周星驰,朱茵,莫文蔚", "time": "2014-10-24", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/135c612860fae899df2220149664d97a173555.jpg@160w_220h_1e_1c", "name": "小鞋子", "actor": "默罕默德·阿米尔·纳吉,Kamal Mirkarimi,Behzad Rafi", "time": "1999-01-22(美国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/1199dc6273680f175fd9b06c9c36d08a219658.jpg@160w_220h_1e_1c", "name": "教父", "actor": "马龙·白兰度,阿尔·帕西诺,詹姆斯·肯恩", "time": "2015-04-18", "score": "9.3"}

{"index": "", "image": "https://p0.meituan.net/movie/19653e8af59cf473cd40f9ccc0658d93692304.jpg@160w_220h_1e_1c", "name": "素媛", "actor": "李来,薛耿求,严志媛", "time": "2013-10-02(韩国)", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/5420be40e3b755ffe04779b9b199e935256906.jpg@160w_220h_1e_1c", "name": "萤火之森", "actor": "内山昂辉,佐仓绫音,后藤弘树", "time": "2011-09-17(日本)", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/d5970e36c8868a4b746c80f3b3f8a404174615.jpg@160w_220h_1e_1c", "name": "穿条纹睡衣的男孩", "actor": "阿沙·巴特菲尔德,维拉·法梅加,大卫·休里斯", "time": "2008-09-12(英国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/3985eaf3858bea0f2a3d966bf7ee2103178217.jpg@160w_220h_1e_1c", "name": "窃听风暴", "actor": "乌尔里希·穆埃,塞巴斯蒂安·科赫,马蒂娜·格德克", "time": "2006-03-23(德国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/4abc8c932cfacfc0089e2883765d02d1295222.jpg@160w_220h_1e_1c", "name": "时空恋旅人", "actor": "瑞秋·麦克亚当斯,多姆纳尔·格里森,比尔·奈伊", "time": "2013-09-04(英国)", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/6a6e74b2c289f9fa4433dd2dc04a7741331638.jpg@160w_220h_1e_1c", "name": "7号房的礼物", "actor": "柳承龙,郑镇荣,朴信惠", "time": "2013-01-23(韩国)", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/b5ff0216e689b3fcc065590c48cd5105255305.jpg@160w_220h_1e_1c", "name": "恐怖直播", "actor": "河正宇,李璟荣,李大为", "time": "2013-07-31(韩国)", "score": "8.8"}

{"index": "", "image": "https://p0.meituan.net/movie/7373dbba07b50ce6f24336edb96b2ea4271536.jpg@160w_220h_1e_1c", "name": "海豚湾", "actor": "里克·奥巴瑞,路易·西霍尤斯,哈迪·琼斯", "time": "2009-07-31(美国)", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/c835b3588d0061ed3b992388a0a96f15160913.jpg@160w_220h_1e_1c", "name": "忠犬八公物语", "actor": "仲代达矢,春川真澄,井川比佐志", "time": "1987-08-01(日本)", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/b553d13f30100db731ab6cf45668e52d94703.jpg@160w_220h_1e_1c", "name": "上帝之城", "actor": "亚历桑德雷·罗德里格斯,艾莉丝·布拉加,莱安德鲁·菲尔米诺", "time": "2002-08-30(巴西)", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/92198a6fc8c3f5d13aa1bdf203572c0f99438.jpg@160w_220h_1e_1c", "name": "美国往事", "actor": "罗伯特·德尼罗,詹姆斯·伍兹,伊丽莎白·麦戈文", "time": "2015-04-23", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/4ffca83fd972f71e291f8ea8d78a4b58594878.jpg@160w_220h_1e_1c", "name": "七武士", "actor": "三船敏郎,志村乔,千秋实", "time": "1954-04-26(日本)", "score": "9.1"}

{"index": "", "image": "https://p1.meituan.net/movie/30310858fdab34c7a17cfd7ec8ad8bfc112201.jpg@160w_220h_1e_1c", "name": "完美的世界", "actor": "凯文·科斯特纳,克林特·伊斯特伍德,T·J·劳瑟", "time": "1993-11-24(美国)", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/2c0a5fedf4b43d142121b91c6ccabe1b59051.jpg@160w_220h_1e_1c", "name": "一一", "actor": "吴念真,金燕玲,李凯莉", "time": "2000-09-20(法国)", "score": "8.9"}

{"index": "", "image": "https://p0.meituan.net/movie/3e5f5f3aa4b7e5576521e26c2c7c894d253975.jpg@160w_220h_1e_1c", "name": "英雄本色", "actor": "狄龙,张国荣,周润发", "time": "2017-11-17", "score": "9.2"}

{"index": "", "image": "https://p0.meituan.net/movie/885fc379c614a2b4175587b95ac98eb95045650.jpg@160w_220h_1e_1c", "name": "阿飞正传", "actor": "张国荣,张曼玉,刘德华", "time": "2018-06-25", "score": "8.8"}

{"index": "", "image": "https://p1.meituan.net/movie/36a893c53a13f9bb934071b86ae3b5c492427.jpg@160w_220h_1e_1c", "name": "爱·回家", "actor": "俞承豪,金艺芬,童孝熙", "time": "2002-04-05(韩国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/0018b57299d0d4540330a31244c880a9112971.jpg@160w_220h_1e_1c", "name": "海洋", "actor": "雅克·贝汉,姜文,兰斯洛特·佩林", "time": "2011-08-12", "score": "9.0"}

{"index": "", "image": "https://p1.meituan.net/movie/ed50b58bf636d207c56989872a91f4cf305138.jpg@160w_220h_1e_1c", "name": "我爱你", "actor": "宋在浩,李顺才,尹秀晶", "time": "2011-02-17(韩国)", "score": "9.0"}

{"index": "", "image": "https://p0.meituan.net/movie/cd18ed2c5cda9e71e17e5e6ef61ced172912303.jpg@160w_220h_1e_1c", "name": "黄金三镖客", "actor": "克林特·伊斯特伍德,李·范·克里夫,埃里·瓦拉赫", "time": "1966-12-23(意大利)", "score": "8.9"}

{"index": "", "image": "https://p1.meituan.net/movie/a1634f4e49c8517ae0a3e4adcac6b0dc43994.jpg@160w_220h_1e_1c", "name": "迁徙的鸟", "actor": "雅克·贝汉,Philippe Labro", "time": "2001-12-12(法国)", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/c304c687e287c7c2f9e22cf78257872d277201.jpg@160w_220h_1e_1c", "name": "龙猫", "actor": "秦岚,糸井重里,岛本须美", "time": "2018-12-14", "score": "9.1"}

{"index": "", "image": "https://p0.meituan.net/movie/30b20139e68c46d02e0893277d633b701292458.jpg@160w_220h_1e_1c", "name": "千与千寻", "actor": "柊瑠美,周冬雨,入野自由", "time": "2019-06-21", "score": "9.3"}






完整代码




import requests

import re

import json

from requests.exceptions import RequestException

from multiprocessing import  Pool

class maoyan():

    # 获取单个网页html

    def get_one_page(self,url):

        try:

            response =requests.get(url)

            if response.status_code==200:

                return response.text

            return None

        except RequestException:

            return None

    # 利用正则表达式解析

    def pare_one_page(self,html):

        pattern = re.compile(
'<dd>.*?board-index.*?>(\d+)</i>.*?data-src="(.*?)"'
   '.*?name"><a.*?>(.*?)</a>.*?star">(.*?)</p>.*?releasetime">(.*?)'
     '</p>.*?integer">(.*?)</i>.*?fraction">(.*?)</i>.*?</dd>',re.S)
        items = re.findall(pattern,html)

        for item in items:

            yield {

                # 索引

                'index':item[0],

                # 封面

                'image':item[1],

                # 电影名称

                'name': item[2],

                # 演员

                'actor': item[3].strip()[3:],

                # 上映时间

                'time': item[4].strip()[5:],

                # 评分

                'score': item[5]+item[6]

            }

    # 写入文件

    def write_to_file(self,content):

        with open('maoyan.txt','a',encoding='utf-8') as f:

            f.write(json.dumps(content,ensure_ascii=False)+'\n')

            f.close()

    def main(self,offset):

        # 猫眼电影TOP榜url拼接

        url = 'https://maoyan.com/board/4?offset='+str(offset)

        html = self.get_one_page(url)

        for item in self.pare_one_page(html):

            self.write_to_file(item)

            print(item)

if __name__ == '__main__':

   m = maoyan()

   for i in range(10):

       m.main(i*10)

   # pool = Pool()

   # pool.map(m.main,[i*10 for i in range(10)])










												


											
Requests+正则表达式爬取猫眼电影(TOP100榜)的更多相关文章
	

    
								
  1. PYTHON 爬虫笔记八:利用Requests+正则表达式爬取猫眼电影top100（实战项目一）
		
    利用Requests+正则表达式爬取猫眼电影top100 目标站点分析 流程框架 爬虫实战 使用requests库获取top100首页: import requests def get_one_pag ...
    
		
    2. 
						
  2. Python爬虫实战之Requests+正则表达式爬取猫眼电影Top100
		
    import requests from requests.exceptions import RequestException import re import json # from multip ...
    
		
    3. 
						
  3. python爬虫从入门到放弃（九）之 Requests+正则表达式爬取猫眼电影TOP100
		
    import requests from requests.exceptions import RequestException import re import json from multipro ...
    
		
    4. 
						
  4. 50 行代码教你爬取猫眼电影 TOP100 榜所有信息
		
    对于Python初学者来说,爬虫技能是应该是最好入门,也是最能够有让自己有成就感的,今天,恋习Python的手把手系列,手把手教你入门Python爬虫,爬取猫眼电影TOP100榜信息,将涉及到基础爬虫 ...
    
		
    5. 
						
  5. 40行代码爬取猫眼电影TOP100榜所有信息
		
    主要内容: 一.基础爬虫框架的三大模块 二.完整代码解析及效果展示 1️⃣  基础爬虫框架的三大模块 1.HTML下载器:利用requests模块下载HTML网页. 2.HTML解析器:利用re正则表 ...
    
		
    6. 
						
  6. 爬虫系列（1）-----python爬取猫眼电影top100榜
		
    对于Python初学者来说,爬虫技能是应该是最好入门,也是最能够有让自己有成就感的,今天在整理代码时,整理了一下之前自己学习爬虫的一些代码,今天先上一个简单的例子,手把手教你入门Python爬虫,爬取 ...
    
		
    7. 
						
  7. Python爬虫之requests+正则表达式抓取猫眼电影top100以及瓜子二手网二手车信息(四)
		
    requests+正则表达式抓取猫眼电影top100 一.首先我们先分析下网页结构 可以看到第一页的URL和第二页的URL的区别在于offset的值,第一页为0,第二页为10,以此类推. 二.< ...
    
		
    8. 
						
  8. Python爬虫项目--爬取猫眼电影Top100榜
		
    本次抓取猫眼电影Top100榜所用到的知识点: 1. python requests库 2. 正则表达式 3. csv模块 4. 多进程 正文 目标站点分析 通过对目标站点的分析, 来确定网页结构,  ...
    
		
    9. 
						
  9. 用requests库爬取猫眼电影Top100
		
    这里需要注意一下,在爬取猫眼电影Top100时,网站设置了反爬虫机制,因此需要在requests库的get方法中添加headers,伪装成浏览器进行爬取 import requests from re ...
    
		
    10. 
		

	


随机推荐
	

    
									
  1. 「PowerBI相关」一款极其优秀的DAX建模工具Tabular Editor
			
    做Excel插件开发的时间久了,大部分人的印象是笔者是做软件开发,谈不上软件,也是插件开发级别. 但笔者更希望大家记住的身份是做BI开发,特别是企业级的BI. 故也借着Excel催化剂这个平台,输送一 ...
    
			
    2. 
						
  2. SpringBoot集成beetl模板快速入门
			
    SpringBoot集成beetl模板快速入门 首次探索 beetl官方网址:http://ibeetl.com/ 创建SpringBoot工程(idea) 新建工程 选择创建Spring工程 书写包 ...
    
			
    3. 
						
  3. javaee+tomcat新特性，乱码问题
			
    Tomcat版本问题,servlet乱码问题 我在学习的时候,老师用的是Tomcat1.7版本,在jsp发送get请求的时候,Servlet中还要对get请求传递过来的参数进行解码编码,因为tomca ...
    
			
    4. 
						
  4. android实现倒计时，最简单实现RecyclerView倒计时+SwipeRefreshLayout下拉刷新
			
    先上效果图: RecyclerView + SwipeRefreshLayout 实现倒计时效果 MainActivity.java package top.wintp.counttimedemo1; ...
    
			
    5. 
						
  5. 小白开学Asp.Net Core 《一》
			
    在开篇中介绍了项目的搭建以及项目中所用到的第三方工具 本篇介绍SqlSugar和Dapper在本项目的实现 一.SqlSugar SqlSuagr的介绍就直接浏览官方网站,官网地址将在底部给出. 在本 ...
    
			
    6. 
						
  6. 【MySQL】（六）锁
			
    开发多用户.数据库驱动的应用时,最大的一个难点是:一方面要最大程度地利用数据库的并发访问,另一方面还要确保每个用户能以一致的方式读取和修改数据.为此就有了锁(locking)的机制,同时这也是数据库系 ...
    
			
    7. 
						
  7. PHP与ECMAScript_4_常用数学相关函数
			
    PHP ECMAScript 向上取整 ceil($number) Math.ceil( number ) 向下取整 floor($number) Math.floor( number ) 绝对值 a ...
    
			
    8. 
						
  8. Javaweb入门 数据库第二天
			
    接着说昨天语法中提到的drop,delete和truncate的区别 drop用于删除库和表,不能用于删除表记录 delete和truncate都可以用于删除表记录,不能用于删除库和表 而delete ...
    
			
    9. 
						
  9. SpringBoot Admin 使用指南
			
    什么是 SpringBoot Admin? Spring Boot Admin 是一个管理和监控你的 Spring Boot 应用程序的应用程序.这些应用程序通过 Spring Boot Admin  ...
    
			
    10. 
						
  10. 2.2.2python的BeautifulSoup库
			
    from bs4 import BeautifulSoupimport rebroken_html = '<ul class="country"><li>A ...
    
			
    11.