定时爬虫抓当日免费应用：Scrapy + Tkinter + LaunchControl

花了个周末学了下Scrapy，正好一直想买mindnode，于是顺手做了个爬虫，抓取爱范儿每天的限免应用信息。

Thinking

大概思路就是使用LaunchControl每天定时（比如早上9点50，这时正好刚到公司不久）跑一下爬虫脚本，如果找到感兴趣的应用在限免，就使用Tkinter弹出提示。当然，也可以直接用Scrapy做定时任务，以后再说。

Coding

Scrapy ＋ Tkinter

# -*- coding: utf-8 -*-
import scrapy
import Tkinter
from scrapy.shell import inspect_response
import json

# 设置感兴趣的app名称
I_want_apps = set(['mindnode pro', 'u.memory'])

class XianmianSpider(scrapy.Spider):
    user_agent = 'Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/49.0.2623.112 Safari/537.36'
    name = "xianmian"
    allowed_domains = ["app.so"]
    start_urls = (
        'http://app.so/api/v1.1/appso/discount/?platform=web&limit=10',
    )

    def parse(self, response):

        jsonresponse = json.loads(response.body_as_unicode())

        apps = jsonresponse['objects']

        appTitles = {item['display_name'].lower() for item in apps}

        self.logger.info('today\' apps are: ' + str(appTitles))

        the_apps = appTitles & I_want_apps
        if the_apps:
            self.showMsg('found the apps: {}'.format(list(the_apps)))

    def showMsg(self, msg):
        import Tkinter
        root = Tkinter.Tk()
        root.title('福利到！')
        label = Tkinter.Label(root, text=msg)
        label.pack()
        center_window(root, 300, 240)
        root.maxsize(600, 400)
        root.minsize(300, 240)
        Tkinter.mainloop()

def get_screen_size(window):
    return window.winfo_screenwidth(),window.winfo_screenheight()  

def get_window_size(window):
    return window.winfo_reqwidth(),window.winfo_reqheight()  

def center_window(root, width, height):
    screenwidth = root.winfo_screenwidth()
    screenheight = root.winfo_screenheight()
    size = '%dx%d+%d+%d' % (width, height, (screenwidth - width)/2, (screenheight - height)/2)
    print(size)
    root.geometry(size)

LaunchControl

LaunchControl用起来比较直观。当然，也可以直接用mac自带的launchctl，具体可参考launchctl使用说明