根据 savefrom条例

本实例及教程只用于学习交流用，权利归savefrom.net所有

最后代码+注释大概100行左右，具体代码以github代码为主(可以会在上面修复bug)，本文只做具体讲解

项目地址

github仓库

思路

用python做youtube自动化下载器 思路

流程

1. post

根据思路里的第一步，我们首先需要用post方式取到加密后的js字段，笔者使用了requests第三方库来执行，关于爬虫可以参考我之前的文章

i. 先把post中的headers格式化

# set the headers or the website will not return information

    # the cookies in here you may need to change

    headers = {

        "cache-Control": "no-cache",

        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,"

                  "*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",

        "accept-encoding": "gzip, deflate, br",

        "accept-language": "zh-CN,zh;q=0.9,en;q=0.8",

        "content-type": "application/x-www-form-urlencoded",

        "cookie": "lang=en; country=CN; uid=fd94a82a406a8dd4; sfHelperDist=72; reference=14; "

                  "clickads-e2=90; poropellerAdsPush-e=63; promoBlock=64; helperWidget=92; "

                  "helperBanner=42; framelessHdConverter=68; inpagePush2=68; popupInOutput=9; "

                  "_ga=GA1.2.799702638.1610248969; _gid=GA1.2.628904587.1610248969; "

                  "PHPSESSID=030393eb0776d20d0975f99b523a70d4; x-requested-with=; "

                  "PHPSESSUD=islilfjn5alth33j9j8glj9776; _gat_helperWidget=1; _gat_inpagePush2=1",

        "origin": "https://en.savefrom.net",

        "pragma": "no-cache",

        "referer": "https://en.savefrom.net/1-youtube-video-downloader-4/",

        "sec-ch-ua": "\"Google Chrome\";v=\"87\", \"Not;A Brand\";v=\"99\",\"Chromium\";v=\"87\"",

        "sec-ch-ua-mobile": "?0",

        "sec-fetch-dest": "iframe",

        "sec-fetch-mode": "navigate",

        "sec-fetch-site": "same-origin",

        "sec-fetch-user": "?1",

        "upgrade-insecure-requests": "1",

        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "

                      "Chrome/87.0.4280.88 Safari/537.36"}

其中cookie部分可能要改，然后最好以你们浏览器上的为主，具体每个参数的含义不是本文范围，可以自行去搜索引擎搜

ii.然后把参数也格式化

# set the parameter, we can get from chrome

    kv = {"sf_url": url,

          "sf_submit": "",

          "new": "1",

          "lang": "en",

          "app": "",

          "country": "cn",

          "os": "Windows",

          "browser": "Chrome"}

其中sf_url字段是我们要下载的youtube视频的url，其他参数都不变

iii. 最后再执行requests库的post请求

# do the POST request

    r = requests.post(url="https://en.savefrom.net/savefrom.php", headers=headers,

                      data=kv)

    r.raise_for_status()

注意是data=kv

iv. 封装成一个函数

import requests

def gethtml(url):

    # set the headers or the website will not return information

    # the cookies in here you may need to change

    headers = {

        "cache-Control": "no-cache",

        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,"

                  "*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",

        "accept-encoding": "gzip, deflate, br",

        "accept-language": "zh-CN,zh;q=0.9,en;q=0.8",

        "content-type": "application/x-www-form-urlencoded",

        "cookie": "lang=en; country=CN; uid=fd94a82a406a8dd4; sfHelperDist=72; reference=14; "

                  "clickads-e2=90; poropellerAdsPush-e=63; promoBlock=64; helperWidget=92; "

                  "helperBanner=42; framelessHdConverter=68; inpagePush2=68; popupInOutput=9; "

                  "_ga=GA1.2.799702638.1610248969; _gid=GA1.2.628904587.1610248969; "

                  "PHPSESSID=030393eb0776d20d0975f99b523a70d4; x-requested-with=; "

                  "PHPSESSUD=islilfjn5alth33j9j8glj9776; _gat_helperWidget=1; _gat_inpagePush2=1",

        "origin": "https://en.savefrom.net",

        "pragma": "no-cache",

        "referer": "https://en.savefrom.net/1-youtube-video-downloader-4/",

        "sec-ch-ua": "\"Google Chrome\";v=\"87\", \"Not;A Brand\";v=\"99\",\"Chromium\";v=\"87\"",

        "sec-ch-ua-mobile": "?0",

        "sec-fetch-dest": "iframe",

        "sec-fetch-mode": "navigate",

        "sec-fetch-site": "same-origin",

        "sec-fetch-user": "?1",

        "upgrade-insecure-requests": "1",

        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "

                      "Chrome/87.0.4280.88 Safari/537.36"}

    # set the parameter, we can get from chrome

    kv = {"sf_url": url,

          "sf_submit": "",

          "new": "1",

          "lang": "en",

          "app": "",

          "country": "cn",

          "os": "Windows",

          "browser": "Chrome"}

    # do the POST request

    r = requests.post(url="https://en.savefrom.net/savefrom.php", headers=headers,

                      data=kv)

    r.raise_for_status()

    # get the result

    return r.text

2. 调用解密函数

i. 分析

这其中的难点在于在python里执行javascript代码，而晚上的解决方法有PyV8等，本文选用execjs。在思路部分我们可以发现js部分的最后几行是解密函数，所以我们只需要在execjs中先执行一遍全部，然后再单独执行解密函数就好了

ii. 先取出js部分

# target(youtube address) url

    url = "https://www.youtube.com/watch?v=YPvtz1lHRiw"

    # get the target text

    reo = gethtml(url)

    # Remove the code from the head and tail (we need the javascript part, information store with encryption in js part)

    reo = reo.split("<script type=\"text/javascript\">")[1].split("</script>")[0]

这里其实可以用正则，不过由于笔者正则表达式还不太熟练就直接用split了

iii. 取第一个解密函数作为我们用的解密函数

当你多取几次不同视频的结果，你就会发现每次的解密函数都不一样，不过位置都是还是在固定行数

# split each line(help us find the decrypt function in last few line)

    reA = reo.split("\n")

    # get the depcrypt function

    name = reA[len(reA) - 3].split(";")[0] + ";"

所以name就是我们的解密函数了(变量名没取太好hhh)

iv. 用execjs执行

# use execjs to execute the js code, and the cwd is the result of `npm root -g`(the path of npm in your computer)

    ct = execjs.compile(reo)

    # do the decryption

    text = ct.eval(name.split("=")[1].replace(";", ""))

其中只取=后面的和去掉分号是指指执行这个函数而不用赋值，当先执行赋值+解密然后取值也不是不可以

但是我们可以发现马上就报错了(要是有这么简单就好了)

1. this也就是window变量不存在

如果没记错是报错this或者$b，笔者尝试把全部this去掉或者把全部框在一个class里面(这样子this就变成那个class了)不过都没有成功，然后发现在npm下有个jsdom可以在execjs里模拟window变量(其实应该有更好方法的)，所以我们需要下载npm和里面的jsdom，然后改写以上代码

    addition = """

    const jsdom = require("jsdom");

    const { JSDOM } = jsdom;

    const dom = new JSDOM(`<!DOCTYPE html><p>Hello world</p>`);

    window = dom.window;

    document = window.document;

    XMLHttpRequest = window.XMLHttpRequest;

    """

    # use execjs to execute the js code, and the cwd is the result of `npm root -g`(the path of npm in your computer)

    ct = execjs.compile(addition + reo, cwd=r'C:\Users\xxx\AppData\Roaming\npm\node_modules')

其中

  • cwd字段是npm root -g的结果，也就是npm的modules路径
  • addition是用来模拟window的

    但是我们又可以发现下一个错误

2. alert不存在

这个错误是因为在execjs下执行alert函数是没有意义的，因为我们没有浏览器让他弹窗，且原本alert函数的定义是来源window而我们自定义了window，所以我们要在代码前重写覆盖alert函数(相当于定义一个alert)

# override the alert function, because in the code there has one place using

    # and we cannot do the alerting in execjs(it is meaningless) however, if we donnot override, the code will raise a error

    reo = reo.replace("(function(){", "(function(){\nthis.alert=function(){};")

v. 整合代码

# target(youtube address) url

    url = "https://www.youtube.com/watch?v=YPvtz1lHRiw"

    # get the target text

    reo = gethtml(url)

    # Remove the code from the head and tail (we need the javascript part, information store with encryption in js part)

    reo = reo.split("<script type=\"text/javascript\">")[1].split("</script>")[0]

    # override the alert function, because in the code there has one place using

    # and we cannot do the alerting in execjs(it is meaningless) however, if we donnot override, the code will raise a error

    reo = reo.replace("(function(){", "(function(){\nthis.alert=function(){};")

    # split each line(help us find the decrypt function in last few line)

    reA = reo.split("\n")

    # get the depcrypt function

    name = reA[len(reA) - 3].split(";")[0] + ";"

    # add jsdom into the execjs because the code will use(maybe there is a solution without jsdom, but i have no idea)

    addition = """

    const jsdom = require("jsdom");

    const { JSDOM } = jsdom;

    const dom = new JSDOM(`<!DOCTYPE html><p>Hello world</p>`);

    window = dom.window;

    document = window.document;

    XMLHttpRequest = window.XMLHttpRequest;

    """

    # use execjs to execute the js code, and the cwd is the result of `npm root -g`(the path of npm in your computer)

    ct = execjs.compile(addition + reo, cwd=r'C:\Users\19308\AppData\Roaming\npm\node_modules')

    # do the decryption

    text = ct.eval(name.split("=")[1].replace(";", ""))

3. 分析解密结果

i. 取关键json

运行完上面的部分，解密结果就存在text里了，而我们在思路中可以发现，真正对我们重要的就是存在window.parent.sf.videoResult.show()里的json，所以用正则表达式取这一部分的json

# get the result in json

    result = re.search('show\((.*?)\);;', text, re.I | re.M).group(0).replace("show(", "").replace(");;", "")

ii. 格式化json

python可以格式化json的库有很多，这里笔者用了json库(记得import)

# use `json` to load json

    j = json.loads(result)

iii. 取下载地址

接下来就到了最后一步，根据思路里和json格式化工具我们可以发现j["url"][num]["url"]就是下载链接，而num是我们要的视频格式(不同分辨率和类型)

# the selection of video(in this case, num=1 mean the video is

    # - 360p known from j["url"][num]["quality"]

    # - MP4 known from j["url"][num]["type"]

    # - audio known from j["url"][num]["audio"]

    num = 1

    downurl = j["url"][num]["url"]

    # do some download

    # thanks :)

    # - EOF -

3. 全部代码

# -*- coding: utf-8 -*-

# @Time: 2021/1/10

# @Author: Eritque arcus

# @File: Youtube.py

# @License: MIT

# @Environment:

#           - windows 10

#           - python 3.6.2

# @Dependence:

#           - jsdom in npm(windows also can use)

#           - requests, execjs, re, json in python

import requests

import execjs

import re

import json

def gethtml(url):

    # set the headers or the website will not return information

    # the cookies in here you may need to change

    headers = {

        "cache-Control": "no-cache",

        "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,"

                  "*/*;q=0.8,application/signed-exchange;v=b3;q=0.9",

        "accept-encoding": "gzip, deflate, br",

        "accept-language": "zh-CN,zh;q=0.9,en;q=0.8",

        "content-type": "application/x-www-form-urlencoded",

        "cookie": "lang=en; country=CN; uid=fd94a82a406a8dd4; sfHelperDist=72; reference=14; "

                  "clickads-e2=90; poropellerAdsPush-e=63; promoBlock=64; helperWidget=92; "

                  "helperBanner=42; framelessHdConverter=68; inpagePush2=68; popupInOutput=9; "

                  "_ga=GA1.2.799702638.1610248969; _gid=GA1.2.628904587.1610248969; "

                  "PHPSESSID=030393eb0776d20d0975f99b523a70d4; x-requested-with=; "

                  "PHPSESSUD=islilfjn5alth33j9j8glj9776; _gat_helperWidget=1; _gat_inpagePush2=1",

        "origin": "https://en.savefrom.net",

        "pragma": "no-cache",

        "referer": "https://en.savefrom.net/1-youtube-video-downloader-4/",

        "sec-ch-ua": "\"Google Chrome\";v=\"87\", \"Not;A Brand\";v=\"99\",\"Chromium\";v=\"87\"",

        "sec-ch-ua-mobile": "?0",

        "sec-fetch-dest": "iframe",

        "sec-fetch-mode": "navigate",

        "sec-fetch-site": "same-origin",

        "sec-fetch-user": "?1",

        "upgrade-insecure-requests": "1",

        "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) "

                      "Chrome/87.0.4280.88 Safari/537.36"}

    # set the parameter, we can get from chrome

    kv = {"sf_url": url,

          "sf_submit": "",

          "new": "1",

          "lang": "en",

          "app": "",

          "country": "cn",

          "os": "Windows",

          "browser": "Chrome"}

    # do the POST request

    r = requests.post(url="https://en.savefrom.net/savefrom.php", headers=headers,

                      data=kv)

    r.raise_for_status()

    # get the result

    return r.text

if __name__ == '__main__':

    # target(youtube address) url

    url = "https://www.youtube.com/watch?v=YPvtz1lHRiw"

    # get the target text

    reo = gethtml(url)

    # Remove the code from the head and tail (we need the javascript part, information store with encryption in js part)

    reo = reo.split("<script type=\"text/javascript\">")[1].split("</script>")[0]

    # override the alert function, because in the code there has one place using

    # and we cannot do the alerting in execjs(it is meaningless) however, if we donnot override, the code will raise a error

    reo = reo.replace("(function(){", "(function(){\nthis.alert=function(){};")

    # split each line(help us find the decrypt function in last few line)

    reA = reo.split("\n")

    # get the depcrypt function

    name = reA[len(reA) - 3].split(";")[0] + ";"

    # add jsdom into the execjs because the code will use(maybe there is a solution without jsdom, but i have no idea)

    addition = """

    const jsdom = require("jsdom");

    const { JSDOM } = jsdom;

    const dom = new JSDOM(`<!DOCTYPE html><p>Hello world</p>`);

    window = dom.window;

    document = window.document;

    XMLHttpRequest = window.XMLHttpRequest;

    """

    # use execjs to execute the js code, and the cwd is the result of `npm root -g`(the path of npm in your computer)

    ct = execjs.compile(addition + reo, cwd=r'C:\Users\19308\AppData\Roaming\npm\node_modules')

    # do the decryption

    text = ct.eval(name.split("=")[1].replace(";", ""))

    # get the result in json

    result = re.search('show\((.*?)\);;', text, re.I | re.M).group(0).replace("show(", "").replace(");;", "")

    # use `json` to load json

    j = json.loads(result)

    # the selection of video(in this case, num=1 mean the video is

    # - 360p known from j["url"][num]["quality"]

    # - MP4 known from j["url"][num]["type"]

    # - audio known from j["url"][num]["audio"]

    num = 1

    downurl = j["url"][num]["url"]

    # do some download

    # thanks :)

    # - EOF -
  • 总计102行
  • 开发环境
# @Environment:

#           - windows 10

#           - python 3.6.2
  • 依赖
# @Dependence:

#           - jsdom in npm(windows also can use)

#           - requests, execjs, re, json in python

-end-

For 爬虫

版权声明：本文为博主原创文章，遵循CC 4.0 BY-SA版权协议，转载请附上原文出处链接和本声明。

本文作者: https://www.cnblogs.com/Eritque-arcus/ 或https://blog.csdn.net/qq_40832960

用python做youtube自动化下载器 代码的更多相关文章

  1. 用python做youtube自动化下载器 思路

    目录 0. 思路 1.准备 i.savfrom.net 2. 探索并规划获取方式 i.总览 ii. 获取该网页取到下载url的请求 iii. 在本地获取请求 iv.解析请求结果 v.解析解密后的结果 ...

  2. Python实现多线程HTTP下载器

    本文将介绍使用Python编写多线程HTTP下载器,并生成.exe可执行文件. 环境:windows/Linux + Python2.7.x 单线程 在介绍多线程之前首先介绍单线程.编写单线程的思路为 ...

  3. 使用appium+python做UI自动化的demo

    使用appium+python做UI自动化的demo 案例使用的知乎app,下载最新的知乎apk,存在了电脑上,只需要配置本机上app目录,不需要再配置appPackage和appActivity # ...

  4. python多进程断点续传分片下载器

    python多进程断点续传分片下载器 标签:python 下载器 多进程 因为爬虫要用到下载器,但是直接用urllib下载很慢,所以找了很久终于找到一个让我欣喜的下载器.他能够断点续传分片下载,极大提 ...

  5. Python + Selenium +Chrome 批量下载网页代码修改【新手必学】

    Python + Selenium +Chrome 批量下载网页代码修改主要修改以下代码可以调用 本地的 user-agent.txt 和 cookie.txt来达到在登陆状态下 批量打开并下载网页, ...

  6. Qt+Python开发百度图片下载器

    一.资源下载地址 https://www.aliyundrive.com/s/jBU2wBS8poH 本项目路径:项目->收费->百度图片下载器(可试用5分钟) 安装包直接下载地址:htt ...

  7. python的内置下载器

    python有个内置下载器,有时候在内部提供文件下载很好用. 进入提供下载的目录 # ls abc.aaa chpw.py finance.py lsdir.py ping.py u2d-partia ...

  8. python ddt数据驱动（简化重复代码）

    在接口自动化测试中,往往一个接口的用例需要考虑 正确的.错误的.异常的.边界值等诸多情况,然后你需要写很多个同样代码,参数不同的用例.如果测试接口很多,不但需要写大量的代码,测试数据和代码柔合在一起, ...

  9. 基于iOS 10、realm封装的下载器

    代码地址如下:http://www.demodashi.com/demo/11653.html 概要 在决定自己封装一个下载器前,我本以为没有那么复杂,可在实际开发过程中困难重重,再加上iOS10和X ...

随机推荐

  1. tensorflow GPU的使用

    参考:https://blog.csdn.net/mzpmzk/article/details/78647711 import os # 默认情况,TF 会占用所有 GPU 的所有内存, 我们可以指定 ...

  2. Java为什么称为动态编译？

    Java在程序运行时产生Java类并编译成.class文件.

  3. DVWA各等级XSS

    xss原理及基本介绍 XSS,全称Cross Site Scripting,即跨站脚本攻击,某种意义上也是一种注入攻击,是指攻击者在页面中注入恶意的脚本代码,当受害者访问该页面时,恶意代码会在其浏览器 ...

  4. ab test压力测试

    之前做性能调试的时候一直用的JMeter压测,最近发现一款简单易用的压力测试工具. ab(Apache benchmark)是一款常用的压力测试工具,是Apache附带的一个小工具 , 专门用于HTT ...

  5. ASP.NET Core 中间件的使用（二）：依赖注入的使用

    写在前面 上一篇大家已经粗略接触了解到.NET Core中间件的使用:ASP .Net Core 中间件的使用(一):搭建静态文件服务器/访问指定文件, .NET Core框架中很多核心对象都是通过依 ...

  6. 个人微信公众号搭建Python实现 -接收和发送消息-基本说明与实现（14.2.1)

    @ 目录 1.原理 2.接收普通消息 3.接收代码普通消息代码实现 1.原理 2.接收普通消息 其他消息类似参考官方文档 3.接收代码普通消息代码实现 from flask import Flask, ...

  7. 【Go语言绘图】图片添加文字（一）

    前一篇讲解了利用gg包来进行图片旋转的操作,这一篇我们来看看怎么在图片上添加文字. 绘制纯色背景 首先,我们先绘制一个纯白色的背景,作为添加文字的背景板. package main import &q ...

  8. linux -bash: unzip: 未找到命令(实测有效!)

    今天使用linux解压的时候遇到了不能解压的问题,然后就看了一些文档,写一个解决方案 Linux version 3.10.0-957.10.1.el7.x86_64 (mockbuild@kbuil ...

  9. 【实时渲染】实时3D渲染如何加速汽车线上体验应用推广

    在过去,一支优秀的广告片足以让消费者对一辆汽车产生兴趣.完美的底盘线条或引擎的轰鸣声便会让潜在买家跑到经销商那里试驾.现在,广告还是和往常一样,并没有失去其特性,但86%的买家在与销售交流之前会在网上 ...

  10. python序列（十）字典

    字典是无序可变序列. 定义字典是,每个元素的键和值用冒号分隔,元素之间用逗号分隔,所有的元素放在一对大括号"{ }"中. 字典中的键可以为任意不可变数据,比如.整数.实数.复数.字 ...