java_爬虫_从腾讯视频播放界面爬取视频真实地址

由于想在微信公众号里爬一点儿考研的视频

花了差不多一天的时间把这个爬虫做好（其实也不算爬虫吧，就算个能批量处理的地址解析器，半个爬虫）

不多说，进正题

（本文适合有java基础的同学，没基础的用客户端缓存然后格式转换吧）

所需条件：

1.一台联网的有java环境的电脑

2.耐心

访问后台接口网址：

http://vv.video.qq.com/getinfo（低清的只要这一个就好了）

http://vv.video.qq.com/getkey（高清的需要访问这个）

原理（获取低清视频，先把原理打通，高清后期有时间会更）：

步骤一：

获取你想要下载的视频的腾讯视频页面地址（这个很容易啦，就不赘述）

此处以：https://v.qq.com/x/page/f08302y6rof.html为例

步骤二：

获取视频vid

此处的vid为f08302y6rof，就是上边儿网址上那一串长长的东西

步骤三：

用获取到的视频的vid替换下面红色标明的vid（这一步是访问后台接口得到json报文）

http://vv.video.qq.com/getinfo?vids=f08302y6rof&platform=101001&charge=0&otype=json&defn=shd

然后访问

步骤四：

在页面返回浏览器的json报文中，找到fn 和 fvkey

我这边儿传回的报文如下

QZOutputJson={"dltype":1,"exem":0,"fl":{"cnt":2,"fi":[{"id":100701,"name":"msd","lmt":0,"sb":1,"cname":"标清;(270P)","br":29,"profile":2,"drm":0,"video":1,"audio":1,"fs":101567331,"super":0,"hdr10enh":0,"sl":1},{"id":2,"name":"mp4","lmt":0,"sb":1,"cname":"高清;(480P)","br":34,"profile":1,"drm":0,"video":1,"audio":1,"fs":130427092,"super":0,"hdr10enh":0,"sl":0}]},"hs":0,"ip":"111.79.225.65","ls":0,"preview":3383,"s":"o","sfl":{"cnt":0},"tm":1556431150,"vl":{"cnt":1,"vi":[{"br":29,"ch":0,"cl":{"fc":0,"keyid":"f08302y6rof.100701"},"ct":21600,"drm":0,"dsb":0,"fmd5":"74e3040ce70af50716abead16c9fba50","fn":"f08302y6rof.m701.mp4","fs":101567331,"fst":5,"fvkey":"D351DB69FA6EC791CB6DE47266F80B21BFFFAA3616A7B42975903ED5EA68589C0E2454137002A84799CF43B4FD972B415259C1F23C21CD34F2C34BC64F6D7D16F21BF3BF94F22B09491FC9D8C96CFFA3B3177345807F34EFDDAF94449E72FC3B8C55751EE9EADC5F","head":0,"hevc":0,"iflag":0,"level":0,"lnk":"f08302y6rof","logo":1,"mst":8,"pl":null,"share":1,"sp":0,"st":2,"tail":0,"td":"3383.47","ti":"2020考研数学寒假计划（第一次课)","tie":0,"type":3,"ul":{"ui":[{"url":"http://ugcws.video.gtimg.com/uwMROfz0r5zAoaQXGdGnC2dfhzlOR5XW60pRw41PvMP8tDlH/","vt":106,"dtc":0,"dt":2},{"url":"http://ugcydzd.qq.com/uwMROfz0r5zAoaQXGdGlC2dfhznfaJdqBNmJ_NLSRfZb0kLT/","vt":146,"dtc":0,"dt":2},{"url":"http://ugcsjy.qq.com/uwMROfz0r5zAoaQXGdGlK2dfhzmm-mdByiC0fycrmmUBpCVq/","vt":176,"dtc":0,"dt":2},{"url":"http://video.dispatch.tc.qq.com/uwMROfz0r5zAoaQXGdGlLGdfhzn3bYHHUWfJ-3lk8pLFnjzb/","vt":0,"dtc":0,"dt":2}]},"vh":480,"vid":"f08302y6rof","videotype":0,"vr":0,"vst":2,"vw":270,"wh":0.5625,"wl":{"wi":[{"id":19,"x":14,"y":14,"w":84,"h":27,"a":100,"md5":"dcc9dc5c478c4100ea2817c5e6020f26","url":"http://puui.qpic.cn/vcolumn_pic/0/logo_qing_xi_color_336_108.png/0","surl":"https://puui.qpic.cn/vcolumn_pic/0/logo_qing_xi_color_336_108.png/0"}]},"uptime":1548118095,"fvideo":0,"fvpint":0,"swhdcp":0}]}};

（传回来的就是一行，我直接用java代码解析了，手动找费眼睛）

步骤五：

利用获取到的fn和fvkey构建视频下载地址

此处构建的是：

http://ugcws.video.gtimg.com/f08302y6rof.m701.mp4?vkey=2E657DF01414A1F95E0B3CF7F187CEB84B3E439F5D0BA2D7F052967654DEFDE53292F0BE8BCD373FA0F269BA6BE5CC1AD5CC4AEE269AB0B1C72261815608260190B1D14D9B1820B0394DAB0C8DA1D8561F3B3455FBE5BA27D618C81A0A233256DDDAB6429E3A05FF

把获取到的fn替换前边儿一个短的标红内容

fvkey替换后边儿长的标红内容

这就是完整的视频下载地址了，可以用迅雷下载

完成

源码如下（有错误或者不规范请大佬指出，个人机器上可以运行）：

package catchVedio;

import java.io.BufferedReader;
import java.io.BufferedWriter;
import java.io.File;
import java.io.FileReader;
import java.io.FileWriter;
import java.io.IOException;
import java.io.InputStreamReader;
import java.io.UnsupportedEncodingException;
import java.net.HttpURLConnection;
import java.net.URL;
import java.util.ArrayList;
import java.util.List;

/**
 * 获取视频接口的json
 * @author Administrator
 *
 */
public class CatchVedio {
//    Socket client = new Scoket();
    private URL url;
    private HttpURLConnection urlConnection;
    private int responseCode;
    private BufferedReader reader;
    private BufferedWriter writer;

    public static void main(String[] args) {
        CatchVedio cv = new CatchVedio();
        try {

            String[] VedioURL = cv.get_VedioURL();//接收
            for(String temp:VedioURL) {//temp是每一个视频的播放地址
                cv.toDownloadURL(cv.analyse(cv.get_Json(temp)));//写出到文件
            }
        } catch (IOException e) {
            // TODO 自动生成的 catch 块
            e.printStackTrace();
        }finally {
            try {
                cv.reader.close();
                cv.writer.close();
            } catch (IOException e) {
                // TODO 自动生成的 catch 块
                e.printStackTrace();
            }
        }

    }

    void toDownloadURL(String real_url) throws IOException {//将对应下载地址输出到文件
        this.writer = new BufferedWriter(new FileWriter("D:/worm/downloadURL.txt",true));//定义追加方式写入的流
//        this.writer.append(real_url);
        this.writer.write(real_url+"\r\n");
        this.writer.flush();
    }

    String analyse(String json) {//分析json,传回完整下载地址
        int fvkey_index = json.indexOf("\"fvkey\":\"")+9;
        int endIndex = json.indexOf("\"",fvkey_index);
        String fvkey = json.substring(fvkey_index,endIndex);//获取到fvkey
//        System.out.println(fvkey);

        int fn_index = json.indexOf("\"fn\":\"")+6;
        int fn_end = json.indexOf("\"",fn_index);
        String fn = json.substring(fn_index,fn_end);//获取到视频文件名
//        System.out.println(fn);

        String head = "http://ugcws.video.gtimg.com/";

        StringBuffer real_url = new StringBuffer();
        real_url.append(head);//加入头部
        real_url.append(fn+"?");//加入文件名
        real_url.append("vkey="+fvkey);//加入解锁码
        /*构造成功*/
//        System.out.println(real_url.toString());
        return real_url.toString();

    }

    String get_Json(String url) throws UnsupportedEncodingException, IOException {
        String line = "";
        StringBuffer sb = new StringBuffer();
        this.url = new URL(url);
        this.urlConnection = (HttpURLConnection)this.url.openConnection();
        this.responseCode = this.urlConnection.getResponseCode();
        if (this.responseCode == 200) {
            this.reader = new BufferedReader(new InputStreamReader(this.urlConnection.getInputStream(), "UTF-8"));
            while ((line = this.reader.readLine()) != null) {
                sb.append(line);// 网页传回的只有一行
            }
            return sb.toString();
        }
        return "";
    }

    String[] get_VedioURL() throws IOException {
//    void get_VedioURL() throws IOException {
        File file = new File("D:/worm/vedioURL.txt");
        String line = "";
        this.reader = new BufferedReader(new FileReader(file));
        String[] t = new String[0];
        List<String> container = new ArrayList<String>();
        while(null!=(line = this.reader.readLine())) {
            if(line.equals("")) {
                continue;
            }
            line = this.change(line);//转换一下
            container.add(line);//装入容器
        }
        return container.toArray(t);
    }
    /**
     * http://vv.video.qq.com/getinfo?vids=x0164ytbgov&platform=101001&charge=0&otype=json&defn=shd //格式
     * @param str
     * @return
     * https://v.qq.com/x/page/f08302y6rof.html//页面地址示例
     * https://v.qq.com/x/page/y083158hphd.html
     * https://v.qq.com/x/page/c08503oe58c.html
     */
    String change(String str) {//定义从页面播放地址获取vid转换到后台接口地址的方法
        String head = "http://vv.video.qq.com/getinfo?vids=";
        String tail = "&platform=101001&charge=0&otype=json&defn=shd";
        String vid = str.substring(str.indexOf("page/")+5,str.indexOf(".html"));
        return head+vid+tail;
    }
}

我是输入输出都是文件操作

希望对大家有所帮助

以上