1.记我的第一次python爬虫爬取网页视频

It is my first time to public some notes on this platform, and I just want to improve myself by recording something that I learned everyday.

Partly , I don't know much about network crawler , and that makes me just understanding something that floats on the surface.

But since I was learning three days when I got a method to craw some videos on the web.

I am very excited, I just know how to craw something from the internet to computer hard disk. It is a start,  surely, this is the first step, I just got to keep moving.

Step 1: Find a video on the web page, then plays the video online, press the keyboard shortcuts F12, it occurs element-checked page

as the following pictures:

Click .ts file and then you will see the URL, that is the point.

Step 2: Writing python code,  as following:

 from multiprocessing import Pool
 import requests

 def demo(i):
     try:
         url = "https://vip.holyshitdo.com/2019/5/8/c2417/playlist%0d.ts"%i
         #simulate browser
         print(url)
         headers={'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36Name','Referer':'http://91.com','Content-Type': 'multipart/form-data; session_language=cn_CN'}
         r = requests.get(url, headers=headers)
         #print(r.content) save the video with binary format
         with open('./mp4/{}'.format(url[-10:]),'wb')as f:
             f.write(r.content)
     except:
         return ""

 if __name__=='__main__':　　　　　　　　　　 #　program entry
     pool = Pool(10)　　　　　　　　　　　　　 #　create a process pool
     for i in range(193):
         pool.apply_async(demo,(i,))　　　　#　execute

     pool.close()
     pool.join()

Step 3:Running code

Step 4 : Last but not least, merge .ts fragments into MP4 format.

Get to the terminal interface , under the saved diretory and use command line "copy /b *.ts newfile.mp4"

-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------

THAT IS ALL FOR NOW, TO BE CONTINUED～(￣▽￣～)~