[PyData] 01 - Web Crawler
前言
Let's go to https://www.kaggle.com/
Kaggle Notebook 有实践记录的案例。
一、线性拟合噪声数据
[Sklearn] Linear regression models to fit noisy data
二、打造 Pipeline
[Feature] Final pipeline: custom transformers
资源队列
阅读目录
- Algorithmic Trading Challenge25
- Allstate Purchase Prediction Challenge3
- Amazon.com – Employee Access Challenge6
- AMS 2013-2014 Solar Energy Prediction Contest2
- Belkin Energy Disaggregation Competition1
- Challenges in Representation Learning: Facial Expression Recognition Challenge4
- Challenges in Representation Learning: The Black Box Learning Challenge1
- Challenges in Representation Learning: Multi-modal Learning2
- Detecting Insults in Social Commentary
- EMI Music Data Science Hackathon
- Galaxy Zoo – The Galaxy Challenge
- Global Energy Forecasting Competition 2012 – Wind Forecasting
- KDD Cup 2013 – Author-Paper Identification Challenge (Track 1)2
- KDD Cup 2013 – Author Disambiguation Challenge (Track 2)1
- Large Scale Hierarchical Text Classification4
- Loan Default Prediction – Imperial College London
- Merck Molecular Activity Challenge1
- MLSP 2013 Bird Classification Challenge
- Observing the Dark World
- PAKDD 2014 – ASUS Malfunctional Components Prediction
- Personalize Expedia Hotel Searches – ICDM 2013
- Predicting a Biological Response1
- Predicting Closed Questions on Stack Overflow
- See Click Predict Fix1
- See Click Predict Fix – Hackathon1
- StumbleUpon Evergreen Classification Challenge
- [The Analytics Edge (15.071x)](The%20Analytics Edge (15.071x))
- The Marinexplore and Cornell University Whale Detection Challenge
- Walmart Recruiting – Store Sales Forecasting1
Kaggle比赛源代码和讨论的收集整理。
Algorithmic Trading Challenge25
Allstate Purchase Prediction Challenge3
- Rank 2 solution code21 by Alessandro Mariani.
- Rank 10 solution code5 by B1aine.
- Rank 36 solution cod1e by Hiroyuki.
- Rank 159 solution code by MrCanard.
- Solution thread.
Amazon.com – Employee Access Challenge6
- Rank 1 solution code24 by Paul Duan and Benjamin Solecki team.
- Rank 1 solution Q&A5 by Paul Duan.
- Rank 2 solution code1 by Owen Zhang.
- Rank 3 solution code3 by Dmitry & Leustagos.
- Rank 289 solution code by Foxtrot with original blog post here.
- Solution thread.
AMS 2013-2014 Solar Energy Prediction Contest2
- Rank 1 solution code5 and description4 by Leustagos team.
- Rank 2 solution code and description by Toulouse.
- Rank 3 solution code1 and description by Owen Zhang.
- Rank 4 solution escription by Peter Prettenhofer.
- Rank 5 solution description by Domcastro.
- Rank 58 solution code and description by Davit.
- Solution thread here.
- Ridge Regression starter code with MAE about 2.2M by Alec Radford, original thread here.
- Improved starter code by Foxtrot.
- Baseline code with MAE about 2.6M using Catmull-Rom Spline interpolation, also available in R here andhere.
Belkin Energy Disaggregation Competition1
Challenges in Representation Learning: Facial Expression Recognition Challenge4
- Rank 1 solution code6 and description2 by Charlie Tang.
- Rank 3 solution description3 by Maxim Milakov.
- Solution thread.
Challenges in Representation Learning: The Black Box Learning Challenge1
- Rank 1 solution description1 by David Thaler.
- Rank 2 solution code and description by sayit.
Challenges in Representation Learning: Multi-modal Learning2
- Rank 1 solution1 by MMDL.
- Solution thread.
Detecting Insults in Social Commentary
- Rank 1 solution description4 and code by Vivek Sharma.
- Rank 2 solution1 by tuzzeg.
- Rank 3 solution description Andrei Olariu.
- Rank 4 solution by Chris Brew.
- Rank 5 solution description by Yasser Tabandeh.
- Rank 6 solution by Andreas Mueller, code available here.
- Rank 8 solution description by Steve Poulson.
- Solution thread.
EMI Music Data Science Hackathon
- Rank 4 solution description1 by Steffen Rindle.
- Rank 18 solution code and description by Vlad Gusev.
- Rank 34 solution code and description by zenog.
- Solution thread.
Galaxy Zoo – The Galaxy Challenge
- Rank 1 solution code2 and description1 by Sander Dieleman.
- Rank 2 solution code and description by Maxim Milakov.
- Rank 3 solution code and description by tund.
- Rank 5 solution code and description by Julian de Wit.
- Rank 9 solution code and description by Soumith Chintala.
- Rank 13 solution code and description by Xiaoxiang Zhang.
- Rank 28 solution code and description by utdiscant.
- Rank 38 solution code and description by sugi.
- Rank 57 solution code and description1 by hxu.
- Rank 58 solution code and description by yr.
- Solution thread.
Global Energy Forecasting Competition 2012 – Wind Forecasting
- Rank 1 solution by Leustagos.
- Solution thread here1.
KDD Cup 2013 – Author-Paper Identification Challenge (Track 1)2
- Rank 1 solution with code and description4 by Team Algorithm, Github link to code here1.
KDD Cup 2013 – Author Disambiguation Challenge (Track 2)1
- Rank 1 solution with code and description4 by Team Algorithm, Github link to code here1.
- Rank 2 solution1 by SmallData Team.
- Rank 3 solution1 by hustmonk.
- Rank 4 solution1 by Ben S.
- Solution thread1.
Large Scale Hierarchical Text Classification4
- Rank 1 solution code and description7 by anttip.
- Rank 3 solution code2 and description2 by nagadomi.
- Solution thread one3.
- Solution thread two2.
Loan Default Prediction – Imperial College London
- Rank 2 solution and description1 by HelloWorld.
- Rank 12 solution and description by David McGarry.
- Solution thread.
Merck Molecular Activity Challenge1
MLSP 2013 Bird Classification Challenge
- Rank 1 solution code3 and description by beluga.
- Rank 2 solution code1 and description by Herbal Candy (W and thomeou).
- Rank 3 solution description by Anil Thomas.
- Rank 4 solution description by Maxim Milakov.
- Solution thread.
Observing the Dark World
- Rank 2 solution by Iain Murray, code available here.
PAKDD 2014 – ASUS Malfunctional Components Prediction
Personalize Expedia Hotel Searches – ICDM 2013
- Presentation paper/slides1 for ICDM 2013.
- Solution thread1.
Predicting a Biological Response1
- Rank 6 solution by Shea Parkes & Neil Schneider team.
- Rank 17 solution of Ensemble of RandomForests, GradientBoostingTrees and ExtraTreesRegressorby Emanuele Olivetti.
- Another solution code by Oblique Random Forest (oRF) by Shea Parkes & Neil Schneider team.
- The code of my best submission thread. Talks about Multi-core training Oblique Random Forests, and Stacking.
- Question about the process of ensemble learning thread. Talks about applying ensembles in practice, and how can problems arise and how to deal with them.
Predicting Closed Questions on Stack Overflow
- Rank 10 solution by Marco Lui.
- Rank 33 solution by Foxtrot.
See Click Predict Fix1
See Click Predict Fix – Hackathon1
StumbleUpon Evergreen Classification Challenge
- Benchmark beater 1.
- Benchmark beater 2.
- Benchmark beater 3.
- Solution thread.
- My own solution, which is a good example of what is overfitting. (Public rank: 57, Private rank: 291)
[The Analytics Edge (15.071x)](The%20Analytics Edge (15.071x))
- Rank 17 solution code and description by Foxtrot.
- Solution thread.
The Marinexplore and Cornell University Whale Detection Challenge
- Rank 1 solution by Nick Kridler.
- Rank 7 solution by Gilles Louppe and Peter Prettenhofer team.
- Rank 8 solution by Sander Dieleman.
- Rank 56 solution by Sudeep Juvekar.
- Solution discussion thread.
- Mean spectogram thread.
- Official interview from the Marinexplorer and Cornell at Kaggle.
Walmart Recruiting – Store Sales Forecasting1
- Rank 1 solution code5 and description by David Thaler.
- Rank 2 solution description1 by sriok.
- Rank 3 solution code and description1 by James King.
- Rank 5 solution description by ACS69.
- Rank 6 solution description by T. Henry.
- Rank 8 solution description by BreakfastPirate.
- Rank 9 solution description by Neil Summers.
- Rank 10 solution description by Gilberto Titericz Junior.
- Rank 11 solution description by citynight.
- Rank 16 solution code and description by yr.
- Rank 29 solution code and description by Mike Kim.
- Rank 30 solution description by dkay.
- Solution thread.
Thank you Foxtrot, James Petterson, Ben S for providing some of the links and solutions above.
[PyData] 01 - Web Crawler的更多相关文章
- A web crawler design for data mining
Abstract The content of the web has increasingly become a focus for academic research. Computer prog ...
- [CareerCup] 10.5 Web Crawler 网络爬虫
10.5 If you were designing a web crawler, how would you avoid getting into infinite loops? 这道题问如果让我们 ...
- (92) Web Crawling: How can I build a web crawler from scratch? - Quora
(92) Web Crawling: How can I build a web crawler from scratch? - Quora How can I build a web crawler ...
- <Web Crawler><Java><thread-safe queue>
Basic Solution The simplest way is to build a web crawler that runs on a single machine with single ...
- Free web scraping | Data extraction | Web Crawler | Octoparse, Free web scraping
Free web scraping | Data extraction | Web Crawler | Octoparse, Free web scraping 人才知了
- 01.Web大前端时代之：HTML5+CSS3入门系列~初识HTML5
Web大前端时代之:HTML5+CSS3入门系列:http://www.cnblogs.com/dunitian/p/5121725.html 文档申明 <!--文档类型申明,html代表是ht ...
- Tomcat笔记 #01# WEB应用管理工具简介
索引 查看JVM以及SERVLET/接口的情况 动态管理WEB应用 Tomcat自带了一个基于网页的web应用管理工具,可以帮助我们监控&管理部署上去的WEB APP,特别方便!恰好之前碰到的 ...
- 【Web crawler】simulated DFS web crawler
Finish crawl web learned from udacity 提示:在某些时候,你必须在page上调用get_page.这似乎违反直觉,但是我们用 page 这个词时,指的网页的网址 ( ...
- 01 Web框架介绍
一.Web框架本质 所有的web应用程序本质上都是socket,用户的浏览器其实就是一个socket客户端. python中常用的web框架有: Django Flask web.py WSGI(we ...
随机推荐
- C++调用matlab函数
C++与matlab混合编程——C++调用MATLAB函数 笔者最近在从事一个MFC相关的项目,要求将用Matlab实现的算法通过应用MFC制作成一个小应用.其中有一部分内容需要求一个多元函数的最值. ...
- python 安装pip setuptools
注意操作前提一定要使用管理员方式运行 python目录要完全允许控制 windows7 下 0.先安装python2.7.13 32位:https://www.python.org/ftp/pytho ...
- ArcGIS10.6的新功能
ArcMap 10.6 中引入了新的要素和功能,下面的章节将针对这些内容进行介绍. 要查看有关新特性的最新信息,请参阅 ArcMap web 帮助中的相关主题. 地理处理 3D Analyst 工具箱 ...
- CAP：Alantany 谈 CAP
引用Alantany的话:“CAP理论提出就是针对分布式数据库环境的,所以,P这个属性是必须具备的.P就是在分布式环境中,由于网络的问题可能导致某个节点和其它节点失去联系,这时候就形成了P(parti ...
- tex---就是tex文件，这个地球人都知道，是文章所在的主要文件
本demo中包括两个文件example.tex和myref.bib,用的时候參照改即可啦. example.tex:用的时候将'\'和后面文字中间的空格去掉 ----------------- \do ...
- springboot自定义http反馈状态码
最近在开发一些http server类型程序,通过spring boot构建一些web程序,这些web程序之间通过http进行数据访问.共享,如下图: 假设现在client发起一次保存数据的请求到se ...
- Android APK 打包过程 MD
Markdown版本笔记 我的GitHub首页 我的博客 我的微信 我的邮箱 MyAndroidBlogs baiqiantao baiqiantao bqt20094 baiqiantao@sina ...
- [Canvas]飞机飞越河谷（背景向下平滑移动）
请点击此处下载代码并用Chrome浏览器打开观看. 图例: 代码: <!DOCTYPE html> <html lang="utf-8"> <meta ...
- webservice接口测试，使用SoapUI工具进行接口测试
首先,接口使用cxf编写接口,测试工具使用SoapUI 5.2.1 安装之后是这样的图标: 测试操作步骤如下: (1)首先找到cxf-webservice.xml配置信息中地址,在浏览器中出入:htt ...
- linux实现共享内存同步的四种方法
https://blog.csdn.net/sunxiaopengsun/article/details/79869115 本文主要对实现共享内存同步的四种方法进行了介绍. 共享内存是一种最为高效的进 ...