Lessons learned developing a practical large scale machine learning system
原文:http://googleresearch.blogspot.jp/2010/04/lessons-learned-developing-practical.html
Lessons learned developing a practical large scale machine learning system
When faced with a hard prediction problem, one possible approach is to attempt to perform statistical miracles on a small training set. If data is abundant then often a more fruitful approach is to design a highly scalable learning system and use several orders of magnitude more training data.
This general notion recurs in many other fields as well. For example, processing large quantities of data helps immensely for information retrieval and machine translation.
Several years ago we began developing a large scale machine learning system, and have been refining it over time. We gave it the codename “Seti” because it searches for signals in a large space. It scales to massive data sets and has become one of the most broadly used classification systems at Google.
After building a few initial prototypes, we quickly settled on a system with the following properties:
- Binary classification (produces a probability estimate of the class label)
- Parallelized
- Scales to process hundreds of billions of instances and beyond
- Scales to billions of features and beyond
- Automatically identifies useful combinations of features
- Accuracy is competitive with state-of-the-art classifiers
- Reacts to new data within minutes
Seti’s accuracy appears to be pretty decent. For example, tests on standard smaller datasets indicate that it is comparable with modern classifiers.
Seti has the flexibility to be used on a broad range of training set sizes and feature sets. These sizes are substantially larger than those typically used in academia (e.g., the largest UCI datasethas 4 million instances). A sample of the data sets used with Seti gives the following statistics:
Training set size | Unique features | |
Mean | 100 Billion | 1 Billion |
Median | 1 Billion | 10 Million |
A good machine learning system is all about accuracy, right?
In the process of designing Seti we made plenty of mistakes. However, we made some good key decisions as well. Here are a few of the practical lessons that we learned. Some are obvious in hindsight, but we did not necessarily realize their importance at the time.
Lesson: Keep it simple (even at the expense of a little accuracy).
- Ease of use: Teams are more willing to experiment with a machine learning system that is simple to set up and use. Those teams are not necessarily die-hard machine learning experts, and so they do not want to waste much time figuring out how to get a system up and running.
- System reliability: Teams are much more willing to deploy a reliable machine learning system in a live environment. They want a system that is dependable and unlikely to crash or need constant attention. Early versions of Seti had marginally better accuracy on large data sets, but were complex, stressed the network and GFS architecture considerably, and needed constant babysitting. The number of teams willing to deploy these versions was low.
Seti is typically used in places where a machine learning system will provide a significant improvement in accuracy over the existing system. The gains are usually large enough that most teams do not care about the small differences in accuracy between different flavors of algorithms. And, in practice, the small differences are often washed out by other effects such as better data filtering, adding another useful feature, parameter tuning, etc. Teams much prefer having a stable, scalable and easy-to-use classification system. We found that these other aspects can be the difference between a deployable system and one that gets abandoned.
It is perhaps less academically interesting to design an algorithm that is slightly worse in accuracy, but that has greater ease of use and system reliability. However, in our experience, it is very valuable in practice.
Lesson: Start with a few specific applications in mind.
- We could examine what the small number of domains had in common. By building something that would work for a few domains, it was likely the resulting system would be useful for others.
- More importantly, it helped us quickly decide what aspects were unnecessary. We noticed that it was surprisingly easy to over-generalize or over-engineer a machine learning system. The domains grounded our project in reality and drove our decision making. Without them, even deciding how broad to make the input file format would have been harder (e.g., is it important to permit binary/categorical/real-valued features? Multiple classes? Fractional labels? Weighted instances?).
- Working with a few different teams as initial guinea pigs allowed us to learn about common teething problems, and helped us smooth the process of deployment for future teams.
Lesson: Know when to say “no”.
Seti is often used in places where there is a good chance of significantly improving predictive accuracy over the incumbent system. And we usually advise teams against trying the system when we believe there is likely to be only a small improvement.
Large-scale machine learning is an important and exciting area of research. It can be applied to many real world problems. We hope that we have given a flavor of the challenges that we face, and some of the practical lessons that we have learned.
Lessons learned developing a practical large scale machine learning system的更多相关文章
- 【原】Coursera—Andrew Ng机器学习—课程笔记 Lecture 17—Large Scale Machine Learning 大规模机器学习
Lecture17 Large Scale Machine Learning大规模机器学习 17.1 大型数据集的学习 Learning With Large Datasets 如果有一个低方差的模型 ...
- [C12] 大规模机器学习(Large Scale Machine Learning)
大规模机器学习(Large Scale Machine Learning) 大型数据集的学习(Learning With Large Datasets) 如果你回顾一下最近5年或10年的机器学习历史. ...
- 大规模机器学习(Large Scale Machine Learning)
本博客是针对Andrew Ng在Coursera上的machine learning课程的学习笔记. 目录 在大数据集上进行学习(Learning with Large Data Sets) 随机梯度 ...
- (原创)Stanford Machine Learning (by Andrew NG) --- (week 10) Large Scale Machine Learning & Application Example
本栏目来源于Andrew NG老师讲解的Machine Learning课程,主要介绍大规模机器学习以及其应用.包括随机梯度下降法.维批量梯度下降法.梯度下降法的收敛.在线学习.map reduce以 ...
- 斯坦福第十七课:大规模机器学习(Large Scale Machine Learning)
17.1 大型数据集的学习 17.2 随机梯度下降法 17.3 微型批量梯度下降 17.4 随机梯度下降收敛 17.5 在线学习 17.6 映射化简和数据并行 17.1 大型数据集的学习
- 吴恩达机器学习笔记60-大规模机器学习(Large Scale Machine Learning)
一.随机梯度下降算法 之前了解的梯度下降是指批量梯度下降:如果我们一定需要一个大规模的训练集,我们可以尝试使用随机梯度下降法(SGD)来代替批量梯度下降法. 在随机梯度下降法中,我们定义代价函数为一个 ...
- Ng第十七课:大规模机器学习(Large Scale Machine Learning)
17.1 大型数据集的学习 17.2 随机梯度下降法 17.3 微型批量梯度下降 17.4 随机梯度下降收敛 17.5 在线学习 17.6 映射化简和数据并行 17.1 大型数据集的学习 ...
- Coursera在线学习---第十节.大规模机器学习(Large Scale Machine Learning)
一.如何学习大规模数据集? 在训练样本集很大的情况下,我们可以先取一小部分样本学习模型,比如m=1000,然后画出对应的学习曲线.如果根据学习曲线发现模型属于高偏差,则应在现有样本上继续调整模型,具体 ...
- 吴恩达机器学习笔记(十一) —— Large Scale Machine Learning
主要内容: 一.Batch gradient descent 二.Stochastic gradient descent 三.Mini-batch gradient descent 四.Online ...
随机推荐
- [java][jboss]改变jboss部署目录(虚拟目录)
原文: [java][jboss]改变jboss部署目录(虚拟目录) 在开发过程中,有时候我们希望将程序放在我们的源代码目录中,比如d:\code下,而不是放在jboss的deploy下,怎么办? 我 ...
- BZOJ 4554: [Tjoi2016&Heoi2016]游戏 二分图匹配
4554: [Tjoi2016&Heoi2016]游戏 题目连接: http://www.lydsy.com/JudgeOnline/problem.php?id=4554 Descripti ...
- HDU 3282 Running Median 动态中位数,可惜数据范围太小
Running Median Time Limit: 1 Sec Memory Limit: 256 MB 题目连接 http://acm.hdu.edu.cn/showproblem.php?pi ...
- mysql+mycat分片环境部署
说明: 1.操作系统:64位CentOS Linux release 7.2.1511 (Core) 2.jdk版本:1.8.0_121 3.mysql版本: 5.7.17 4.两台mysql服务器: ...
- Java_基础知识回顾
1.ByteArrayInputStream. ByteArrayOutputStream String str = "ZHANGSAN"; //System.out.printl ...
- tomcat部署应用仅需ip和port访问
一.使用ip和port访问应用项目: 打开tomcat安装根目录,打开conf目录下server.xml,找到<Host>节点,并且在该节点下新增: <Context docBa ...
- 理解linux下源码、yum和rpm安装方法的特点
1.yum可看作在线安装,只需yum install 软件名,系统就自动根据yum源配置文件中的镜像位置去下载安装包,并可以自动分析所需的软件依赖关系,自动安装所需的依赖软件包.简单方便,不易出错,不 ...
- mysql 3.2.49 源代码安装-redhat 5 x64
[mysql@localhost ~]$ uname -r2.6.32 [root@localhost ~]#cp /usr/include/pthread.h /usr/include/pthrea ...
- Android native层动态库注射
1.简单介绍 本文解说在Android native层.root权限下.注射动态库到目标进程,从而hook目标进程中动态库的函数的实现方式. 文中的源代码所有来源于网络.我仅仅是略微加以整理. 环境: ...
- 模拟出ios中流行的黑色背景底
[activityIndicatorView setBackgroundColor:[UIColor colorWithRed: green: blue: alpha:0.4]]; 用上面的语法,可以 ...