參考:http://scikit-learn.org/stable/modules/grid_search.html

GridSearchCV通过(蛮力)搜索參数空间(參数的全部可能组合)。寻找最好的 Cross-validation:
evaluating estimator performance
 score相应的超參数(翻译文章參考:http://blog.csdn.net/mmc2015/article/details/47099275)。比如Support
Vector Classifier的 Ckernel and gamma ,Lasso的alpha。etc。

A search consists of:

  • an estimator (regressor or classifier such as sklearn.svm.SVC());
  • a parameter space;
  • a method for searching or sampling candidates;
  • a cross-validation scheme
  • score
    function
    .

RandomizedSearchCV 通过一定的分布sample候选參数。而不是搜索全部參数组合。

本节我们介绍 GridSearchCVRandomizedSearchCV 、以及parameter
search的小Tips,最后介绍蛮力搜索的alternatives

1、Exhaustive
Grid Search

GridSearchCV的參数param_grid定义搜索网格。

两个样例说明一切:

2、Randomized
Parameter Optimization

RandomizedSearchCV 通过在參数可能的取值的某个分布中sample一组參数。优点是:能够设定独立于參数(及全部取值)详细数量的一个搜索次数;加入无效的參数也不会减少效率。

搜索的次数通过 n_iter 设定,对于每个參数,假设是连续的取值。则通过一定的分布sample,假设是离散的取值,则通过uniform分布sample,比如:

[{'C': scipy.stats.expon(scale=100), 'gamma': scipy.stats.expon(scale=.1),
'kernel': ['rbf'], 'class_weight':['auto', None]}]

scipy.stats module提供了非常多用来sample參数的distributions,如expongammauniform or randint.

对于连续的參数,如 C ,一定要选择连续的分布来sample,而且适当增大 n_iter 通常会搜索到更好的參数组合。

给个样例:

3、Tips
for parameter search(这几个建议很靠谱。。。

1)详细化目标函数

參数搜索默认使用score function(
即,分类用sklearn.metrics.accuracy_score 回归用sklearn.metrics.r2_score )来衡量參数的好坏对于有些应用(比方分类unbalance,score不是非常好的标准),通过详细化GridSearchCVRandomizedSearchCV 的scoring parameter。See The
scoring parameter: defining model evaluation rules
 for more details.

2)综合estimators和parameter sapces(同一时候考虑预測器和參数空间)

Pipeline:
chaining estimators
 describes building composite estimators whose parameter space can be searched with these tools.

3)模型选择:先训练、再评估

用训练集选择模型。用測试集验证模型(using
the cross_validation.train_test_split utility
function.
)(it is recommended to split the data into a development set (to be
fed to the GridSearchCV instance)
and an evaluation set to compute performance metrics.)

4)并行搜索

n_jobs=-1.
自己主动使用全部核。

5)robustness to failure(增强搜索错误的鲁棒性)

有些參数组合对于某些folds
of the data会failure,进而导致整个search failure,虽然其它的參数组合没有问题。

设定 error_score=0 (or =np.NaN)
能够使search过程忽略这种failure,只抛出一个warning,并将这种search结果设为0 (or =np.NaN)
,可以提高搜索遇到错误时的鲁棒性!

4、Alternatives
to brute force parameter search(没太看懂,还是不翻译了)

3.2.4.1. Model specific cross-validation

Some models can fit data for a range of value of some parameter almost as efficiently as fitting the estimator for a single value of the parameter. This feature can be leveraged to perform
a more efficient cross-validation used for model selection of this parameter.

The most common parameter amenable to this strategy is the parameter encoding the strength of the regularizer. In this case we say that we compute theregularization path of
the estimator.

Here is the list of such models:

linear_model.ElasticNetCV([l1_ratio, eps, ...]) Elastic Net model with iterative fitting along a regularization path
linear_model.LarsCV([fit_intercept, ...]) Cross-validated Least Angle Regression model
linear_model.LassoCV([eps, n_alphas, ...]) Lasso linear model with iterative fitting along a regularization path
linear_model.LassoLarsCV([fit_intercept, ...]) Cross-validated Lasso, using the LARS algorithm
linear_model.LogisticRegressionCV([Cs, ...]) Logistic Regression CV (aka logit, MaxEnt) classifier.
linear_model.MultiTaskElasticNetCV([...]) Multi-task L1/L2 ElasticNet with built-in cross-validation.
linear_model.MultiTaskLassoCV([eps, ...]) Multi-task L1/L2 Lasso with built-in cross-validation.
linear_model.OrthogonalMatchingPursuitCV([...]) Cross-validated Orthogonal Matching Pursuit model (OMP)
linear_model.RidgeCV([alphas, ...]) Ridge regression with built-in cross-validation.
linear_model.RidgeClassifierCV([alphas, ...]) Ridge classifier with built-in cross-validation.

3.2.4.2. Information Criterion

Some models can offer an information-theoretic closed-form formula of the optimal estimate of the regularization parameter by computing a single regularization path (instead of several when
using cross-validation).

Here is the list of models benefitting from the Aikike Information Criterion (AIC) or the Bayesian Information Criterion (BIC) for automated model selection:

linear_model.LassoLarsIC([criterion, ...]) Lasso model fit with Lars using BIC or AIC for model selection

3.2.4.3. Out of Bag Estimates

When using ensemble methods base upon bagging, i.e. generating new training sets using sampling with replacement, part of the training set remains unused. For each classifier in the ensemble,
a different part of the training set is left out.

This left out portion can be used to estimate the generalization error without having to rely on a separate validation set. This estimate comes “for free” as no additional data is needed and
can be used for model selection.

This is currently implemented in the following classes:

ensemble.RandomForestClassifier([...]) A random forest classifier.
ensemble.RandomForestRegressor([...]) A random forest regressor.
ensemble.ExtraTreesClassifier([...]) An extra-trees classifier.
ensemble.ExtraTreesRegressor([n_estimators, ...]) An extra-trees regressor.
ensemble.GradientBoostingClassifier([loss, ...]) Gradient Boosting for classification.
ensemble.GradientBoostingRegressor([loss, ...]) Gradient Boosting for regression.

scikit-learn:3.2. Grid Search: Searching for estimator parameters的更多相关文章

  1. 3.2. Grid Search: Searching for estimator parameters

    3.2. Grid Search: Searching for estimator parameters Parameters that are not directly learnt within ...

  2. How to Grid Search Hyperparameters for Deep Learning Models in Python With Keras

    Hyperparameter optimization is a big part of deep learning. The reason is that neural networks are n ...

  3. scikit learn 模块 调参 pipeline+girdsearch 数据举例:文档分类 (python代码)

    scikit learn 模块 调参 pipeline+girdsearch 数据举例:文档分类数据集 fetch_20newsgroups #-*- coding: UTF-8 -*- import ...

  4. (原创)(四)机器学习笔记之Scikit Learn的Logistic回归初探

    目录 5.3 使用LogisticRegressionCV进行正则化的 Logistic Regression 参数调优 一.Scikit Learn中有关logistics回归函数的介绍 1. 交叉 ...

  5. Grid search in the tidyverse

    @drsimonj here to share a tidyverse method of grid search for optimizing a model's hyperparameters. ...

  6. (原创)(三)机器学习笔记之Scikit Learn的线性回归模型初探

    一.Scikit Learn中使用estimator三部曲 1. 构造estimator 2. 训练模型:fit 3. 利用模型进行预测:predict 二.模型评价 模型训练好后,度量模型拟合效果的 ...

  7. Extjs4.2 Grid搜索Ext.ux.grid.feature.Searching的使用

    背景 Extjs4.2 默认提供的Search搜索,功能还是非常强大的,只是对于国内的用户来说,还是不习惯在每列里面单击好几下再筛选,于是相当当初2.2里面的搜索,更加的实用点,于是在4.2里面实现. ...

  8. Ext.ux.grid.feature.Searching 解析查询参数,动态产生linq lambda表达式

    上篇文章中http://www.cnblogs.com/qidian10/p/3209439.html我们介绍了如何使用Grid的查询组建,而且将查询的参数传递到了后台. 那么我们后台如何介绍参数,并 ...

  9. Grid Search学习

    转自:https://www.cnblogs.com/ysugyl/p/8711205.html Grid Search:一种调参手段:穷举搜索:在所有候选的参数选择中,通过循环遍历,尝试每一种可能性 ...

随机推荐

  1. AC自动机笔记

    AC自动机 #include<iostream> #include<cstring> #include<cstdio> #include<cmath> ...

  2. docker-ce-17.03.2 离线安装RPM包

    [root@docker05 docker]# ll total 20796 -rw-r--r-- 1 root root    75032 Mar 26 23:52 audit-libs-pytho ...

  3. C语言计算字符串数组中每个字符串出现的个数

    unsigned int str_num(char *str[], int num[], int len) { int i, j; int count; int flag[len]; ; i < ...

  4. C语言回调

    来源:https://www.cnblogs.com/jiangzhaowei/p/9129105.html 1. 什么是回调函数? 回调函数,光听名字就比普通函数要高大上一些,那到底什么是回调函数呢 ...

  5. JDK的安装及配置环境变量

    开发java程序的必备工具:JDK,全名是Java Development Kit, 是Java语言的软件开发工具包. 第一步:下载安装包 从Oracle官网可以选择自己所需的版本下载,(附Oracl ...

  6. m_Orchestrate learning system---二十四、thinkphp里面的ajax如何使用

    m_Orchestrate learning system---二十四.thinkphp里面的ajax如何使用 一.总结 一句话总结:其实ajax非常简单:前台要做的事情就是发送ajax请求过来,后台 ...

  7. WHERE、ORDER BY、GROUP BY、HAVING语句解析(二十八)

    之前啊,我们提及到,对于update和delete,若不带where条件,则对所有记录都有效. 一.WHERE条件表达式 (1)对记录进行过滤,如果没有指定WHERE子句,则显示所有记录. (2)在W ...

  8. java中"".equals(A)与A.equals("")一样不?

    不一样如果a为nulla = null;a.equals("")出错nullPointerException如果写为"".equals(a)-->就可以防 ...

  9. for循环练习-----ATM取款

    要求: 代码: package com.jianglai.atm; import java.util.Scanner; public class ATM { public static void ma ...

  10. CUDA学习笔记(五)

    终于实质分析线程的内容了:按照SIMD的方式,每32个线程称为一个线程束,这些线程都执行同一指令,且每个线程都使用私有寄存器进行这一操作请求. 忽然觉得,做CUDA的程序就像是去北京上班:写MPI之后 ...