超参数调整

详细可以参考官方文档

定义

在拟合模型之前需要定义好的参数

适用

  • Linear regression: Choosing parameters
  • Ridge/lasso regression: Choosing alpha
  • k-Nearest Neighbors: Choosing n_neighbors
  • Parameters like alpha and k: Hyperparameters
  • Hyperparameters cannot be learned by tting the model

GridsearchCV

sklearn.model_selection.GridSearchCV

  • 超参数自动搜索模块
  • 网格搜索+交叉验证
  • 指定的参数范围内,按步长依次调整参数,利用调整的参数训练学习器,从所有的参数中找到在验证集上精度最高的参数,这其实是一个训练和比较的过程
class sklearn.model_selection.GridSearchCV(estimator, param_grid, scoring=None, n_jobs=None, iid='deprecated', refit=True, cv=None, verbose=0, pre_dispatch='2*n_jobs', error_score=nan, return_train_score=False

参数

  • estimator:模型对象
  • param_grid:dict or list of dictionaries,字典类型的参数,定义一个字典然后都放进去
  • scoring:string, callable, list/tuple, dict or None, default: None,就是metrics,损失函数定义rmse,mse等
  • Number of jobs to run in parallel. None means 1 unless in a joblib.parallel_backend context. -1 means using all processors. See Glossary for more details.控制cop,core并行运行数量

    -cv:int, cross-validation generator or an iterable, optional,k折交叉验证数,默认5折

    • Determines the cross-validation splitting strategy. Possible inputs for cv are:
    • None, to use the default 5-fold cross validation,integer, to specify the number of folds in a (Stratified)KFold,CV splitter,
    • An iterable yielding (train, test) splits as arrays of indices.
    • For integer/None inputs, if the estimator is a classifier and y is either binary or multiclass, StratifiedKFold is used. In all other cases, KFold is used.
  • verbose:控制输出信息的详细程度,愈高输出越多。

属性

常见:

  • cv_results_dict of numpy (masked) ndarrays输出交叉验证的每一个结果
  • best_estimator_:最好的估计器
  • best_params_:dict
    • 返回最优模型参数
    • Parameter setting that gave the best results on the hold out data.
    • For multi-metric evaluation, this is present only if refit is specified.
  • best_score_:float
    • 返回最优模型参数的得分
    • Mean cross-validated score of the best_estimator
    • For multi-metric evaluation, this is present only if refit is specified.
    • This attribute is not available if refit is a function.

复现

# Import necessary modules
from sklearn.model_selection import GridSearchCV from sklearn.linear_model import LogisticRegression
# Setup the hyperparameter grid
# 创建一个参数集
c_space = np.logspace(-5, 8, 15)
# 这里是创建一个字典保存参数集
param_grid = {'C': c_space} # Instantiate a logistic regression classifier: logreg
# 针对回归模型进行的超参数调整
logreg = LogisticRegression() # Instantiate the GridSearchCV object: logreg_cv
logreg_cv = GridSearchCV(logreg, param_grid, cv=5) # Fit it to the data
logreg_cv.fit(X,y) # Print the tuned parameters and score
# 得到最好的模型
print("Tuned Logistic Regression Parameters: {}".format(logreg_cv.best_params_))
# 得到最好的模型的最好的结果
print("Best score is {}".format(logreg_cv.best_score_)) <script.py> output:
Tuned Logistic Regression Parameters: {'C': 3.727593720314938}
Best score is 0.7708333333333334

GridSearchCV can be computationally expensive, especially if you are searching over a large hyperparameter space and dealing with multiple hyperparameters. A solution to this is to use RandomizedSearchCV, in which not all hyperparameter values are tried out. Instead, a fixed number of hyperparameter settings is sampled from specified probability distributions.

grid相当于一个for循环,会遍历每一个参数,因此,当调参很多的时候,会导致计算量非常的大,因此,使用随机抽样的随机搜索会好一些

RandomizedSearchCV的使用方法其实是和GridSearchCV一致的,但它以随机在参数空间中采样的方式代替了GridSearchCV对于参数的网格搜索,在对于有连续变量的参数时,RandomizedSearchCV会将其当作一个分布进行采样这是网格搜索做不到的,它的搜索能力取决于设定的n_iter参数,同样的给出代码

csdn

RandomizedSearchCV

  • 随机搜索法
  • 不是每一个参数都被选取,而是从指定概率分布的参数中,抽取一定量的参数

    我还是没太能明白?

    可以比较一下时间

比较网格搜索而言,参数略有不同

算了,还是都列一下常见的吧,剩下的可以参照官方文档

比Grid 多了一个属性

  • .cv_results_,可以交叉验证的每一轮的结果

复现

# Import necessary modules
from scipy.stats import randint
from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import RandomizedSearchCV # Setup the parameters and distributions to sample from: param_dist
# 以决策树为例,注意定一个字典的形式哦
param_dist = {"max_depth": [3, None],
"max_features": randint(1, 9),
"min_samples_leaf": randint(1, 9),
"criterion": ["gini", "entropy"]} # Instantiate a Decision Tree classifier: tree
tree = DecisionTreeClassifier() # Instantiate the RandomizedSearchCV object: tree_cv
tree_cv = RandomizedSearchCV(tree, param_dist, cv=5) # Fit it to the data
tree_cv.fit(X,y) # Print the tuned parameters and score
print("Tuned Decision Tree Parameters: {}".format(tree_cv.best_params_))
print("Best score is {}".format(tree_cv.best_score_)) <script.py> output:
Tuned Decision Tree Parameters: {'criterion': 'gini', 'max_depth': 3, 'max_features': 5, 'min_samples_leaf': 2}
Best score is 0.7395833333333334

Limits of grid search and random search

调参的限制点

  • grid:

    -random:

Hyperparameter tuning的更多相关文章

  1. 论文笔记系列-Multi-Fidelity Automatic Hyper-Parameter Tuning via Transfer Series Expansion

    论文: Multi-Fidelity Automatic Hyper-Parameter Tuning via Transfer Series Expansion 我们都知道实现AutoML的基本思路 ...

  2. How to Evaluate Machine Learning Models, Part 4: Hyperparameter Tuning

    How to Evaluate Machine Learning Models, Part 4: Hyperparameter Tuning In the realm of machine learn ...

  3. [C2W3] Improving Deep Neural Networks : Hyperparameter tuning, Batch Normalization and Programming Frameworks

    第三周:Hyperparameter tuning, Batch Normalization and Programming Frameworks 调试处理(Tuning process) 目前为止, ...

  4. [C4] Andrew Ng - Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization

    About this Course This course will teach you the "magic" of getting deep learning to work ...

  5. 《Improving Deep Neural Networks:Hyperparameter tuning, Regularization and Optimization》课堂笔记

    Lesson 2 Improving Deep Neural Networks:Hyperparameter tuning, Regularization and Optimization 这篇文章其 ...

  6. Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week3, Hyperparameter tuning, Batch Normalization and Programming Frameworks

    Tuning process 下图中的需要tune的parameter的先后顺序, 红色>黄色>紫色,其他基本不会tune. 先讲到怎么选hyperparameter, 需要随机选取(sa ...

  7. Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week2, Assignment(Optimization Methods)

    声明:所有内容来自coursera,作为个人学习笔记记录在这里. 请不要ctrl+c/ctrl+v作业. Optimization Methods Until now, you've always u ...

  8. 课程二(Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization),第二周(Optimization algorithms) —— 2.Programming assignments:Optimization

    Optimization Welcome to the optimization's programming assignment of the hyper-parameters tuning spe ...

  9. Coursera Deep Learning 2 Improving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimization - week1, Assignment(Regularization)

    声明:所有内容来自coursera,作为个人学习笔记记录在这里. Regularization Welcome to the second assignment of this week. Deep ...

随机推荐

  1. C++ 日期 & 时间(转)

    C++ 标准库没有提供所谓的日期类型.C++ 继承了 C 语言用于日期和时间操作的结构和函数. 为了使用日期和时间相关的函数和结构,需要在 C++ 程序中引用 头文件.有四个与时间相关的类型:cloc ...

  2. 编译原理实验之SLR1文法分析

    ---内容开始--- 这是一份编译原理实验报告,分析表是手动造的,可以作为借鉴. 基于  SLR(1) 分析法的语法制导翻译及中间代码生成程序设计原理与实现1 .理论传授语法制导的基本概念,目标代码结 ...

  3. mysql 支持emoji表情

    在mysql插入emoji表情,出现错误: java.sql.SQLException: Incorrect string value: '\xF0\x9F\x98\x8A' for column ' ...

  4. (转载)Linux平台下安装 python 模块包

    https://blog.csdn.net/aiwangtingyun/article/details/79121145 一.安装Python Windows平台下: 进入Python官网下载页面下载 ...

  5. lwip 2.0.2 snmp mib ipv6

    1.3.6.1.2.1 - SNMP MIB-2 Submitted by Harald.T.Alvestrand at uninett.no from host aun.uninett.no (12 ...

  6. Webpack之optimization.splitChunks代码分割插件的配置

    SplitChunkPlugin插件配置参数详解 对引入的库代码(例如:lodash.jQuery等)进行代码的分割进行优化 若配置时只写chunks:"all",其余则为默认配置 ...

  7. SignalR—实例

    本例子将讲述使用SignalR实现页面实时显示数据,而不是需要用户刷新页面或使用Ajax轮询才能实现实时显示数据. 1.使用NuGet安装SignalR. 2.创建Hub类,起名为testHub. 3 ...

  8. Django 系列

    Django基础 Django框架简介 Django 静态文件 Django request对象与ORM简介 Django路由系统 Django之视图层 Django之模板层 Django的setti ...

  9. 【Go语言系列】2.3、Go语言基本程序结构:变量及常量

    1.什么变量 变量来源于数学,从根本上说,变量相当于是对一块数据存储空间的命名,程序可以通过定义一个变量来申请一块数据存储空间,之后可以通过引用变量名来使用这块存储空间. 1.1变量声明 Go 语言变 ...

  10. 剑指offer-面试题63-股票的最大利润-数组

    /* 题目: 给定一个股价序列,求一次交易的最大利润. */ #include<iostream> #include<vector> using namespace std; ...