The glmnetUtils package provides a collection of tools to streamline the process of fitting elastic net models with glmnet. I wrote the package after a couple of projects where I found myself writing the same boilerplate code to convert a data frame into a predictor matrix and a response vector. In addition to providing a formula interface, it also has a function (cvAlpha.glmnet) to do crossvalidation for both elastic net parameters α and λ, as well as some utility functions.

The formula interface

The interface that glmnetUtils provides is very much the same as for most modelling functions in R. To fit a model, you provide a formula and data frame. You can also provide any arguments that glmnet will accept. Here is a simple example:

mtcarsMod <- glmnet(mpg ~ cyl + disp + hp, data=mtcars)

## Call:
## glmnet.formula(formula = mpg ~ cyl + disp + hp, data = mtcars)
##
## Model fitting options:
## Sparse model matrix: FALSE
## Use model.frame: FALSE
## Alpha: 1
## Lambda summary:
## Min. 1st Qu. Median Mean 3rd Qu. Max.
## 0.03326 0.11690 0.41000 1.02800 1.44100 5.05500

Under the hood, glmnetUtils creates a model matrix and response vector, and passes them to the glmnet package to do the actual model fitting. Prediction also works as you'd expect: just pass a data frame containing the new observations, along with any arguments thatpredict.glmnet needs.

# least squares regression: get predictions for lambda=1
predict(mtcarsMod, newdata=mtcars, s=1)

Building the model matrix

You may have noticed the options "use model.frame" and "sparse model matrix" in the printed output above. glmnetUtils includes a couple of options to improve performance, especially on wide datasets and/or have many categorical (factor) variables.

The standard R method for creating a model matrix out of a data frame uses the model.framefunction, which has a major disadvantage when it comes to wide data. It generates a termsobject, which specifies how the original columns of data relate to the columns in the model matrix. This involves creating and storing a (roughly) square matrix of size p × p, where p is the number of variables in the model. When p > 10000, which isn't uncommon these days, the terms object can exceed a gigabyte in size. Even if there is enough memory to store the object, processing it can be very slow.

Another issue with the standard approach is the treatment of factors. Normally, model.matrixwill turn an N-level factor into an indicator matrix with N−1 columns, with one column being dropped. This is necessary for unregularised models as fit with lm and glm, since the full set of Ncolumns is linearly dependent. However, this may not be appropriate for a regularised model as fit with glmnet. The regularisation procedure shrinks the coefficients towards zero, which forces the estimated differences from the baseline to be smaller. But this only makes sense if the baseline level was chosen beforehand, or is otherwise meaningful as a default; otherwise it is effectively making the levels more similar to an arbitrarily chosen level.

To deal with these problems, glmnetUtils by default will avoid using model.frame, instead building up the model matrix term-by-term. This avoids the memory cost of creating a terms object, and can be much faster than the standard approach. It will also include one column in the model matrix for all levels in a factor; that is, no baseline level is assumed. In this situation, the coefficients represent differences from the overall mean response, and shrinking them to zero is meaningful (usually). Machine learners may also recognise this as one-hot encoding.

glmnetUtils can also generate a sparse model matrix, using the sparse.model.matrix function provided in the Matrix package. This works exactly the same as a regular model matrix, but takes up significantly less memory if many of its entries are zero. A scenario where this is the case would be where many of the predictors are factors, each with a large number of levels.

Crossvalidation for α

One piece missing from the standard glmnet package is a way of choosing α, the elastic net mixing parameter, similar to how cv.glmnet chooses λ, the shrinkage parameter. To fix this, glmnetUtils provides the cvAlpha.glmnet function, which uses crossvalidation to examine the impact on the model of changing α and λ. The interface is the same as for the other functions:

# Leukemia dataset from Trevor Hastie's website:
# http://web.stanford.edu/~hastie/glmnet/glmnetData/Leukemia.RData
load("~/Leukemia.rdata")
leuk <- do.call(data.frame, Leukemia) cvAlpha.glmnet(y ~ ., data=leuk, family="binomial") ## Call:
## cvAlpha.glmnet.formula(formula = y ~ ., data = leuk, family = "binomial")
##
## Model fitting options:
## Sparse model matrix: FALSE
## Use model.frame: FALSE
## Alpha values: 0 0.001 0.008 0.027 0.064 0.125 0.216 0.343 0.512 0.729 1
## Number of crossvalidation folds for lambda: 10

cvAlpha.glmnet uses the algorithm described in the help for cv.glmnet, which is to fix the distribution of observations across folds and then call cv.glmnet in a loop with different values of α. Optionally, you can parallelise this outer loop, by setting the outerParallel argument to a non-NULL value. Currently, glmnetUtils supports the following methods of parallelisation:

  • Via parLapply in the parallel package. To use this, set outerParallel to a valid cluster object created bymakeCluster.
  • Via rxExec as supplied by Microsoft R Server’s RevoScaleR package. To use this, setouterParallel to a valid compute context created by RxComputeContext, or a character string specifying such a context.

Conclusion

The glmnetUtils package is a way to improve quality of life for users of glmnet. As with many R packages, it’s always under development; you can get the latest version from my GitHub repo. The easiest way to install it is via devtools:

library(devtools)
install_github("hong-revo/glmnetUtils")

A more detailed version of this post can also be found at the package vignette. If you find a bug, or if you want to suggest improvements to the package, please feel free to contact me athongooi@microsoft.com.

转自:http://blog.revolutionanalytics.com/2016/11/glmnetutils.html

glmnetUtils: quality of life enhancements for elastic net regression with glmnet的更多相关文章

  1. wlan的QOS配置

    WLAN QoS配置 1.1  WLAN QoS简介 802.11网络提供了基于竞争的无线接入服务,但是不同的应用需求对于网络的要求是不同的,而原始的网络不能为不同的应用提供不同质量的接入服务,所以已 ...

  2. L1和L2特征的适用场景

    How to decide which regularization (L1 or L2) to use? Is there collinearity among some features? L2 ...

  3. Machine and Deep Learning with Python

    Machine and Deep Learning with Python Education Tutorials and courses Supervised learning superstiti ...

  4. Kaggle实战之一回归问题

    0. 前言 1.任务描述 2.数据概览 3. 数据准备 4. 模型训练 5. kaggle实战 0. 前言 "尽管新技术新算法层出不穷,但是掌握好基础算法就能解决手头 90% 的机器学习问题 ...

  5. Google云平台使用方法 | Hail | GWAS | 分布式回归 | LASSO

    参考: Hail Hail - Tutorial  windows也可以安装:Spark在Windows下的环境搭建 spark-2.2.0-bin-hadoop2.7 - Hail依赖的平台,并行处 ...

  6. Overfitting & Regularization

    Overfitting & Regularization The Problem of overfitting A common issue in machine learning or ma ...

  7. stacking method house price in kaggle top10%

    整合几部分代码的汇总 隐藏代码片段 导入python数据和可视化包 导入统计相关的工具 导入回归相关的算法 导入数据预处理相关的方法 导入模型调参相关的包 读取数据 特征工程 缺失值 类别特征处理-l ...

  8. Java Programming Language Enhancements

    引用:Java Programming Language Enhancements Java Programming Language Enhancements Enhancements in Jav ...

  9. EasyMesh - A Two-Dimensional Quality Mesh Generator

    EasyMesh - A Two-Dimensional Quality Mesh Generator eryar@163.com Abstract. EasyMesh is developed by ...

随机推荐

  1. Android5.0水波纹效果ripple实现

    1.如何设置波纹效果 // 波纹有边界 android:background="?android:attr/selectableItemBackground" // 波纹超出边界 ...

  2. Apache URL重写规则

    1.简介 Apached的重写功能,即是mod_rewrite模块功能,它是apache的一个模块.它的功能非常强大,可以操作URL中的所有部分. 因此我们就可以改写url,给用户提供一个简介大方的u ...

  3. python scrapy 抓取脚本之家文章(scrapy 入门使用简介)

    老早之前就听说过python的scrapy.这是一个分布式爬虫的框架,可以让你轻松写出高性能的分布式异步爬虫.使用框架的最大好处当然就是不同重复造轮子了,因为有很多东西框架当中都有了,直接拿过来使用就 ...

  4. .NET插件技术-应用程序热升级

    今天说一说.NET 中的插件技术,即 应用程序热升级.在很多情况下.我们希望用户对应用程序的升级是无感知的,并且尽可能不打断用户操作的. 虽然在Web 或者 WebAPI上,由于多点的存在可以逐个停用 ...

  5. ajax的介绍

    $.ajax({ 11 url: "article.asmx/GetArticleByID", 12 type: "POST", 13 datatype: &q ...

  6. javascript中类式继承和原型式继承的实现方法和区别

    在所有面向对象的编程中,继承是一个重要的话题.一般说来,在设计类的时候,我们希望能减少重复性的代码,并且尽量弱化对象间的耦合(让一个类继承另一个类可能会导致二者产生强耦合).关于“解耦”是程序设计中另 ...

  7. 基于Hadoop分布式集群YARN模式下的TensorFlowOnSpark平台搭建

    1. 介绍 在过去几年中,神经网络已经有了很壮观的进展,现在他们几乎已经是图像识别和自动翻译领域中最强者[1].为了从海量数据中获得洞察力,需要部署分布式深度学习.现有的DL框架通常需要为深度学习设置 ...

  8. python——网络编程

    Socket socket通常也称作"套接字",用于描述IP地址和端口,是一个通信链的句柄,应用程序通常通过"套接字"向网络发出请求或者应答网络请求. sock ...

  9. Springmvc的工作流程

    1.向服务器发送http请求,请求被前端控制器DispatcherServlet捕获. 2.DispatcherServlet根据servlet.xml中的配置进行URL解析后,得到(URL),然后根 ...

  10. 微信公众号、H5、APP三者各有什么优势?

    昨天给大家分享了一个现在很热的H5,众所周知,当下H5手机网站.微信公众号.APP这三种载体都越来越火了,而且三者都有各自的一些优势和劣势. HTML5(H5) H5之所以能引发如此广泛的效应,根本在 ...