Regularization —— linear regression
本节主要是练习regularization项的使用原则。因为在机器学习的一些模型中,如果模型的参数太多,而训练样本又太少的话,这样训练出来的模型很容易产生过拟合现象。因此在模型的损失函数中,需要对模型的参数进行“惩罚”,这样的话这些参数就不会太大,而越小的参数说明模型越简单,越简单的模型则越不容易产生过拟合现象。
Regularized linear regression
From looking at this plot, it seems that fitting a straight line might be too simple of an approximation. Instead, we will try fitting a higher-order polynomial to the data to capture more of the variations in the points.
Let's try a fifth-order polynomial. Our hypothesis will be
This means that we have a hypothesis of six features, because are now all features of our regression. Notice that even though we are producing a polynomial fit, we still have a linear regression problem because the hypothesis is linear in each feature.
Since we are fitting a 5th-order polynomial to a data set of only 7 points, over-fitting is likely to occur. To guard against this, we will use regularization in our model.
Recall that in regularization problems, the goal is to minimize the following cost function with respect to :
The regularization parameter is a control on your fitting parameters. As the magnitues of the fitting parameters increase, there will be an increasing penalty on the cost function. This penalty is dependent on the squares of the parameters as well as the magnitude of
. Also, notice that the summation after
does not include
lamda 越大,训练出的模型越简单 —— 后一项的惩罚越大
Normal equations
Now we will find the best parameters of our model using the normal equations. Recall that the normal equations solution to regularized linear regression is
The matrix following
is an
diagonal matrix with a zero in the upper left and ones down the other diagonal entries. (Remember that
is the number of features, not counting the intecept term). The vector
and the matrix
have the same definition they had for unregularized regression:
Using this equation, find values for
using the three regularization parameters below:
a.
(this is the same case as non-regularized linear regression)
b.
c.
Code
clc,clear
%加载数据
x = load('ex5Linx.dat');
y = load('ex5Liny.dat'); %显示原始数据
plot(x,y,'o','MarkerEdgeColor','b','MarkerFaceColor','r') %将特征值变成训练样本矩阵
x = [ones(length(x),) x x.^ x.^ x.^ x.^];
[m n] = size(x);
n = n -; %计算参数sidta,并且绘制出拟合曲线
rm = diag([;ones(n,)]);%lamda后面的矩阵
lamda = [ ]';
colortype = {'g','b','r'};
sida = zeros(n+,); %初始化参数sida
xrange = linspace(min(x(:,)),max(x(:,)))';
hold on;
for i = :
sida(:,i) = inv(x'*x+lamda(i).*rm)*x'*y;%计算参数sida
norm_sida = norm(sida) % norm 求sida的2阶范数
yrange = [ones(size(xrange)) xrange xrange.^ xrange.^,...
xrange.^ xrange.^]*sida(:,i);
plot(xrange',yrange,char(colortype(i)))
hold on
end
legend('traning data', '\lambda=0', '\lambda=1','\lambda=10')%注意转义字符的使用方法
hold off
Regularization —— linear regression的更多相关文章
- machine learning(14) --Regularization:Regularized linear regression
machine learning(13) --Regularization:Regularized linear regression Gradient descent without regular ...
- Matlab实现线性回归和逻辑回归: Linear Regression & Logistic Regression
原文:http://blog.csdn.net/abcjennifer/article/details/7732417 本文为Maching Learning 栏目补充内容,为上几章中所提到单参数线性 ...
- Stanford机器学习---第二讲. 多变量线性回归 Linear Regression with multiple variable
原文:http://blog.csdn.net/abcjennifer/article/details/7700772 本栏目(Machine learning)包括单参数的线性回归.多参数的线性回归 ...
- Stanford机器学习---第一讲. Linear Regression with one variable
原文:http://blog.csdn.net/abcjennifer/article/details/7691571 本栏目(Machine learning)包括单参数的线性回归.多参数的线性回归 ...
- Regularized Linear Regression with scikit-learn
Regularized Linear Regression with scikit-learn Earlier we covered Ordinary Least Squares regression ...
- 机器学习笔记-1 Linear Regression with Multiple Variables(week 2)
1. Multiple Features note:X0 is equal to 1 2. Feature Scaling Idea: make sure features are on a simi ...
- Simple tutorial for using TensorFlow to compute a linear regression
"""Simple tutorial for using TensorFlow to compute a linear regression. Parag K. Mita ...
- 第五次编程作业-Regularized Linear Regression and Bias v.s. Variance
1.正规化的线性回归 (1)代价函数 (2)梯度 linearRegCostFunction.m function [J, grad] = linearRegCostFunction(X, y, th ...
- [UFLDL] Linear Regression & Classification
博客内容取材于:http://www.cnblogs.com/tornadomeet/archive/2012/06/24/2560261.html Deep learning:六(regulariz ...
随机推荐
- jsLittle源码封装对象合并
JSLi.extend = JSLi.fn.extend = function () { var options, name, src, copy, target = arguments[0],i = ...
- UI Framework-1: Aura Event Handling
Event Handling A diagram of the architecture of this system: HWNDMessageHandler owns the WNDPROC ...
- 瞎折腾-CentOS 7.4 编译4.16.2版kernel 并安装
CentOS 7.4下 原内核版本: 3.10.0-693.el7.x86_64 改后内核版本: 4.16.2 系统版本: CentOS-7-x86_64-Minimal-1708.iso 运行环境: ...
- bzoj 2456: mode 思维题 好题
题目描述: 给你一个 $n$ 个数的数列,其中某个数出现了超过 n div 2 次即众数,请你找出那个数.空间大小:1mb 题解:显然,我们是不能开任何数组的,此题专卡空间.然而我们要求的东西也十分简 ...
- 在 yii2.0 框架中封装导出html 表格样式 Excel 类
在 vendor/yiisoft/yii2/helpers/ 创建一个 Excel.php <?php namespace yii\helpers; class Excel{ ...
- grep常用命令讲解
grep大家应该并不陌生,但是这个命令你确定真的会用吗?ok,接下来我通过举例子的方式,带你看清grep的本质. 首先,把/etc/password的内容复制下来命令为1.txt吧,方便操作,哈哈~ ...
- Strom 简介,ppt
数据的生成可以看做一连串发生的离散事件, 这些事件流会伴随着不同的数据流.操作和分析,都会由一个通用的软件框架和基础设施来处理. Storm 是 实时流计算框架之一.它提供了可容错 ...
- [Angular] *ngIf syntx
<div class="profile"> <img [src]="user.img" width="50px"> ...
- Class C++
为了尽量降低全局变量的使用并提供用户自己定义类型的功能.C++语言提供了一种新的语言机制---类(class).并以类作为构造程序的基本单位 #include<iostream> usin ...
- JSR 303 - Bean Validation 简单介绍及用法
一.JSR-303简单介绍 JSR-303 是 JAVA EE 6 中的一项子规范.叫做 Bean Validation,官方參考实现是Hibernate Validator. 此实现与 Hibern ...