要解决的问题是,给出了具有2个特征的一堆训练数据集,从该数据的分布可以看出它们并不是非常线性可分的,因此很有必要用更高阶的特征来模拟。例如本程序中个就用到了特征值的6次方来求解。

Data

To begin, load the files 'ex5Logx.dat' and ex5Logy.dat' into your program. This dataset represents the training set of a logistic regression problem with two features. To avoid confusion later, we will refer to the two input features contained in 'ex5Logx.dat' as and . So in the 'ex5Logx.dat' file, the first column of numbers represents the feature , which you will plot on the horizontal axis, and the second feature represents , which you will plot on the vertical axis.

After loading the data, plot the points using different markers to distinguish between the two classifications. The commands in Matlab/Octave will be:

x = load('ex5Logx.dat');
y = load('ex5Logy.dat'); figure % Find the indices for the 2 classes
pos = find(y); neg = find(y == 0); plot(x(pos, 1), x(pos, 2), '+')
hold on
plot(x(neg, 1), x(neg, 2), 'o')

After plotting your image, it should look something like this:

Model

the hypothesis function is

 

Let's look at the parameter in the sigmoid function .

In this exercise, we will assign to be all monomials (meaning polynomial terms) of and up to the sixth power:

To clarify this notation: we have made a 28-feature vector where

此时加入了规则项后的系统的损失函数为:

Newton’s method

Recall that the Newton's Method update rule is

1. is your feature vector, which is a 28x1 vector in this exercise.

2. is a 28x1 vector.

3. and are 28x28 matrices.

4. and are scalars.

5. The matrix following in the Hessian formula is a 28x28 diagonal matrix with a zero in the upper left and ones on every other diagonal entry.

After convergence, use your values of theta to find the decision boundary in the classification problem. The decision boundary is defined as the line where

Code

%载入数据
clc,clear,close all;
x = load('ex5Logx.dat');
y = load('ex5Logy.dat'); %画出数据的分布图
plot(x(find(y),),x(find(y),),'o','MarkerFaceColor','b')
hold on;
plot(x(find(y==),),x(find(y==),),'r+')
legend('y=1','y=0') % Add polynomial features to x by
% calling the feature mapping function
% provided in separate m-file
x = map_feature(x(:,), x(:,)); %投影到高维特征空间 [m, n] = size(x); % Initialize fitting parameters
theta = zeros(n, ); % Define the sigmoid function
g = inline('1.0 ./ (1.0 + exp(-z))'); % setup for Newton's method
MAX_ITR = ;
J = zeros(MAX_ITR, ); % Lambda is the regularization parameter
lambda = ;%lambda=,,,修改这个地方,运行3次可以得到3种结果。 % Newton's Method
for i = :MAX_ITR
% Calculate the hypothesis function
z = x * theta;
h = g(z); % Calculate J (for testing convergence) -- 损失函数
J(i) =(/m)*sum(-y.*log(h) - (-y).*log(-h))+ ...
(lambda/(*m))*norm(theta([:end]))^; % Calculate gradient and hessian.
G = (lambda/m).*theta; G() = ; % extra term for gradient
L = (lambda/m).*eye(n); L() = ;% extra term for Hessian
grad = ((/m).*x' * (h-y)) + G;
H = ((/m).*x' * diag(h) * diag(1-h) * x) + L; % Here is the actual update
theta = theta - H\grad; end % Plot the results
% We will evaluate theta*x over a
% grid of features and plot the contour
% where theta*x equals zero % Here is the grid range
u = linspace(-, 1.5, );
v = linspace(-, 1.5, ); z = zeros(length(u), length(v));
% Evaluate z = theta*x over the grid
for i = :length(u)
for j = :length(v)
z(i,j) = map_feature(u(i), v(j))*theta;%这里绘制的并不是损失函数与迭代次数之间的曲线,而是线性变换后的值
end
end
z = z'; % important to transpose z before calling contour % Plot z =
% Notice you need to specify the range [, ]
contour(u, v, z, [, ], 'LineWidth', )%在z上画出为0值时的界面,因为为0时刚好概率为0.,符合要求
legend('y = 1', 'y = 0', 'Decision boundary')
title(sprintf('\\lambda = %g', lambda), 'FontSize', ) hold off % Uncomment to plot J
% figure
% plot(:MAX_ITR-, J, 'o--', 'MarkerFaceColor', 'r', 'MarkerSize', )
% xlabel('Iteration'); ylabel('J')

Result

Regularized logistic regression的更多相关文章

  1. machine learning(15) --Regularization:Regularized logistic regression

    Regularization:Regularized logistic regression without regularization 当features很多时会出现overfitting现象,图 ...

  2. matlab(7) Regularized logistic regression : mapFeature(将feature增多) and costFunctionReg

    Regularized logistic regression : mapFeature(将feature增多) and costFunctionReg ex2_reg.m文件中的部分内容 %% == ...

  3. matlab(6) Regularized logistic regression : plot data(画样本图)

    Regularized logistic regression :  plot data(画样本图) ex2data2.txt 0.051267,0.69956,1-0.092742,0.68494, ...

  4. 编程作业2.2:Regularized Logistic regression

    题目 在本部分的练习中,您将使用正则化的Logistic回归模型来预测一个制造工厂的微芯片是否通过质量保证(QA),在QA过程中,每个芯片都会经过各种测试来保证它可以正常运行.假设你是这个工厂的产品经 ...

  5. matlab(8) Regularized logistic regression : 不同的λ(0,1,10,100)值对regularization的影响,对应不同的decision boundary\ 预测新的值和计算模型的精度predict.m

    不同的λ(0,1,10,100)值对regularization的影响\ 预测新的值和计算模型的精度 %% ============= Part 2: Regularization and Accur ...

  6. 吴恩达机器学习笔记22-正则化逻辑回归模型(Regularized Logistic Regression)

    针对逻辑回归问题,我们在之前的课程已经学习过两种优化算法:我们首先学习了使用梯度下降法来优化代价函数

  7. Stanford机器学习---第三讲. 逻辑回归和过拟合问题的解决 logistic Regression & Regularization

    原文:http://blog.csdn.net/abcjennifer/article/details/7716281 本栏目(Machine learning)包括单参数的线性回归.多参数的线性回归 ...

  8. Machine Learning - 第3周(Logistic Regression、Regularization)

    Logistic regression is a method for classifying data into discrete outcomes. For example, we might u ...

  9. 【机器学习】Octave 实现逻辑回归 Logistic Regression

    ex2data1.txt ex2data2.txt 本次算法的背景是,假如你是一个大学的管理者,你需要根据学生之前的成绩(两门科目)来预测该学生是否能进入该大学. 根据题意,我们不难分辨出这是一种二分 ...

随机推荐

  1. 鼠标点击textarea后,在光标后追加内容

    $("#insertMsg").on("click",function(){ //获取下拉选项框的值 var textFeildValue = $(" ...

  2. 求第区间第k大数 TLE归并树

    题 给定N个正整数构成的序列,将对于指定的闭区间查询其区间内的第K小值. 输入: 第一行包含两个正整数N.M,分别表示序列的长度和查询的个数. 第二行包含N个正整数,表示这个序列各项的数字. 接下来M ...

  3. 前端之CSS介绍

    CSS介绍 CSS(Cascading Style Sheet,层叠样式表)定义如何显示HTML元素. 当浏览器读到一个样式表,它就会按照这个样式表来对文档进行格式化(渲染). CSS的语法 CSS语 ...

  4. Python组织文件 实践:将带有美国风格日期的文件改名为欧洲风格日期

    描述:假设有这样一个任务,你需要将文件名中含有美国风格日期(MM-DD-YYYY)的部分更换为欧洲风格日期(DD-MM-YYYY),并且需要你处理的文件多达上千个 分析:检查当前工作目录的所有文件名, ...

  5. 题解 HDU1565 【方格取数(1)】

    给你一个n*n的格子的棋盘,每个格子里面有一个非负数. 从中取出若干个数,使得任意的两个数所在的格子没有公共边,就是说所取的数所在的2个格子不能相邻,并且取出的数的和最大. 题目清晰明了,这道题应该用 ...

  6. Nrf51822中设置128bit UUID service

    Nrf51822中设置128bit UUID service uint32_tble_dajia_add_service(ble_dajia_t *p_wechat) { uint32_t err_c ...

  7. Class C++

    为了尽量降低全局变量的使用并提供用户自己定义类型的功能.C++语言提供了一种新的语言机制---类(class).并以类作为构造程序的基本单位 #include<iostream> usin ...

  8. numpy 数据类型与 Python 原生数据类型

    查看 numpy 数据类型和 Python 原生数据类型之间的对应关系: In [51]: dict([(d, type(np.zeros(1,d).tolist()[0])) for d in (n ...

  9. 动态代理(jdk--cglib)

    JAVA的动态代理 代理模式 代理模式是常用的java设计模式,他的特征是代理类与委托类有同样的接口,代理类主要负责为委托类预处理消息.过滤消息.把消息转发给委托类,以及事后处理消息等.代理类与委托类 ...

  10. 【引用】Android程序实现完全退出

    这是我在网上找到的,方法不错,都能够实现程序的完全退出http://www.jb51.net/article/37992.htm