CheeseZH: Stanford University: Machine Learning Ex3: Multiclass Logistic Regression and Neural Network Prediction
Handwritten digits recognition (0-9)
Multi-class Logistic Regression
1. Vectorizing Logistic Regression
(1) Vectorizing the cost function
(2) Vectorizing the gradient
(3) Vectorizing the regularized cost function
(4) Vectorizing the regularized gradient
All above 4 formulas can be found in the previous blog: click here.
lrCostFunction.m
function [J, grad] = lrCostFunction(theta, X, y, lambda)
%LRCOSTFUNCTION Compute cost and gradient for logistic regression with
%regularization
% J = LRCOSTFUNCTION(theta, X, y, lambda) computes the cost of using
% theta as the parameter for regularized logistic regression and the
% gradient of the cost w.r.t. to the parameters. % Initialize some useful values
m = length(y); % number of training examples % You need to return the following variables correctly
J = ;
grad = zeros(size(theta)); % ====================== YOUR CODE HERE ======================
% Instructions: Compute the cost of a particular choice of theta.
% You should set J to the cost.
% Compute the partial derivatives and set grad to the partial
% derivatives of the cost w.r.t. each parameter in theta
%
% Hint: The computation of the cost function and gradients can be
% efficiently vectorized. For example, consider the computation
%
% sigmoid(X * theta)
%
% Each row of the resulting matrix will contain the value of the
% prediction for that example. You can make use of this to vectorize
% the cost function and gradient computations.
%
% Hint: When computing the gradient of the regularized cost function,
% there're many possible vectorized solutions, but one solution
% looks like:
% grad = (unregularized gradient for logistic regression)
% temp = theta;
% temp() = ; % because we don't add anything for j = 0
% grad = grad + YOUR_CODE_HERE (using the temp variable)
% hx = sigmoid(X*theta);
reg = lambda/(*m)*sum(theta(:size(theta),:).^);
J = -/m*(y'*log(hx)+(1-y)'*log(-hx)) + reg;
theta() = ;
grad = /m*X'*(hx-y)+lambda/m*theta; % ============================================================= grad = grad(:); end
2. One-vs-all Classification (Training)
Return all the classifier parameters in a matrix Θ (a K x N+1 matrix, K is the num_labels and N is the num_features ), where each row of Θ corresponds to the learned logistic regression parameters for one class. You can do this with a 'for'-loop from 1 to K, training each classifier independently.
oneVsAll.m
function [all_theta] = oneVsAll(X, y, num_labels, lambda)
%ONEVSALL trains multiple logistic regression classifiers and returns all
%the classifiers in a matrix all_theta, where the i-th row of all_theta
%corresponds to the classifier for label i
% [all_theta] = ONEVSALL(X, y, num_labels, lambda) trains num_labels
% logisitc regression classifiers and returns each of these classifiers
% in a matrix all_theta, where the i-th row of all_theta corresponds
% to the classifier for label i % Some useful variables
m = size(X, );
n = size(X, ); % You need to return the following variables correctly
all_theta = zeros(num_labels, n + ); % Add ones to the X data matrix
X = [ones(m, ) X]; % ====================== YOUR CODE HERE ======================
% Instructions: You should complete the following code to train num_labels
% logistic regression classifiers with regularization
% parameter lambda.
%
% Hint: theta(:) will return a column vector.
%
% Hint: You can use y == c to obtain a vector of 's and 0's that tell use
% whether the ground truth is true/false for this class.
%
% Note: For this assignment, we recommend using fmincg to optimize the cost
% function. It is okay to use a for-loop (for c = :num_labels) to
% loop over the different classes.
%
% fmincg works similarly to fminunc, but is more efficient when we
% are dealing with large number of parameters.
%
% Example Code for fmincg:
%
% % Set Initial theta
% initial_theta = zeros(n + , );
%
% % Set options for fminunc
% options = optimset('GradObj', 'on', 'MaxIter', );
%
% % Run fmincg to obtain the optimal theta
% % This function will return theta and the cost
% [theta] = ...
% fmincg (@(t)(lrCostFunction(t, X, (y == c), lambda)), ...
% initial_theta, options);
% for c=:num_labels,
initial_theta = all_theta(c,:)';
options = optimset('GradObj','on','MaxIter',);
theta = fmincg(@(t)(lrCostFunction(t,X,(y==c),lambda)),initial_theta,options);
all_theta(c,:) = theta';
end; % ========================================================================= end
3. One-vs-all Classification (Prediction)
predictOneVsAll.m
Neural Network Prediction
Feedword Propagation and Prediction
predict.m
function p = predict(Theta1, Theta2, X)
%PREDICT Predict the label of an input given a trained neural network
% p = PREDICT(Theta1, Theta2, X) outputs the predicted label of X given the
% trained weights of a neural network (Theta1, Theta2) % Useful values
m = size(X, );
num_labels = size(Theta2, ); % You need to return the following variables correctly
p = zeros(size(X, ), ); % ====================== YOUR CODE HERE ======================
% Instructions: Complete the following code to make predictions using
% your learned neural network. You should set p to a
% vector containing labels between to num_labels.
%
% Hint: The max function might come in useful. In particular, the max
% function can also return the index of the max element, for more
% information see 'help max'. If your examples are in rows, then, you
% can use max(A, [], ) to obtain the max for each row.
%
a1 = X; %*
a1 = [ones(size(X,), ),X]; %*
a2 = sigmoid(a1*Theta1');%5000*25
a2 = [ones(size(a2,),),a2]; %*
a3 = sigmoid(a2*Theta2');%5000*10
[tmp,p] = max(a3,[],);
% ========================================================================= end
Other files and dataset can be download in Coursera.
CheeseZH: Stanford University: Machine Learning Ex3: Multiclass Logistic Regression and Neural Network Prediction的更多相关文章
- CheeseZH: Stanford University: Machine Learning Ex5:Regularized Linear Regression and Bias v.s. Variance
源码:https://github.com/cheesezhe/Coursera-Machine-Learning-Exercise/tree/master/ex5 Introduction: In ...
- CheeseZH: Stanford University: Machine Learning Ex2:Logistic Regression
1. Sigmoid Function In Logisttic Regression, the hypothesis is defined as: where function g is the s ...
- CheeseZH: Stanford University: Machine Learning Ex1:Linear Regression
(1) How to comput the Cost function in Univirate/Multivariate Linear Regression; (2) How to comput t ...
- CheeseZH: Stanford University: Machine Learning Ex4:Training Neural Network(Backpropagation Algorithm)
1. Feedforward and cost function; 2.Regularized cost function: 3.Sigmoid gradient The gradient for t ...
- [Machine Learning]学习笔记-Logistic Regression
[Machine Learning]学习笔记-Logistic Regression 模型-二分类任务 Logistic regression,亦称logtic regression,翻译为" ...
- Andrew Ng Machine Learning 专题【Logistic Regression & Regularization】
此文是斯坦福大学,机器学习界 superstar - Andrew Ng 所开设的 Coursera 课程:Machine Learning 的课程笔记. 力求简洁,仅代表本人观点,不足之处希望大家探 ...
- 机器学习---朴素贝叶斯与逻辑回归的区别(Machine Learning Naive Bayes Logistic Regression Difference)
朴素贝叶斯与逻辑回归的区别: 朴素贝叶斯 逻辑回归 生成模型(Generative model) 判别模型(Discriminative model) 对特征x和目标y的联合分布P(x,y)建模,使用 ...
- machine learning(10) -- classification:logistic regression cost function 和 使用 gradient descent to minimize cost function
logistic regression cost function(single example) 图像分布 logistic regression cost function(m examples) ...
- Machine Learning in Action -- Logistic regression
这个系列,重点关注如何实现,至于算法基础,参考Andrew的公开课 相较于线性回归,logistic回归更适合用于分类 因为他使用Sigmoid函数,因为分类的取值是0,1 对于分类,最完美和自然的函 ...
随机推荐
- bzoj 3996 最小割
公式推出来后想了半天没思路,居然A是01矩阵..... 如果一个问题是求最值,并那么尝试先将所有可能收益加起来,然后矛盾部分能否用最小割表达(本题有两个矛盾,第一个是选还是不选,第二个是i,j有一个不 ...
- 关于List Map Set的线程安全的问题
常见的ArrayList LinkedList HashMap TreeMap LinkedHashMap HashSet TreeSet LinkedHashSet 都是线程不安全的.如果要使用 ...
- BitmapFactory.Options.inSampleSize 的使用方法
BitmapFactory.decodeFile(imageFile); 用BitmapFactory解码一张图片时.有时会遇到该错误. 这往往是因为图片过大造成的. 要想正常使用,则须要分配更少的内 ...
- jQuery 二级菜单,一次显示一个小类 鼠标点击显示小类
jQuery 二级菜单,一次显示一个小类 鼠标点击显示小类 本例有另外2个关联案例,演示地址分别为2.php,3.php 演示 XML/HTML Code <div class="ar ...
- ios 重用UI部分代码的好方法(再也不用为局部变量的命名而烦恼啦!)
重用控件类代码的一个非常好的解决方案:所有一样的控件其名字均用同样的一个名字.只是在最后赋值的时候,将创建好的控件赋给我们需要用到的那个控件. - (id)initWithFrame:(CGRect) ...
- 转 SQL语句的添加、删除、修改多种方法
SQL语句的添加.删除.修改虽然有如下很多种方法,但在使用过程中还是不够用,不知是否有高手把更多灵活的使用方法贡献出来? 添加.删除.修改使用db.Execute(Sql)命令执行操作 ╔------ ...
- Android中关于项目中对Thread的管理(不是线程池)
背景 项目中对于一些并不复杂的耗时操作,比如计算,不频繁操作数据库等,因为没必要使用线程池,所以之前项目会直接使用new Thread的方式,时间一长,回头再看,原来new Thread之处已经很多了 ...
- Windows 8 Metro 应用开发入门(一):开发环境介绍
摘 要 Windows8已经发布,随之而来的基于WinRT的Metro应用也正向我们走来,正像它所宣传的:光滑.快.现代.看习惯了玻璃.立体风格的应用,或许Metro的简洁能给你留下不一样的体验.Vi ...
- 基于tomcat7 web开发中的一点小东西
控制台: org.apache.jasper.compiler.TldLocationsCache tldScanJar 信息: At least one JAR was scanned for TL ...
- Serializable java序列化
Bean Serializable Interface 的接口让BEAN可以串行化,将其变成一个可保存为以后使用的二进制流.当一个BEAN被系列化到磁盘上或者其他任何地方,其状态被保存起来,其中的属性 ...