Coursera, Machine Learning, notes
Basic theory
Linear regression
|
cost function:
![]() % correspoding code to compute gradient decent
h = X * theta;
theta = theta - alpha/m * (X' * (h - y));
![]() Gradient Descent vs Normal Equation
time complexity for Gradient Decent is O(kn2)
|
Locally weighted regression: 只考虑待预测点附件的training data
![]() |
Logistic regression
|
a classfication algorithm
![]() ![]() Cost function:
![]() ![]() ![]()
其中偏导数的推导如下:
![]() ![]() |
Newton's method: much faster than Gradient Decent.
![]() 上图是求f(θ)=0时候的θ, 如果对f(θ)积分的最大值或者最小值
Newton’s method gives a way of getting to f(θ) = 0. What if we want to use it to maximize some function ℓ? The maxima of ℓ correspond to points where its first derivative ℓ ′ (θ) is zero. So, by letting f(θ) = ℓ ′ (θ), we can use the same algorithm to maximize ℓ, and we obtain update rule:
θ := θ − ℓ ′(θ) / ℓ ′′(θ)
|
在python里,
![]() |


Neural Network
|
cost function:
![]() |
back propagation algorithm:
![]() ![]() ![]() |
|
Diagnostic
|
Diagnostic 用来分析学习算法是不是正常工作,如果不正常工作,进一步找出原因
|
怎么来评估learning algorithm 是否工作呢?
可以评估hypothesis 函数, 具体可以把所以input数据分成一部分training set, 另一部分作为test set 来验证,Andrew 建议 70%/30% 这个比例来划分,然后看用training set 得到的hypothesis 在 test set 上是否工作
![]() |
high bias:
high variance: (high gap)
|
Q&A
- How to reduce overfitting problem?
- reduce the number of features
- regularization. Keep all the features, but reduce the magnitude of parameters θ j
- besises Gradient Decent, what other algorithms we can use ?
- besides Gradient Decent, there are some optimization algorithms like Conjugate gradient, BFGS, L-BFGS.
- These 3 optimization algorithms don't need maually pick
, and they are often faster than Gradient Decent, but more
- which has fixed set of parameters Theta, like linear regression
- in which no. of parameters grow with m.
- one specific algo is Locally weighted regression (Loess, or LWR), 这个算法不需要我们自己选feature,原理是只拟合待预测点附近的点的曲线
Coursera, Machine Learning, notes的更多相关文章
- Coursera machine learning 第二周 quiz 答案 Linear Regression with Multiple Variables
https://www.coursera.org/learn/machine-learning/exam/7pytE/linear-regression-with-multiple-variables ...
- 神经网络作业: NN LEARNING Coursera Machine Learning(Andrew Ng) WEEK 5
在WEEK 5中,作业要求完成通过神经网络(NN)实现多分类的逻辑回归(MULTI-CLASS LOGISTIC REGRESSION)的监督学习(SUOERVISED LEARNING)来识别阿拉伯 ...
- 【Coursera - machine learning】 Linear regression with one variable-quiz
Question 1 Consider the problem of predicting how well a student does in her second year of college/ ...
- Coursera, Machine Learning, Anomoly Detection & Recommender system
Algorithm: When to select Anonaly detection or Supervised learning? 总的来说guideline是如果positive e ...
- Coursera, Machine Learning, SVM
Support Vector Machine (large margin classifiers ) 1. cost function and hypothesis 下面那个紫色线就是SVM 的cos ...
- Coursera, Machine Learning, Neural Networks: Representation - week4/5
Neural Network Motivations 想要拟合一条曲线,在feature 很多的情况下,feature的组合也很多,在现实中不适用,比如在computer vision问题中featu ...
- Coursera machine learning 第二周 编程作业 Linear Regression
必做: [*] warmUpExercise.m - Simple example function in Octave/MATLAB[*] plotData.m - Function to disp ...
- Coursera machine learning 第二周 quiz 答案 Octave/Matlab Tutorial
https://www.coursera.org/learn/machine-learning/exam/dbM1J/octave-matlab-tutorial Octave Tutorial 5 ...
- Coursera Machine Learning 作业答案脚本 分享在github上
Github地址:https://github.com/edward0130/Coursera-ML
随机推荐
- Servlet -- 中文乱码解决
请求:对于get和post都有效果 request.setCharacterEncoding("UTF-8"); 相应: 设置服务器输出的编码为UTF-8 response.set ...
- BZOJ 1912: [Apio2010]patrol 巡逻 (树的直径)(详解)
题目: https://www.lydsy.com/JudgeOnline/problem.php?id=1912 题解: 首先,显然当不加边的时候,遍历一棵树每条边都要经过两次.那么现在考虑k==1 ...
- Redis基础、高级特性与性能调优
本文将从Redis的基本特性入手,通过讲述Redis的数据结构和主要命令对Redis的基本能力进行直观介绍.之后概览Redis提供的高级能力,并在部署.维护.性能调优等多个方面进行更深入的介绍和指导. ...
- vue2.0项目实战(2)使用 vue-cli 搭建项目
Vue-cli是官方推荐的快速构建单页应用的脚手架.官方给的建议,如果你是初次尝试Vue,哪就老老实实用普通的书写引入js文件,这里牵扯太多的东西,例如webpack .npm .nodejs 等等, ...
- python与java的猜拳游戏
python版: import randomprint("-----猜拳游戏-----")print("---0.剪刀--1.石头--2.布---")while ...
- MySQL排序函数field()详解
在日常开发过程中,排序是经常用到的,有时候有这样的需求. 比如,需要在查询结果中根据某个字段的具体值来排序.如下面例子 上面是一张个人信息 表,假如我们想按照'seiki','iris','xut'来 ...
- MySQL表结构的优化和设计
仅供自己学习 结论写在前面: 1.给字段选取最合适的数据类型 2.数据类型的宽度尽可能的小 3.给where条件的字段设置索引 4.允许部分数据冗余 5.字段要尽可能的设置为not null,特别 ...
- django框架中的全文检索Haystack
1.什么是Haystack Haystack是django的开源全文搜索框架(全文检索不同于特定字段的模糊查询,使用全文检索的效率更高 ),该框架支持Solr,Elasticsearch,Whoosh ...
- 【精】搭建redis cluster集群,JedisCluster带密码访问【解决当中各种坑】!
转: [精]搭建redis cluster集群,JedisCluster带密码访问[解决当中各种坑]! 2017年05月09日 00:13:18 冉椿林博客 阅读数:18208 版权声明:本文为博主 ...
- vim: 基本知识;
1. 函数: function! funcName(para.) content; endfunction 如果添加!,将覆盖已存在的重名函数: 注: 该博文为扩展型: 2.调用外部命令: exe ...