Basic theory

(i) Supervised learning (parametric/non-parametric algorithms, support vector machines, kernels, neural networks, )  regression, classification.
(ii) Unsupervised learning (clustering, dimensionality reduction, recommender systems, deep learning)
 
Others:
Reinforcement learning, recommender systems
 
SSE(和方差、误差平方和):The sum of squares due to error
 
MSE(均方差、方差):Mean squared error
 
RMSE(均方根、标准差):Root mean squared error
 
 
Linear regression
  • Gradient Descent algorithm:
 
     cost function:
  
  % correspoding code to compute gradient decent 
    h = X * theta;
    theta = theta - alpha/m * (X' * (h - y));
    
  • Normal Equation 标准方程/正规方程
 
Gradient Descent vs Normal Equation
time complexity for Gradient Decent is O(kn2) 
Locally weighted regression: 只考虑待预测点附件的training data
 
 
 
Logistic regression
 
    a classfication algorithm
 
 
Cost function:
 
 
 
 
 
  • Gradient Descent algorithm
          其中偏导数的推导如下:
       
 
Newton's method: much faster than Gradient Decent.
 
上图是求f(θ)=0时候的θ, 如果对f(θ)积分的最大值或者最小值
Newton’s method gives a way of getting to f(θ) = 0. What if we want to use it to maximize some function ℓ? The maxima of ℓ correspond to points where its first derivative ℓ ′ (θ) is zero. So, by letting f(θ) = ℓ ′ (θ), we can use the same algorithm to maximize ℓ, and we obtain update rule: 
θ := θ − ℓ ′(θ) / ℓ ′′(θ)
在python里,
 
 
Logistic Regression (Classification) decision boundary:
 

 

 
        
      
Neural Network
cost function:
 
           
          
back propagation algorithm:
 
 
  1. we can use Gradient checking to check if the backpropagation algorithm is working correctly.
  2. need randomly initialize theta
 
 
Diagnostic
Diagnostic 用来分析学习算法是不是正常工作,如果不正常工作,进一步找出原因
                       
怎么来评估learning algorithm 是否工作呢? 
 
可以评估hypothesis 函数, 具体可以把所以input数据分成一部分training set, 另一部分作为test set 来验证,Andrew 建议 70%/30% 这个比例来划分,然后看用training set 得到的hypothesis 在 test set 上是否工作
 
 
  • 一旦发现hypothesis 不工作,可以用model selection 来重新找hypothesis
 
  • 怎么确定 degree of polynomial (hypothesis方程的次数),
  1. 把input data 按照60/20/20% 分成3组 training set/ cross validation set/ test set
  2. 基于traing按照1-10 degree 先给出不同次数的hypothesis 函数,然后用cross validation set 实验不同次数的hypothesis方程, 得到最好结果的hypothesis
  3. 基于test set 可以给出 test report
 
  • 怎么确定 lamda (0, 0,01, 0,02, 0.04, 0.08, ..., 10)
 
 
 
  • Learning curves:
 
high bias:
 
high variance: (high gap)
 
If a learning algorithm is suffering from high bias, getting more training data will not (by itself) help much.
If a learning algorithm is suffering from high variance, getting more training data is likely to help.
 
  • How to upgrade your model?
 
  1. Getting more training examples: Fixes high variance
  2. Trying smaller sets of features: Fixes high variance
  3. Adding features: Fixes high bias
  4. Adding polynomial features: Fixes high bias
  5. Decreasing λ: Fixes high bias
  6. Increasing λ: Fixes high variance.
 
 
 
 
Unsupervised Learning   
 
Generative learning algo
 
 
 

Q&A

What is overfitting problem?
        
  1. How to reduce overfitting problem?
  • reduce the number of features
  • regularization. Keep all the features, but reduce the magnitude of parameters θ j
 
  1. besises Gradient Decent, what other algorithms we can use ?
  • besides Gradient Decent,  there are some optimization algorithms like Conjugate gradient, BFGS, L-BFGS.
  • These 3 optimization algorithms don't need maually pick , and they are often faster than Gradient Decent, but more
 
Reference:
 
 
Terms:
'parametric' learning algo
  • which has fixed set of parameters Theta, like linear regression
'non-parameter' learning algo
  • in which no. of parameters grow with m.
  • one specific algo is Locally weighted regression (Loess, or LWR), 这个算法不需要我们自己选feature,原理是只拟合待预测点附近的点的曲线
 
Discriminative algo: 对y建模 p(y|x)
Generative algo: 对x建模p(x|y)
 
GDA - Gaussian Discriminant Analysis,x 连续,y 不连续 
Naive Bayes - 比如垃圾邮件分类器,x 是不连续的,y 也是不连续的

Coursera, Machine Learning, notes的更多相关文章

  1. Coursera machine learning 第二周 quiz 答案 Linear Regression with Multiple Variables

    https://www.coursera.org/learn/machine-learning/exam/7pytE/linear-regression-with-multiple-variables ...

  2. 神经网络作业: NN LEARNING Coursera Machine Learning(Andrew Ng) WEEK 5

    在WEEK 5中,作业要求完成通过神经网络(NN)实现多分类的逻辑回归(MULTI-CLASS LOGISTIC REGRESSION)的监督学习(SUOERVISED LEARNING)来识别阿拉伯 ...

  3. 【Coursera - machine learning】 Linear regression with one variable-quiz

    Question 1 Consider the problem of predicting how well a student does in her second year of college/ ...

  4. Coursera, Machine Learning, Anomoly Detection & Recommender system

      Algorithm:     When to select Anonaly detection or Supervised learning? 总的来说guideline是如果positive e ...

  5. Coursera, Machine Learning, SVM

    Support Vector Machine (large margin classifiers ) 1. cost function and hypothesis 下面那个紫色线就是SVM 的cos ...

  6. Coursera, Machine Learning, Neural Networks: Representation - week4/5

    Neural Network Motivations 想要拟合一条曲线,在feature 很多的情况下,feature的组合也很多,在现实中不适用,比如在computer vision问题中featu ...

  7. Coursera machine learning 第二周 编程作业 Linear Regression

    必做: [*] warmUpExercise.m - Simple example function in Octave/MATLAB[*] plotData.m - Function to disp ...

  8. Coursera machine learning 第二周 quiz 答案 Octave/Matlab Tutorial

    https://www.coursera.org/learn/machine-learning/exam/dbM1J/octave-matlab-tutorial Octave Tutorial 5  ...

  9. Coursera Machine Learning 作业答案脚本 分享在github上

    Github地址:https://github.com/edward0130/Coursera-ML

随机推荐

  1. Servlet -- 中文乱码解决

    请求:对于get和post都有效果 request.setCharacterEncoding("UTF-8"); 相应: 设置服务器输出的编码为UTF-8 response.set ...

  2. BZOJ 1912: [Apio2010]patrol 巡逻 (树的直径)(详解)

    题目: https://www.lydsy.com/JudgeOnline/problem.php?id=1912 题解: 首先,显然当不加边的时候,遍历一棵树每条边都要经过两次.那么现在考虑k==1 ...

  3. Redis基础、高级特性与性能调优

    本文将从Redis的基本特性入手,通过讲述Redis的数据结构和主要命令对Redis的基本能力进行直观介绍.之后概览Redis提供的高级能力,并在部署.维护.性能调优等多个方面进行更深入的介绍和指导. ...

  4. vue2.0项目实战(2)使用 vue-cli 搭建项目

    Vue-cli是官方推荐的快速构建单页应用的脚手架.官方给的建议,如果你是初次尝试Vue,哪就老老实实用普通的书写引入js文件,这里牵扯太多的东西,例如webpack .npm .nodejs 等等, ...

  5. python与java的猜拳游戏

    python版: import randomprint("-----猜拳游戏-----")print("---0.剪刀--1.石头--2.布---")while ...

  6. MySQL排序函数field()详解

    在日常开发过程中,排序是经常用到的,有时候有这样的需求. 比如,需要在查询结果中根据某个字段的具体值来排序.如下面例子 上面是一张个人信息 表,假如我们想按照'seiki','iris','xut'来 ...

  7. MySQL表结构的优化和设计

    仅供自己学习   结论写在前面: 1.给字段选取最合适的数据类型 2.数据类型的宽度尽可能的小 3.给where条件的字段设置索引 4.允许部分数据冗余 5.字段要尽可能的设置为not null,特别 ...

  8. django框架中的全文检索Haystack

    1.什么是Haystack Haystack是django的开源全文搜索框架(全文检索不同于特定字段的模糊查询,使用全文检索的效率更高 ),该框架支持Solr,Elasticsearch,Whoosh ...

  9. 【精】搭建redis cluster集群,JedisCluster带密码访问【解决当中各种坑】!

    转: [精]搭建redis cluster集群,JedisCluster带密码访问[解决当中各种坑]! 2017年05月09日 00:13:18 冉椿林博客 阅读数:18208  版权声明:本文为博主 ...

  10. vim: 基本知识;

    1. 函数: function!   funcName(para.) content; endfunction 如果添加!,将覆盖已存在的重名函数: 注: 该博文为扩展型: 2.调用外部命令: exe ...