solving the problem of overfitting:regularization

  • 发生的在linear regression上面的overfitting问题

  • 发生在logistic regression上面的overfitting

  • 怎么解决overfitting

  • regularization: cost function of linear regression

    • parameters小的话，这样hypothesis就会变得简单，这样就不会overfitting
    • 一般不会对θ0进行regularization
    • 上式是进行regularization的linear regression的cost function,要使上式的值取最小值
  •  对这个cost function 的分析

    • 由两个式子(两个目标)组成，第一个式子是为了对trainning data更好的拟合（fitting the training data），第二个式子是为了避免overfitting
    • 第二个式子叫regularization term, λ叫regularization parameter, λ是为了平衡两个目标用的
    • 如果 λ非常大的话(这时θ1-θn几乎为0,hypothesis变得很简单，只有常数)，就会出现underfitting,对trainning data/ new data很低的fitting
    • 所以并不是regularization在任何情况下(当 λ非常大的情况下),都能使model更适应new data或者training data
    • The regularization term puts a penalty on the cost J，随着模型参数的增多，the penalty increases as well.

machine learning(13) -- solving the problem of overfitting:regularization的更多相关文章

  1. Solving the Problem of Overfitting

    The Problem of Overfitting Cost Function Regularized Linear Regression Note: [8:43 - It is said that ...

  2. Advice for applying Machine Learning

    https://jmetzen.github.io/2015-01-29/ml_advice.html Advice for applying Machine Learning This post i ...

  3. How do I learn mathematics for machine learning?

    https://www.quora.com/How-do-I-learn-mathematics-for-machine-learning   How do I learn mathematics f ...

  4. [C2P2] Andrew Ng - Machine Learning

    ##Linear Regression with One Variable Linear regression predicts a real-valued output based on an in ...

  5. Machine Learning - 第3周（Logistic Regression、Regularization）

    Logistic regression is a method for classifying data into discrete outcomes. For example, we might u ...

  6. Course Machine Learning Note

    Machine Learning Note Introduction Introduction What is Machine Learning? Two definitions of Machine ...

  7. 【Machine Learning is Fun!】1.The world’s easiest introduction to Machine Learning

    Bigger update: The content of this article is now available as a full-length video course that walks ...

  8. [C2P1] Andrew Ng - Machine Learning

    About this Course Machine learning is the science of getting computers to act without being explicit ...

  9. Introduction to Machine Learning

    Chapter 1 Introduction 1.1 What Is Machine Learning? To solve a problem on a computer, we need an al ...

随机推荐

  1. Meerkat软件

    一.准备工作 meerkat 0.189版本和以前的版本相比,支持bwa mem 输出的bam文件,还支持全外显子数据count SV. meerkat原理 1.1 需要准备的软件 unix/Linu ...

  2. hdoj1247（字典树）

    题目链接:https://vjudge.net/problem/HDU-1247 题意:给定n个字符串(n<=50000),判断其中哪些字符串恰能由另外两个不同的字符串连接而成. 思路: 暴力字 ...

  3. shrio学习笔记

    Thymeleaf扩展坐标 <!--thyemleaf对shrio的扩展坐标--> <dependency> <groupId>com.github.thebora ...

  4. Redis--hash类型操作命令

    哈希类型hash redis hash 是一个 string 类型的 field 和 value 的映射表,hash 特别适合用于存储对象 哈希类型hash——基本命令 hset /hget /hms ...

  5. PAT(B) 1027 打印沙漏（Java）

    题目链接:1027 打印沙漏 (20 point(s)) 题目描述 本题要求你写个程序把给定的符号打印成沙漏的形状.例如给定17个"*",要求按下列格式打印 ***** *** * ...

  6. Python字符串图解

    >>> word = "Python" >>> word[:2]  # character from the beginning to posi ...

  7. 【hash】Power Strings

    [题意]: 给出s串出来,能否找到一个前缀 ,通过多次前缀进行拼接.构成s串.如果有多个,请输出最多次数那个. 如:aaaa 可以用1个a,进行4次拼接 可以用2个a,进行2次拼接 可以用4个a,进行 ...

  8. 【字符串hash】DNA

    DNA 题目描述 小X身为奆老,兴趣爱好广泛,他还非常喜欢研究DNA序列……小X进行了一项关于DNA序列研究,发现人某条染色体上的一段DNA序列中连续的k个碱基组成的碱基序列与做题的AC率有关!于是他 ...

  9. 异常:[vue/no-parsing-error] Parsing error:x-invalid-end-tag

  10. Dijkstra+Heap模板

    普通Dijkstra: void DijkstraPath(int v0,int vis[],int dist[],int path[]) { int onePath[maxn]; int d; in ...