solving the problem of overfitting:regularization

  • 发生的在linear regression上面的overfitting问题

  • 发生在logistic regression上面的overfitting

  • 怎么解决overfitting

  • regularization: cost function of linear regression

    • parameters小的话,这样hypothesis就会变得简单,这样就不会overfitting
    • 一般不会对θ0进行regularization
    • 上式是进行regularization的linear regression的cost function,要使上式的值取最小值
  •  对这个cost function 的分析

    • 由两个式子(两个目标)组成,第一个式子是为了对trainning data更好的拟合(fitting the training data),第二个式子是为了避免overfitting
    • 第二个式子叫regularization term, λ叫regularization parameter, λ是为了平衡两个目标用的
    • 如果 λ非常大的话(这时θ1n几乎为0,hypothesis变得很简单,只有常数),就会出现underfitting,对trainning data/ new data很低的fitting
    • 所以并不是regularization在任何情况下(当 λ非常大的情况下),都能使model更适应new data或者training data
    • The regularization term puts a penalty on the cost J,随着模型参数的增多,the penalty increases as well.

machine learning(13) -- solving the problem of overfitting:regularization的更多相关文章

  1. Solving the Problem of Overfitting

    The Problem of Overfitting Cost Function Regularized Linear Regression Note: [8:43 - It is said that ...

  2. Advice for applying Machine Learning

    https://jmetzen.github.io/2015-01-29/ml_advice.html Advice for applying Machine Learning This post i ...

  3. How do I learn mathematics for machine learning?

    https://www.quora.com/How-do-I-learn-mathematics-for-machine-learning   How do I learn mathematics f ...

  4. [C2P2] Andrew Ng - Machine Learning

    ##Linear Regression with One Variable Linear regression predicts a real-valued output based on an in ...

  5. Machine Learning - 第3周(Logistic Regression、Regularization)

    Logistic regression is a method for classifying data into discrete outcomes. For example, we might u ...

  6. Course Machine Learning Note

    Machine Learning Note Introduction Introduction What is Machine Learning? Two definitions of Machine ...

  7. 【Machine Learning is Fun!】1.The world’s easiest introduction to Machine Learning

    Bigger update: The content of this article is now available as a full-length video course that walks ...

  8. [C2P1] Andrew Ng - Machine Learning

    About this Course Machine learning is the science of getting computers to act without being explicit ...

  9. Introduction to Machine Learning

    Chapter 1 Introduction 1.1 What Is Machine Learning? To solve a problem on a computer, we need an al ...

随机推荐

  1. pycharm 提示:this license **** has been cancelled(2)

    pycharm安装激活过程中,提示 this license **** has been cancelled .这个问题并不是你的激活码不对,而是需要修改系统的hosts文件,下面详细讲解下如何修改h ...

  2. linux 下mongo 基础配置

    配置文件dbpath=/usr/local/mongodb/dblogpath=/usr/local/mongodb/log/mongo.logport=27017fork=truejournal=f ...

  3. 将自己的项目作为jar包发布到maven中央仓库

    maven版本是3.5.0,jdk是1.8(注意,不是说项目是1.8就行,必须是环境变量里的也是,不能超过1.8,否则一大堆问题,执行mvn前用javac -version看下版本) 一:先在sona ...

  4. LeetCode 897. 递增顺序查找树(Increasing Order Search Tree)

    897. 递增顺序查找树 897. Increasing Order Search Tree 题目描述 给定一个树,按中序遍历重新排列树,使树中最左边的结点现在是树的根,并且每个结点没有左子结点,只有 ...

  5. Spring Boot开启Druid数据库监控功能

    Druid是一个关系型数据库连接池,它是阿里巴巴的一个开源项目.Druid支持所有JDBC兼容的数据库,包括Oracle.MySQL.Derby.PostgreSQL.SQL Server.H2等.D ...

  6. Visual Studio 设置为 UTF-8 保存格式

    1. 步骤 Visual Studio 提供高级保存选项功能,它能指定特定代码文件的编码规范和行尾所使用的换行符. 在 VS 中,该命令没有默认显示在「文件」菜单中.用户需要手工设置,才能显示该命令. ...

  7. 题解 CF1216A 【Prefixes】

    题目大意:给你一个长度为$n$($n$为偶数)的字符串,且这个字符串仅由$'a'$与$'b'$两种字符组成,要你用最少的操作次数使得 $s[i]!=s[i-1] (i/2=0,1≤i≤n)$(若字符串 ...

  8. STM32F103芯片SPI控制NRF24L012.4G无线模块交互通信实验

    1.NRF24L01模块的资料百度搜索一下就有很多.这里我就不做介绍本文主要侧重于应用层面实验介绍与分享. 2.先看下原理图. 根据原理图:写出NRF24L01  C语言驱动文件如下: #includ ...

  9. Apache Rewrite 规则详解知识大全

    Rewrite是一种服务器的重写脉冲技术,它可以使得服务器可以支持 URL 重写,是一种最新流行的服务器技术.它还可以实现限制特定IP访问网站的功能. 1.Rewrite标志 R[=code](for ...

  10. Python--遍历文件夹下所有文件和目录的方法(os.walk(rootdir)函数返回一个三元素元祖)

    import os import os.path # This folder is custom rootdir = '/Users/macbookpro/Desktop/test' for pare ...