solving the problem of overfitting:regularization

  • 发生的在linear regression上面的overfitting问题

  • 发生在logistic regression上面的overfitting

  • 怎么解决overfitting

  • regularization: cost function of linear regression

    • parameters小的话，这样hypothesis就会变得简单，这样就不会overfitting
    • 一般不会对θ0进行regularization
    • 上式是进行regularization的linear regression的cost function,要使上式的值取最小值
  •  对这个cost function 的分析

    • 由两个式子(两个目标)组成，第一个式子是为了对trainning data更好的拟合（fitting the training data），第二个式子是为了避免overfitting
    • 第二个式子叫regularization term, λ叫regularization parameter, λ是为了平衡两个目标用的
    • 如果 λ非常大的话(这时θ1-θn几乎为0,hypothesis变得很简单，只有常数)，就会出现underfitting,对trainning data/ new data很低的fitting
    • 所以并不是regularization在任何情况下(当 λ非常大的情况下),都能使model更适应new data或者training data
    • The regularization term puts a penalty on the cost J，随着模型参数的增多，the penalty increases as well.

machine learning(13) -- solving the problem of overfitting:regularization的更多相关文章

  1. Solving the Problem of Overfitting

    The Problem of Overfitting Cost Function Regularized Linear Regression Note: [8:43 - It is said that ...

  2. Advice for applying Machine Learning

    https://jmetzen.github.io/2015-01-29/ml_advice.html Advice for applying Machine Learning This post i ...

  3. How do I learn mathematics for machine learning?

    https://www.quora.com/How-do-I-learn-mathematics-for-machine-learning   How do I learn mathematics f ...

  4. [C2P2] Andrew Ng - Machine Learning

    ##Linear Regression with One Variable Linear regression predicts a real-valued output based on an in ...

  5. Machine Learning - 第3周（Logistic Regression、Regularization）

    Logistic regression is a method for classifying data into discrete outcomes. For example, we might u ...

  6. Course Machine Learning Note

    Machine Learning Note Introduction Introduction What is Machine Learning? Two definitions of Machine ...

  7. 【Machine Learning is Fun!】1.The world’s easiest introduction to Machine Learning

    Bigger update: The content of this article is now available as a full-length video course that walks ...

  8. [C2P1] Andrew Ng - Machine Learning

    About this Course Machine learning is the science of getting computers to act without being explicit ...

  9. Introduction to Machine Learning

    Chapter 1 Introduction 1.1 What Is Machine Learning? To solve a problem on a computer, we need an al ...

随机推荐

  1. PHP比较两个版本的方法

    PHP比较两个版本的方法 写app接口经常要用到 PHP接口的版本和app里面的版本做比较 如果app版本低于接口版本 那么就要提示他升级 <pre><?php/** * 版本处理类 ...

  2. Transformer的numpy实现

    下面的代码自下而上的实现Transformer的相关模块功能.这份文档只实现了主要代码.由于时间关系,我无法实现所有函数.对于没有实现的函数,默认用全大写函数名指出,如SOFTMAX 由于时间限制,以 ...

  3. show processlist说明

    mysql> show processlist; 有时候输出太多,可以用下面的语句代替 mysql> select * from information_schema.processlis ...

  4. Webpack将静态资源拷贝并压缩至输出文件夹

    就拿Vue项目来说,比如要将src/assets/js下的静态js文件,直接在public/index.html中引用: 这时候没有在项目中引用,不会经过wenpack的loader,也就不会自己打包 ...

  5. PHP Math常量

    常量名 常量名 常量值 PHP M_E e 2.7182818284590452354 4 M_EULER Euler 常量 0.57721566490153286061 5.2.0 M_LNPI l ...

  6. 使用dockers安装MySQL

    事前准备 关闭selinux setenforce 0 vim /etc/sysconfig/selinux SELINUX=disabled # 若不关闭,使用docker启动mysql5.7镜像容 ...

  7. docker 启动 容器----bootstrap checks failed

    错误信息: bootstrap checks failed 解决方法: 1.修改elasticsearch.yml配置文件,允许外网访问. vim config/elasticsearch.yml,增 ...

  8. JavaDoc工具和Ideade javadoc工具

    命令参考: javadoc -locale zh_CN -protected -notree -nonavbar -noindex -use -author -version -encoding UT ...

  9. Snort Rule Infographic

    Snort Rule Infographic Official Documentation Snort FAQ  Snort Team / Open Source Community Snort Us ...

  10. Python 简单web服务器的实现

    import re import socket def service_cilent(new_socket): request = new_socket.recv(1024).decode(" ...