PRML读书笔记——Introduction
1.1. Example: Polynomial Curve Fitting
1. Movitate a number of concepts:
(1) linear models: Functions which are linear in the unknow parameters. Polynomail is a linear model. For the Polynomail curve fitting problem, the models is :
which is a linear model.
(2) error function: error function measures the misfit between the prediction and the training set point. For instance, sum of the squares of the errors is one simple function, which is widely used, and is given:
(3) model comparison or model selection
(4) over-fitting: the model abtains excellent fit to training data and give a very poor performance on test data. And this behavior is known as over-fitting.
(5) regularization: One technique which is often used to control the over-fitting phenomenon, and it involves adding a penalty term to the error function in order to discourage the coefficients from reaching large values. The simplest such penalty term takes the form of a sum of aquares of all of the coefficients, leading to a modified error function of the form:
And this particular case of a quadratic regularizer is called ridge regression (Hoerl and Kennard, 1970). In the context of neural networks, this approach is known as weight decay.
(6) validation set, also called a hold-out set: If we were trying to solve a practical application using this approach of minimizing an error function, we would have to find a way to determine a suitable value for the model complexity. a simple way of achieving this, namely by taking the available data and partitioning it into a training set, used to determine the coefficients w, and a separate validation set, also called a hold-out set, used to optimize the model complexity.
1.2. Probability Theory
1. The rules of probability. Sum rule and product rule.
2. Bayes’ theorem.
3. Probability densities
4. Expectations and covariances
5. Bayesian probabilities.
Bayes’ theorem was used to convert a prior probability into a posterior probability by incorporating the evidence provided by the observed data.
6. Gaussian distribution
7.maximizing the posterior distribution is equivalent to minimizing the regularized sum-of-squares error function.
1.3. Model Selection
1.6. Information Theory
1 entropy
PRML读书笔记——Introduction的更多相关文章
- PRML读书笔记——3 Linear Models for Regression
Linear Basis Function Models 线性模型的一个关键属性是它是参数的一个线性函数,形式如下: w是参数,x可以是原始的数据,也可以是关于原始数据的一个函数值,这个函数就叫bas ...
- PRML读书笔记——机器学习导论
什么是模式识别(Pattern Recognition)? 按照Bishop的定义,模式识别就是用机器学习的算法从数据中挖掘出有用的pattern. 人们很早就开始学习如何从大量的数据中发现隐藏在背后 ...
- PRML读书笔记——2 Probability Distributions
2.1. Binary Variables 1. Bernoulli distribution, p(x = 1|µ) = µ 2.Binomial distribution + 3.beta dis ...
- PRML读书笔记——Mathematical notation
x, a vector, and all vectors are assumed to be column vectors. M, denote matrices. xT, a row vcetor, ...
- 【PRML读书笔记-Chapter1-Introduction】1.6 Information Theory
熵 给定一个离散变量,我们观察它的每一个取值所包含的信息量的大小,因此,我们用来表示信息量的大小,概率分布为.当p(x)=1时,说明这个事件一定会发生,因此,它带给我的信息为0.(因为一定会发生,毫无 ...
- 【PRML读书笔记-Chapter1-Introduction】1.5 Decision Theory
初体验: 概率论为我们提供了一个衡量和控制不确定性的统一的框架,也就是说计算出了一大堆的概率.那么,如何根据这些计算出的概率得到较好的结果,就是决策论要做的事情. 一个例子: 文中举了一个例子: 给定 ...
- 【PRML读书笔记-Chapter1-Introduction】1.4 The Curse of Dimensionality
维数灾难 给定如下分类问题: 其中x6和x7表示横轴和竖轴(即两个measurements),怎么分? 方法一(simple): 把整个图分成:16个格,当给定一个新的点的时候,就数他所在的格子中,哪 ...
- 【PRML读书笔记-Chapter1-Introduction】1.3 Model Selection
在训练集上有个好的效果不见得在测试集中效果就好,因为可能存在过拟合(over-fitting)的问题. 如果训练集的数据质量很好,那我们只需对这些有效数据训练处一堆模型,或者对一个模型给定系列的参数值 ...
- 【PRML读书笔记-Chapter1-Introduction】1.2 Probability Theory
一个例子: 两个盒子: 一个红色:2个苹果,6个橘子; 一个蓝色:3个苹果,1个橘子; 如下图: 现在假设随机选取1个盒子,从中.取一个水果,观察它是属于哪一种水果之后,我们把它从原来的盒子中替换掉. ...
随机推荐
- TYVJ P1103 多项式输出 Label:模拟 有点儿坑
描述 一元 n 次多项式可用如下的表达式表示: f(x)=an*x^n+an-1*x^n-1+...+a1*x+a0,an<>0其中,ai*a^x 称为i次项,ai称为i次项的系数.给出 ...
- Android在智能终端领域的关键技术专题讲座(成都站)
Android系统引领了终端智能化的浪潮,在民用.公 共及工业等诸多领域得到了广泛的应用,涉及手持终端.电视.汽车导航.工业控制等,在云计算.设备智能化等方面表现卓越.Android也凭借着自身的优 ...
- JAVA_DES 加密 解密 生成随机密钥
package com.test; import java.io.FileInputStream; import java.io.FileOutputStream; import java.io.In ...
- 20145330《Java程序设计》第二次实验报告
20145330<Java程序设计>第二次实验报告 实验二 Java面向对象程序设计 实验内容 1.初步掌握单元测试和TDD 2.理解并掌握面向对象三要素:封装.继承多态 3.初步掌握UM ...
- SolrCloud-如何在.NET程序中使用
https://github.com/vladen/SolrNet 原来我们在我们的项目里用的是根据数据库路由到不同的单机Solr服务器,但是这样的话,每次Solr配置的修改都要修改三台不通的服务器, ...
- 用GruntJS合并、压缩CSS资源文件
合并.压缩CSS资源文件用到了grunt-contrib-concat.grunt-css插件,自己npm就可以了,下面直接呈上package.json.Gruntfile.js代码 package. ...
- 用atom写LaTeX文档
下载并安装Tex Live: 下载页面 下载并安装atom:下载页面 打开atom File -> Settings -> Install 搜索并安装: language-latex la ...
- [LintCode] Segment Tree Build II 建立线段树之二
The structure of Segment Tree is a binary tree which each node has two attributes startand end denot ...
- 制作本地 odoo deb包安装镜像
[本来这不是个事,可是在阿里云部署的时候,这个网速真是让我无语,本来10分钟就能解决的事,得俩三个小时,太没效率了!] 原文转自 http://www.cnblogs.com/xwdreamer/p/ ...
- Java 中的转义字符
注意斜杠方向,为键盘右上角的斜杠 \t 在当前编辑位置插入一个 tab \b 在当前编辑位置插入一个空格 \n 换行(在当前编辑位置插入 a newline) \r 在当前编辑位置插入一个回车 ...