2.4 statistical decision theory

在讲完最小二乘(linear regression)和K近邻后,进入本节. 引入符号: $X\in R^p$ X为维度为p的输入向量 $Y\in R$ Y为输出,实数 $P(X,Y)$ 为两者的联合概率分布 $f(X)$ 为预测函数,给定X,输出Y a.使用squared error loss(L2)作为损失函数 $L(Y,f(X))={(Y-f(X))}^2$ EPE(excepted prediction error)为 $EPE(f)=E({(Y-f(X))}^2) \\ \ \ =\in…

决策论 | 信息论 | decision theory | information theory

参考: 模式识别与机器学习(一):概率论.决策论.信息论 Decision Theory - Principles and Approaches 英文图书 What are the best beginners books about decision theory? - Quora Statistical Decision Theory 了解一些AI方面的前沿知识!!! 待续~…

【PRML读书笔记-Chapter1-Introduction】1.5 Decision Theory

初体验: 概率论为我们提供了一个衡量和控制不确定性的统一的框架,也就是说计算出了一大堆的概率.那么,如何根据这些计算出的概率得到较好的结果,就是决策论要做的事情. 一个例子: 文中举了一个例子: 给定一个X射线图x,目标是如何判断这个病人是否得癌症(C1或C2).我们把它看作是一个二分类问题,根据bayes的概率理论模型,我们可以得到: 因此,就是的先验概率:(假设Ck表示患病,那么就表示普通人患病的概率) 则作为是后验概率. 假设,我们的目标是:在给定一个x的情况下,我们希望最小化误分类的概率…

[PR & ML 5] [Introduction] Decision Theory

…

统计决策——贝叶斯决策理论（Bayesian Decision Theory）

(本文为原创学习笔记,主要参考<模式识别(第三版)>(张学工著,清华大学出版社出版)) 1.概念将分类看做决策,进行贝叶斯决策时考虑各类的先验概率和类条件概率,也即后验概率.考虑先验概率意味着对样本总体的认识,考虑类条件概率是对每一类中某个特征出现频率的认识.由此不难发现,贝叶斯决策的理论依据就是贝叶斯公式. 2.理论依据 2.1 最小错误率贝叶斯决策贝叶斯决策的基本理论依据就是贝叶斯公式(式1),由总体密度P(E).先验概率P(H)和类条件概率P(E|H)计算出后验概率P(H|E),判决…

Random/Stochastic

---恢复内容开始--- ===================================================== A random variable's possible values might represent the possible outcomes of a yet-to-be-performed experiment, or the possible outcomes of a past experiment whose already-existing va…

机器学习经典书籍&论文

原文地址:http://blog.sina.com.cn/s/blog_7e5f32ff0102vlgj.html 入门书单 1.<数学之美>PDF6 作者吴军大家都很熟悉.以极为通俗的语言讲述了数学在机器学习和自然语言处理等领域的应用. 2.<Programming Collective Intelligence>(<集体智慧编程>)PDF3 作者Toby Segaran也是<BeautifulData : The Stories Behind Elegant…

Image Processing and Analysis_8_Edge Detection：Statistical edge detection_ learning and evaluating edge cues——2003

此主要讨论图像处理与分析.虽然计算机视觉部分的有些内容比如特征提取等也可以归结到图像分析中来,但鉴于它们与计算机视觉的紧密联系,以及它们的出处,没有把它们纳入到图像处理与分析中来.同样,这里面也有一些也可以划归到计算机视觉中去.这都不重要,只要知道有这么个方法,能为自己所用,或者从中得到灵感,这就够了. 8. Edge Detection 边缘检测也是图像处理中的一个基本任务.传统的边缘检测方法有基于梯度算子,尤其是 Sobel 算子,以及经典的 Canny 边缘检测.到现在,Cann…

【原】对频率论（Frequentist）方法和贝叶斯方法（Bayesian Methods）的一个总结

注: 本文是对<IPython Interactive Computing and Visualization Cookbook>一书中第七章[Introduction to statistical data analysis in Python – frequentist and Bayesian methods]的简单翻译和整理,这部分内容主要将对统计学习中的频率论方法和贝叶斯统计方法进行介绍. 本文将介绍如何洞察现实世界的数据,以及如何在存在不确定性的情况下做出明智的决定. 统计数据分析…

R Language

向量定义:x1 = c(1,2,3); x2 = c(1:100) 类型显示:mode(x1) 向量长度:length(x2) 向量元素显示:x1[c(1,2,3)] 多维向量:multi-dimensional vector:rbind(x1,x2); cbind(x1,x2) > x = c(1,2,3,4,5,6) > y = c(6,5,4,3,2,1) > z = rbind(x,y) > z [,1] [,2] [,3] [,4] [,5] [,6] x 1 2 3 4…

FAQ: Machine Learning: What and How

What: 就是将统计学算法作为理论,计算机作为工具,解决问题.statistic Algorithm. How: 如何成为菜鸟一枚? http://www.quora.com/How-can-a-beginner-train-for-machine-learning-contests 链接内容总结: "学习任何一门学科,framework是必不可少的东西.没有framework的东西,那是研究." -- Jason Hawk One thing is for sure; you ca…

Introduction to Machine Learning

Chapter 1 Introduction 1.1 What Is Machine Learning? To solve a problem on a computer, we need an algorithm. An algorithm is a sequence of instructions that should be carried out to transform the input to output. For example, one can devise an algori…

CV code references

转:http://www.sigvc.org/bbs/thread-72-1-1.html 一.特征提取Feature Extraction: SIFT [1] [Demo program][SIFT Library] [VLFeat] PCA-SIFT [2] [Project] Affine-SIFT [3] [Project] SURF [4] [OpenSURF] [Matlab Wrapper] Affine Covariant Features [5] [Oxfo…

How do I learn machine learning?

https://www.quora.com/How-do-I-learn-machine-learning-1?redirected_qid=6578644 How Can I Learn X? Learning Machine Learning Learning About Computer Science Educational Resources Advice Artificial Intelligence How-to Question Learning New Things Lea…

学习笔记之Machine Learning Crash Course | Google Developers

Machine Learning Crash Course | Google Developers https://developers.google.com/machine-learning/crash-course/ Google's fast-paced, practical introduction to machine learning ML Concepts Introduction to Machine Learning As you'll discover, machine…

PMP模拟考试-2

1. Increasing resources on the critical path activities may not always shorten the length of the project with the following reasons or examples .which may bring the LEAST influence to resource increase ? A certain activities are time dependent rather…

SVM总结（转）

支持向量机看了JULY君的博客和文档后,个人对SVM的理解和总结,欢迎交流和指正.其理论部分可以查看下面文档链接,通俗易懂. 支持向量机通俗导论(理解SVM的三层境界) 第一篇:从四个关键词理解SVM 第二篇:SVM的原理(全面理解SVM) 第三篇:SVM的特点与不足第四篇:SVM实现第五篇:从应用上理解SVM 第一篇:从四个关键词理解SVM 理解支持向量机SVM(Support Vector Machine)有四个关键名词:分离超平面.最大边缘超平面.软边缘.核函数. 分离超平面…

机器学习经典论文/survey合集

Active Learning Two Faces of Active Learning, Dasgupta, 2011 Active Learning Literature Survey, Settles, 2010 Applications A Survey of Emerging Approaches to Spam Filtering, Caruana, 2012 Ambient Intelligence: A Survey, Sadri, 2011 A Survey of Online…

100 Most Popular Machine Learning Video Talks

100 Most Popular Machine Learning Video Talks 26971 views, 1:00:45, Gaussian Process Basics, David MacKay, 8 comments 7799 views, 3:08:32, Introduction to Machine Learning, Iain Murray 16092 views, 1:28:05, Introduction to Support Vector Machines, C…

[ML] I'm back for Machine Learning

Hi, Long time no see. Briefly, I plan to step into this new area, data analysis. In the past few years, I have tried Linux programming, device driver development, android application development and RF SOC development. Thus, "data analysis become my…

AI人工智能专业词汇集

作为最早关注人工智能技术的媒体,机器之心在编译国外技术博客.论文.专家观点等内容上已经积累了超过两年多的经验.期间,从无到有,机器之心的编译团队一直在积累专业词汇.虽然有很多的文章因为专业性我们没能尽善尽美的编译为中文呈现给大家,但我们一直在进步.一直在积累.一直在提高自己的专业性.两年来,机器之心编译团队整理过翻译词汇对照表「红宝书」,编辑个人也整理过类似的词典.而我们也从机器之心读者留言中发现,有些人工智能专业词汇没有统一的翻译标准,这可能是因地区.跨专业等等原因造成的.举个例子,DeepM…

Machine Learning and Data Mining（机器学习与数据挖掘）

Problems[show] Classification Clustering Regression Anomaly detection Association rules Reinforcement learning Structured prediction Feature engineering Feature learning Online learning Semi-supervised learning Unsupervised learning Learning to rank…

CV codes代码分类整理合集《转》

from:http://www.sigvc.org/bbs/thread-72-1-1.html 一.特征提取Feature Extraction: SIFT [1] [Demo program][SIFT Library] [VLFeat] PCA-SIFT [2] [Project] Affine-SIFT [3] [Project] SURF [4] [OpenSURF] [Matlab Wrapper] Affine Covariant Features [5] [O…

Bayesian machine learning

from: http://www.metacademy.org/roadmaps/rgrosse/bayesian_machine_learning Created by: Roger Grosse(http://www.cs.toronto.edu/~rgrosse/) Intended for: beginning machine learning researchers, practitioners Bayesian statistics is a branch of statistics…