More 3D Graphics (rgl) for Classification with Local Logistic Regression and Kernel Density Estimates (from The Elements of Statistical Learning)(转)
This post builds on a previous post, but can be read and understood independently.
As part of my course on statistical learning, we created 3D graphics to foster a more intuitive understanding of the various methods that are used to relax the assumption of linearity (in the predictors) in regression and classification methods.
The authors of our text (The Elements of Statistical Learning, 2nd Edition) provide a Mixture Simulation data set that has two continuous predictors and a binary outcome. This data is used to demonstrate classification procedures by plotting classification boundaries in the two predictors, which are determined by one or more surfaces (e.g., a probability surface such as that produced by logistic regression, or multiple intersecting surfaces as in linear discriminant analysis). In our class laboratory, we used the R package rgl to create a 3D representation of these surfaces for a variety of semiparametric classification procedures.
Chapter 6 presents local logistic regression and kernel density classification, among other kernel (local) classification and regression methods. Below is the code and graphic (a 2D projection) associated with the local linear logistic regression in these data:
library(rgl)
load(url("http://statweb.stanford.edu/~tibs/ElemStatLearn/datasets/ESL.mixture.rda"))
dat <- ESL.mixture
ddat <- data.frame(y=dat$y, x1=dat$x[,1], x2=dat$x[,2]) ## create 3D graphic, rotate to view 2D x1/x2 projection
par3d(FOV=1,userMatrix=diag(4))
plot3d(dat$xnew[,1], dat$xnew[,2], dat$prob, type="n",
xlab="x1", ylab="x2", zlab="",
axes=FALSE, box=TRUE, aspect=1) ## plot points and bounding box
x1r <- range(dat$px1)
x2r <- range(dat$px2)
pts <- plot3d(dat$x[,1], dat$x[,2], 1,
type="p", radius=0.5, add=TRUE,
col=ifelse(dat$y, "orange", "blue"))
lns <- lines3d(x1r[c(1,2,2,1,1)], x2r[c(1,1,2,2,1)], 1) ## draw Bayes (True) classification boundary in blue
dat$probm <- with(dat, matrix(prob, length(px1), length(px2)))
dat$cls <- with(dat, contourLines(px1, px2, probm, levels=0.5))
pls0 <- lapply(dat$cls, function(p) lines3d(p$x, p$y, z=1, color="blue")) ## compute probabilities plot classification boundary
## associated with local linear logistic regression
probs.loc <-
apply(dat$xnew, 1, function(x0) {
## smoothing parameter
l <- 1/2
## compute (Gaussian) kernel weights
d <- colSums((rbind(ddat$x1, ddat$x2) - x0)^2)
k <- exp(-d/2/l^2)
## local fit at x0
fit <- suppressWarnings(glm(y ~ x1 + x2, data=ddat, weights=k,
family=binomial(link="logit")))
## predict at x0
as.numeric(predict(fit, type="response", newdata=as.data.frame(t(x0))))
}) dat$probm.loc <- with(dat, matrix(probs.loc, length(px1), length(px2)))
dat$cls.loc <- with(dat, contourLines(px1, px2, probm.loc, levels=0.5))
pls <- lapply(dat$cls.loc, function(p) lines3d(p$x, p$y, z=1)) ## plot probability surface and decision plane
sfc <- surface3d(dat$px1, dat$px2, probs.loc, alpha=1.0,
color="gray", specular="gray")
qds <- quads3d(x1r[c(1,2,2,1)], x2r[c(1,1,2,2)], 0.5, alpha=0.4,
color="gray", lit=FALSE)
In the above graphic, the solid blue line represents the true Bayes decision boundary (i.e., {x: Pr("orange"|x) = 0.5}), which is computed from the model used to simulate these data. The probability surface (generated by the local logistic regression) is represented in gray, and the corresponding Bayes decision boundary occurs where the plane f(x) = 0.5 (in light gray) intersects with the probability surface. The solid black line is a projection of this intersection. Here is a link to the interactive version of this graphic: local logistic regression.
Below is the code and graphic associated with the kernel density classification (note: this code below should only be executed after the above code, since the 3D graphic is modified, rather than created anew):
## clear the surface, decision plane, and decision boundary
pop3d(id=sfc); pop3d(id=qds)
for(pl in pls) pop3d(id=pl) ## kernel density classification
## compute kernel density estimates for each class
dens.kde <-
lapply(unique(ddat$y), function(uy) {
apply(dat$xnew, 1, function(x0) {
## subset to current class
dsub <- subset(ddat, y==uy)
## smoothing parameter
l <- 1/2
## kernel density estimate at x0
mean(dnorm(dsub$x1-x0[1], 0, l)*dnorm(dsub$x2-x0[2], 0, l))
})
}) ## compute prior for each class (sample proportion)
prir.kde <- table(ddat$y)/length(dat$y) ## compute posterior probability Pr(y=1|x)
probs.kde <- prir.kde[2]*dens.kde[[2]]/
(prir.kde[1]*dens.kde[[1]]+prir.kde[2]*dens.kde[[2]]) ## plot classification boundary associated
## with kernel density classification
dat$probm.kde <- with(dat, matrix(probs.kde, length(px1), length(px2)))
dat$cls.kde <- with(dat, contourLines(px1, px2, probm.kde, levels=0.5))
pls <- lapply(dat$cls.kde, function(p) lines3d(p$x, p$y, z=1)) ## plot probability surface and decision plane
sfc <- surface3d(dat$px1, dat$px2, probs.kde, alpha=1.0,
color="gray", specular="gray")
qds <- quads3d(x1r[c(1,2,2,1)], x2r[c(1,1,2,2)], 0.5, alpha=0.4,
color="gray", lit=FALSE)
Here are links to the interactive versions of both graphics: local logistic regression, kernel density classification
This entry was posted in Technical and tagged data, graphics, programming, R, statistics on February 7, 2015.
More 3D Graphics (rgl) for Classification with Local Logistic Regression and Kernel Density Estimates (from The Elements of Statistical Learning)(转)的更多相关文章
- Some 3D Graphics (rgl) for Classification with Splines and Logistic Regression (from The Elements of Statistical Learning)(转)
This semester I'm teaching from Hastie, Tibshirani, and Friedman's book, The Elements of Statistical ...
- 李宏毅机器学习笔记3:Classification、Logistic Regression
李宏毅老师的机器学习课程和吴恩达老师的机器学习课程都是都是ML和DL非常好的入门资料,在YouTube.网易云课堂.B站都能观看到相应的课程视频,接下来这一系列的博客我都将记录老师上课的笔记以及自己对 ...
- Logistic Regression Using Gradient Descent -- Binary Classification 代码实现
1. 原理 Cost function Theta 2. Python # -*- coding:utf8 -*- import numpy as np import matplotlib.pyplo ...
- Classification week2: logistic regression classifier 笔记
华盛顿大学 machine learning: Classification 笔记. linear classifier 线性分类器 多项式: Logistic regression & 概率 ...
- Classification and logistic regression
logistic 回归 1.问题: 在上面讨论回归问题时.讨论的结果都是连续类型.但假设要求做分类呢?即讨论结果为离散型的值. 2.解答: 假设: 当中: g(z)的图形例如以下: 由此可知:当hθ( ...
- Android Programming 3D Graphics with OpenGL ES (Including Nehe's Port)
https://www3.ntu.edu.sg/home/ehchua/programming/android/Android_3D.html
- Logistic Regression and Classification
分类(Classification)与回归都属于监督学习,两者的唯一区别在于,前者要预测的输出变量\(y\)只能取离散值,而后者的输出变量是连续的.这些离散的输出变量在分类问题中通常称之为标签(Lab ...
- Logistic Regression求解classification问题
classification问题和regression问题类似,区别在于y值是一个离散值,例如binary classification,y值只取0或1. 方法来自Andrew Ng的Machine ...
- 分类和逻辑回归(Classification and logistic regression)
分类问题和线性回归问题问题很像,只是在分类问题中,我们预测的y值包含在一个小的离散数据集里.首先,认识一下二元分类(binary classification),在二元分类中,y的取值只能是0和1.例 ...
随机推荐
- Spring Boot 学习笔记--整合Redis
1.新建Spring Boot项目 添加spring-boot-starter-data-redis依赖 <dependency> <groupId>org.springfra ...
- idea 集成sonarLint
1.目标 idea集成sonar的代码检查,实现可以在提交代码前就检查你的代码,而不是将代码提交之后,之后再去检查. Sonar可以从以下七个维度检测代码质量,而作为开发人员至少需要处理前5种代码质量 ...
- Elasticsearch1.7服务搭建与入门操作
ElasticSearch是一个基于Lucene的搜索服务器.它提供了一个分布式多用户能力的全文搜索引擎,基于RESTful web接口.Elasticsearch是用Java开发的,并作为Apach ...
- Ubuntu下安装Tomcat7
第一部分:基本安装 1.打开http://tomcat.apache.org/download-70.cgi,下载apache-tomcat-7.0.68.zip. 2.拷贝至合适位置,如/usr/l ...
- 分享一个随机更改 MAC地址 软件
有些软件 是根据 MAC地址 来判断 是不是 已经 安装过 这个 软件 (针对 有些软件是 可以 免费 使用的 ) 要想 一直 使用 的话 只需要 修改一下 mac地址 就可以 继续 使用! 在百度中 ...
- Array和ArrayCollection作为数据源的一个应用区别
在不用[Enabled]元标签的前提下,将一个Array赋值给DataGrid.DataList等控件的DataProvider后,当Array值发生改变时,控件显示内容不会及时更新(可调用控件的in ...
- Block Token 原理分析
介绍 文件权限检查由NameNode执行,而不是DataNode执行. 默认情况下,任何客户端都可以访问只有其块ID的任何块. 为了解决这个问题,Hadoop引入了块访问令牌的概念. 块访问令牌由Na ...
- 【Vue 入门】使用 Vue2 开发一个展示项目列表的应用
前言 一直没有找到一个合适的展示个人项目的模板,所以自己动手使用 Vue 写了一个.该模板基于 Markdown 文件进行配置,只需要按一定规则编写 Markdown 文件,然后使用一个 在线工具 转 ...
- 【国家集训队2012】tree(伍一鸣)
Description 一棵n个点的树,每个点的初始权值为1.对于这棵树有q个操作,每个操作为以下四种操作之一: + u v c:将u到v的路径上的点的权值都加上自然数c: - u1 v1 u2 ...
- 如果非的让我引起大家的狂拍,那我说一句PHP是世界上最好的语言?
如果你进来了,那就达到了我的目的. 不知不觉.net已经用了3年之久,从最初的的小白到现在的大白,总有一种要骂娘的冲动,.net这一路走来现在越走越迷茫,不知道微软的重心在哪里,一直发现不了他的亮点所 ...