关于one-hot encoding思考
Many learning algorithms either learn a single weight per feature, or they use distances between samples. The former is the case for linear models such as logistic regression, which are easy to explain.
Suppose you have a dataset having only a single categorical feature "nationality", with values "UK", "French" and "US". Assume, without loss of generality, that these are encoded as 0, 1 and 2. You then have a weight w for this feature in a linear classifier, which will make some kind of decision based on the constraint w×x + b > 0, or equivalently w×x < b.
The problem now is that the weight w cannot encode a three-way choice. The three possible values of w×x are 0, w and 2×w. Either these three all lead to the same decision (they're all < b or ≥b) or "UK" and "French" lead to the same decision, or "French" and "US" give the same decision. There's no possibility for the model to learn that "UK" and "US" should be given the same label, with "French" the odd one out.(二分类问题,若dummy encoding,us和uk始终不能单独成为一类,而若one-hot encoding,则可以适应任何情况)
By one-hot encoding, you effectively blow up the feature space to three features, which will each get their own weights, so the decision function is now w[UK]x[UK] + w[FR]x[FR] + w[US]x[US] < b, where all the x's are booleans. In this space, such a linear function can express any sum/disjunction of the possibilities (e.g. "UK or US", which might be a predictor for someone speaking English).
Similarly, any learner based on standard distance metrics (such as k-nearest neighbors) between samples will get confused without one-hot encoding. With the naive encoding and Euclidean distance, the distance between French and US is 1. The distance between US and UK is 2. But with the one-hot encoding, the pairwise distances between [1, 0, 0], [0, 1, 0] and [0, 0, 1] are all equal to √2.
This is not true for all learning algorithms; decision trees and derived models such as random forests, if deep enough, can handle categorical variables without one-hot encoding.
dataframe one-hot encoding:pandas.get_dummies方法
参考:
https://gist.github.com/ramhiser/982ce339d5f8c9a769a0
http://pandas.pydata.org/pandas-docs/version/0.17.0/generated/pandas.get_dummies.html
关于one-hot encoding思考的更多相关文章
- 关于.NET参数传递方式的思考
年关将近,整个人已经没有了工作和写作的激情,估计这个时候很多人跟我差不多,该相亲的相亲,该聚会喝酒的聚会喝酒,总之就是没有了干活的心思(我有很多想法,但就是叫不动我的手脚,所以我只能看着别人在做我想做 ...
- 关于过拟合、局部最小值、以及Poor Generalization的思考
Poor Generalization 这可能是实际中遇到的最多问题. 比如FC网络为什么效果比CNN差那么多啊,是不是陷入局部最小值啊?是不是过拟合啊?是不是欠拟合啊? 在操场跑步的时候,又从SVM ...
- Spring之LoadTimeWeaver——一个需求引发的思考---转
原文地址:http://www.myexception.cn/software-architecture-design/602651.html Spring之LoadTimeWeaver——一个需求引 ...
- 关于学习是UIWebView的一些思考
前几天因为数据中加载有html语言的数据,关于html语言和UIWebView,有一些纠结,经过几天的研究,也有了一些自己的简单的见解. 我有两个页面需要加载html语言(注意,这里 ...
- Python--Cmd窗口运行Python时提示Fatal Python error: Py_Initialize: can't initialize sys standard streams LookupError: unknown encoding: cp65001
源地址连接: http://www.tuicool.com/articles/ryuaUze 最近,我在把一个 Python 2 的视频下载工具 youku-lixian 改写成 Python 3,并 ...
- 基于纯Java代码的Spring容器和Web容器零配置的思考和实现(3) - 使用配置
经过<基于纯Java代码的Spring容器和Web容器零配置的思考和实现(1) - 数据源与事务管理>和<基于纯Java代码的Spring容器和Web容器零配置的思考和实现(2) - ...
- file.encoding到底指的是什么呢?
转载请注明来源:http://blog.csdn.net/loongshawn/article/details/50918506 <Java利用System.getProperty(“file. ...
- Java 小记 — Spring Boot 的实践与思考
前言 本篇随笔用于记录我在学习 Java 和构建 Spring Boot 项目过程中的一些思考,包含架构.组件和部署方式等.下文仅为概要,待闲时逐一整理为详细文档. 1. 组件 开源社区如火如荼,若在 ...
- Android图表库MPAndroidChart(六)——换一种思考方式,水平条形图的实现过程
Android图表库MPAndroidChart(六)--换一种思考方式,水平条形图的实现过程 一.基本实现 我们之前实现了条形图,现在来看下水平条形图是怎么实现的,说白了就是横起来,看下效果: 说起 ...
随机推荐
- (57)zabbix Slide shows幻灯片展示
定义好screen之后,我们想了解服务器状况之时,一般会一个个screen点过去,zabbix提供了幻灯片展示方法,可以定义多个Slide,一个slide中可以包含多个screen. 创建slide ...
- docker的网络(进阶)
overlay网络 overlay网络驱动程序会在多个docker守护程序(即多个主机上的docker守护程序)之间创建分布式网络.该网络(overlays)位于特定于主机的网络之上,允许连接到它的容 ...
- CSS3-文本-text-shadow
一.text-shadow 语法: text-shadow : none | <length> none | [<shadow>, ] * <shadow> 或no ...
- js,jq,php使用正则方法
1.js使用正则表达式案例: <script> var str=”543535364565@qq.com”; var res=“/^\d*@(QQ|qq|136)\.(com|cn)$/” ...
- Ubuntu桌面主题设置以及优化
安装好Ubuntu后,觉得桌面不太美观,便动了修改主题的想法.听说Flatabulous不错,在网上搜索看过主题效果后也觉得蛮不错的,于是准备修改. 安装Unity Tweak Tool Unity ...
- python基础——10(三元运算符、匿名函数)
一.三元运算符 本质是if--else--的语法糖 前提:简化if--else--的结构,且两个分支有且只有一条语句 案例: a = 20 b = 30 res = a if a > b els ...
- 如何反馈问题issue?
如何反馈问题issue? 01,请提交的时候换位思考一下:如果别人给你提交一个这样的 Issue,你能快速准确的理解吗?如果不能,烦请重新整理你的语言,按照要求的格式填写.专业一点,减少不必要的口舌浪 ...
- Builder(构造者)
Builder(构造者) <?php class Product { private $name; public function setName($name) { $this->name ...
- 在 Yii2 项目中使用 Composer 添加 FontAwesome 字体资源
2014-06-21 19:05 原文 简体 繁體 2,123 次围观 前天帮同事改个十年前的网站 bug,页面上一堆 include require 不禁让人抱头痛哭.看到 V2EX 上的讨论说,写 ...
- 总结搭建Oracle11g DG踩的坑
此次的操作环境是Oracle11g 单实例,os为Linux,采用duplicate在线创建物理备库 primary上设置相关参数 ALTER SYSTEM SET LOG_ARCHIVE_CONFI ...