机器学习中离散特征的处理方法

Updated: August 25, 2016

Learning with counts is an efficient way to create a compact set of features for a dataset, based on counts of the values. You can use the modules in this section to build a set of counts and features, and later update the counts and the features to take advantage of new data, or merge two sets of count data.

The basic idea underlying count-based featurization is simple: by calculating counts, you can quickly and easily get a summary of what columns contain the most important information. The module counts the number of times a value appears, and then provides that information as a feature for input to a model.

Example of Count-Based Learning

 

Imagine you’re trying to validate a credit card transaction. One crucial piece of information is where this transaction came from, and one of the most common encodings of that location is the postal code. However, there might be as many as 40,000 postal codes, zip codes, and geographical codes to account for. Does your model have the capacity to learn 40,000 more parameters? If you give it that capacity, do you now have enough training data to prevent it from overfitting?

If you had really good data with lots of samples, such fine-grained local granularity could be quite powerful. However, if you have only one sample of a fraudulent transaction from a small locality, does it mean that all of the transactions from that place are bad, or that you don’t have enough data?

One solution to this conundrum is to learn with counts. That is, rather than introduce 40,000 more features, you can observe the counts and proportions of fraud for each postal code. By using these counts as features, you gain a notion of the strength of the evidence for each value. Moreover, by encoding the relevant statistics of the counts, the learner can use the statistics to decide when to back off and use other features.

Count-based learning is very attractive for many reasons: You have fewer features, requiring fewer parameters, which makes for faster learning, faster prediction, smaller predictors, and less potential to overfit.

How Counts are Created

 

An example might help to demonstrate how count-based features are created and applied. This example is highly simplified, to give you an idea of the overall process, and how to use and interpret count-based features.

Suppose you have a table like this, with labels and inputs:

Label column

Input value

0

A

0

A

1

A

0

B

1

B

1

B

1

B

Here is how count-based features are created:

  1. Each case (or row, or sample) has a set of values in columns.

    Here, the values are A, B, and so forth.

  2. For a particular set of values, you find all the other cases in that dataset that have the same value.

    In this case, there are three instances of A and four of B.

  3. Next, you count their class memberships as features in themselves.

    In this case, you get a small matrix, in which there are 2 cases where A=0, 1 case where A = 1, 1 case where B= 0, and 3 cases where B = 1.

When you create features based on this matrix, you get a variety of count-based features, including a calculation of the log-odds ratio as well as the counts for each target class:

Label

0_0_Class000_Count

0_0_Class001_Count

0_0_Class000_LogOdds

0_0_IsBackoff

0

2

1

0.510826

0

0

2

1

0.510826

0

1

2

1

0.510826

0

0

1

3

-0.8473

0

1

1

3

-0.8473

0

1

1

3

-0.8473

0

1

1

3

-0.8473

0

Examples

 

The following article from the Microsoft Machine Learning team provides a detailed walkthrough of how to use counts in machine learning, and compares the efficacy of count-based modeling with other methods.

Using Azure ML to Build Clickthrough Prediction Models

Technical Notes

 
  • How is the log-loss value calculated?

    The Log-loss value is not the plain log-odds; the prior distribution is used to smooth the log-odds computation.

    Suppose you have a data set used for binary classification. In this dataset, the prior frequency for class 0 is p_0, and the prior frequency for class 1 is p_1 = 1 – p_0. For a certain training example feature, the count for class 0 is x_0, and the count for class 1 is x_1.

    Under these assumptions, the log-odds is computed as:

    LogOdds = Log(x_0 + c * p_0) – Log (x_1 + c * p_1)

    Where:

    • c is the prior coefficient, which can be set by the user.

    • Log uses the natural base.

    In other words, for each class i:

    Log_odds[i] = Log( (count[i] + prior_coefficient * prior_frequency[i]) / (sum_of_counts - count[i]) + prior_coefficient * (1 - prior_frequency[i]))

    If the prior coefficient is positive, the log odds can be different from Log(count[i] / (sum_of_counts – count[i])).

  • Why are the log odds not computed for some items?

    By default, all items with a count less than 10 are collected in a single bucket called the "garbage bin". You can change this behavior value by using the Garbage bin threshold option in the Modify Count Table Parameters module.

List of Modules

 

The Learning with Counts category includes the following modules:

Module

Description

Build Counting Transform

Creates a count table and count-based features from a dataset, and saves it as a transformation

Export Count Table

Exports count table from a counting transform

This module supports backward compatibility with experiments that create count-based features using Build Count Table (deprecated) and Count Featurizer (deprecated).

Import Count Table

Imports an existing count table

This module supports backward compatibility with experiments that create count-based features using Build Count Table (deprecated) and Count Featurizer (deprecated). It supports conversion of count tables to count transformations.

Merge Count Transform

Merges two sets of count-based features

Modify Count Table Parameters

Modifies count-based features derived from an existing count table

Data Transformation / Learning with Counts的更多相关文章

  1. 【转】The most comprehensive Data Science learning plan for 2017

    I joined Analytics Vidhya as an intern last summer. I had no clue what was in store for me. I had be ...

  2. 《从0到1学习Flink》—— Flink Data transformation(转换)

    前言 在第一篇介绍 Flink 的文章 <<从0到1学习Flink>-- Apache Flink 介绍> 中就说过 Flink 程序的结构 Flink 应用程序结构就是如上图 ...

  3. Flink 从 0 到 1 学习 —— Flink Data transformation(转换)

    toc: true title: Flink 从 0 到 1 学习 -- Flink Data transformation(转换) date: 2018-11-04 tags: Flink 大数据 ...

  4. Flink Data transformation(转换)

    Flink Data transformation 算子学习 1.Source:数据源,Flink在流处理和批处理上的source大概有4类: 基于本地集合的source.基于文件的source.基于 ...

  5. Intermediate Python for Data Science learning 2 - Histograms

    Histograms from:https://campus.datacamp.com/courses/intermediate-python-for-data-science/matplotlib? ...

  6. Intermediate Python for Data Science learning 1 - Basic plots with matplotlib

    Basic plots with matplotlib from:https://campus.datacamp.com/courses/intermediate-python-for-data-sc ...

  7. Intro to Python for Data Science Learning 8 - NumPy: Basic Statistics

    NumPy: Basic Statistics from:https://campus.datacamp.com/courses/intro-to-python-for-data-science/ch ...

  8. Intro to Python for Data Science Learning 7 - 2D NumPy Arrays

    2D NumPy Arrays from:https://campus.datacamp.com/courses/intro-to-python-for-data-science/chapter-4- ...

  9. Intro to Python for Data Science Learning 5 - Packages

    Packages From:https://campus.datacamp.com/courses/intro-to-python-for-data-science/chapter-3-functio ...

随机推荐

  1. CSharp 相关知识点小结

    1.JS获取iframe下面的内容document.getElementById('IFRAME1').contentDocument; 2.dialog 弹出层,定位:postion:'bottom ...

  2. linux 维护常见场景小命令 (未完待续)

    1.安装KDE桌面 [root@rhel06 ~]# yum -y groupinstall "X Windows System" "KDE Desktop" ...

  3. UIWebView的使用

    iOS中UIWebView的使用详解 一.初始化与三种加载方式 UIWebView继承与UIView,因此,其初始化方法和一般的view一样,通过alloc和init进行初始化,其加载数据的方式有三种 ...

  4. Window 消息大全

    消息,就是指Windows发出的一个通知,告诉应用程序某个事情发生了.例如,单击鼠标.改变窗口尺寸.按下键盘上的一个键都会使Windows发送一个消息给应用程序. 消息本身是作为一个记录传递给应用程序 ...

  5. 【MySQL】漫谈MySQL中的事务及其实现

    最近一直在做订单类的项目,使用了事务.我们的数据库选用的是MySQL,存储引擎选用innoDB,innoDB对事务有着良好的支持.这篇文章我们一起来扒一扒事务相关的知识. 为什么要有事务? 事务广泛的 ...

  6. js 控制 css3高级运动 keyframes

    <!DOCTYPE html> <html lang="en"> <head> <meta charset="UTF-8&quo ...

  7. php类中常量的定义

    先看下面一段代码: class SVN { const DEFAULT_PATH = "/tmp"; const SVNLOOK_CMD = "/usr/bin/svnl ...

  8. 每天一个 Linux 命令(17):whereis 命令

    whereis命令只能用于程序名的搜索,而且只搜索二进制文件(参数-b).man说明文件(参数-m)和源代码文件(参数-s).如果省略参数,则返回所有信息. 和find相比,whereis查找的速度非 ...

  9. POSTMAN发起请求收到乱码 http 406错误

    web前段异常: The resource identified by this request is only capable of generating responses with charac ...

  10. Twitter-Snowflake,64位自增ID算法详解

    Twitter-Snowflake算法产生的背景相当简单,为了满足Twitter每秒上万条消息的请求,每条消息都必须分配一条唯一的id,这些id还需要一些大致的顺序(方便客户端排序),并且在分布式系统 ...