assuming that you're using xgboost to fit boosted trees for binary classification. The importance matrix is actually a data.table object with the first column listing the names of all the features actually used in the boosted trees.

The meaning of the importance data table is as follows:

  1. The Gain implies the relative contribution of the corresponding feature to the model calculated by taking each feature's contribution for each tree in the model. A higher value of this metric when compared to another feature implies it is more important for generating a prediction.
  2. The Cover metric means the relative number of observations related to this feature. For example, if you have 100 observations, 4 features and 3 trees, and suppose feature1 is used to decide the leaf node for 10, 5, and 2 observations in tree1, tree2 and tree3 respectively; then the metric will count cover for this feature as 10+5+2 = 17 observations. This will be calculated for all the 4 features and the cover will be 17 expressed as a percentage for all features' cover metrics.
  3. The Frequence (frequency) is the percentage representing the relative number of times a particular feature occurs in the trees of the model. In the above example, if feature1 occurred in 2 splits, 1 split and 3 splits in each of tree1, tree2 and tree3; then the weightage for feature1 will be 2+1+3 = 6. The frequency for feature1 is calculated as its percentage weight over weights of all features.

The Gain is the most relevant attribute to interpret the relative importance of each feature.

Gain is the improvement in accuracy brought by a feature to the branches it is on. The idea is that before adding a new split on a feature X to the branch there was some wrongly classified elements, after adding the split on this feature, there are two new branches, and each of these branch is more accurate (one branch saying if your observation is on this branch then it should be classified as 1, and the other branch saying the exact opposite).

Cover measures the relative quantity of observations concerned by a feature.

Frequency is a simpler way to measure the Gain. It just counts the number of times a feature is used in all generated trees. You should not use it (unless you know why you want to use it).

xgboost 里边的gain freq, cover的更多相关文章

  1. 【原创】xgboost 特征评分的计算原理

    xgboost是基于GBDT原理进行改进的算法,效率高,并且可以进行并行化运算: 而且可以在训练的过程中给出各个特征的评分,从而表明每个特征对模型训练的重要性, 调用的源码就不准备详述,本文主要侧重的 ...

  2. 小巧玲珑:机器学习届快刀XGBoost的介绍和使用

    欢迎大家前往腾讯云技术社区,获取更多腾讯海量技术实践干货哦~ 作者:张萌 序言 XGBoost效率很高,在Kaggle等诸多比赛中使用广泛,并且取得了不少好成绩.为了让公司的算法工程师,可以更加方便的 ...

  3. R语言︱XGBoost极端梯度上升以及forecastxgb(预测)+xgboost(回归)双案例解读

    XGBoost不仅仅可以用来做分类还可以做时间序列方面的预测,而且已经有人做的很好,可以见最后的案例. 应用一:XGBoost用来做预测 ------------------------------- ...

  4. XGBoost类库使用小结

    在XGBoost算法原理小结中,我们讨论了XGBoost的算法原理,这一片我们讨论如何使用XGBoost的Python类库,以及一些重要参数的意义和调参思路. 本文主要参考了XGBoost的Pytho ...

  5. 大白话5分钟带你走进人工智能-第32节集成学习之最通俗理解XGBoost原理和过程

    目录 1.回顾: 1.1 有监督学习中的相关概念 1.2  回归树概念 1.3 树的优点 2.怎么训练模型: 2.1 案例引入 2.2 XGBoost目标函数求解 3.XGBoost中正则项的显式表达 ...

  6. XGBboost 特征评分的计算原理

    xgboost是基于GBDT原理进行改进的算法,效率高,并且可以进行并行化运算,而且可以在训练的过程中给出各个特征的评分,从而表明每个特征对模型训练的重要性, 调用的源码就不准备详述,本文主要侧重的是 ...

  7. XGB算法梳理

    学习内容: 1.CART树 2.算法原理 3.损失函数 4.分裂结点算法 5.正则化 6.对缺失值处理 7.优缺点 8.应用场景 9.sklearn参数 1.CART树 CART算法是一种二分递归分割 ...

  8. XGBoost、LightGBM的详细对比介绍

    sklearn集成方法 集成方法的目的是结合一些基于某些算法训练得到的基学习器来改进其泛化能力和鲁棒性(相对单个的基学习器而言)主流的两种做法分别是: bagging 基本思想 独立的训练一些基学习器 ...

  9. xgboost的sklearn接口和原生接口参数详细说明及调参指点

    from xgboost import XGBClassifier XGBClassifier(max_depth=3,learning_rate=0.1,n_estimators=100,silen ...

随机推荐

  1. 【备忘:待完善】nsq集群初体验

    本机的一个节点及监控与管理后台 虚拟机中的一个节点 命令: [root@vm-vagrant nsq]# nsqd --lookupd-tcp-address=192.168.23.150:4160 ...

  2. redis底层数据结构--简单动态字符串 链表 字典 跳跃表 整数集合 压缩列表

    1.动态字符串 redis中使用c语言的字符床存储字面量,默认字符串存储采用自己构建的简单动态字符串SDS(symple dynamic string) redis包含字符串的键值对都是用SDS实现的 ...

  3. find: ‘/run/user/1000/gvfs’: Permission denied

    linux使用命令   find / -name ***  查找文件的时候会遇到以下报错 /gvfs’: Permission denied 其实这个目录是空的,查不查都没关系.所以,以下解决方式比较 ...

  4. codechef January Lunchtime 2017简要题解

    题目地址https://www.codechef.com/LTIME44 Nothing in Common 签到题,随便写个求暴力交集就行了 Sealing up 完全背包算出得到长度≥x的最小花费 ...

  5. Sentinel限流示例:编码和注解限流

    一.Sentinel 是什么? 随着微服务的流行,服务和服务之间的稳定性变得越来越重要.Sentinel 以流量为切入点,从流量控制.熔断降级.系统负载保护等多个维度保护服务的稳定性. Sentine ...

  6. PAT 甲级 1007 Maximum Subsequence Sum (25)(25 分)(0不是负数,水题)

    1007 Maximum Subsequence Sum (25)(25 分) Given a sequence of K integers { N~1~, N~2~, ..., N~K~ }. A ...

  7. [转]SVN 乱码问题

    以下来自:http://godchenmeng.iteye.com/blog/797727 最近研究SVN.发现在创建补丁包的时候出现这种情况. 在文件顶部不论是什么代码都会变成乱码.在文件中如果有注 ...

  8. free 命令结果完全剖析

    free 命令结果完全剖析 total 总物理内存大小. used 已分配的大小,注意,对操作系统来说任何被使用的内存都是used. free 未被分配的物理内存大小. shared 共享内存大小,主 ...

  9. 列表的使用2,深COPY和浅COPY,循环列表,步长切片

    name2=names.copy() 下面我看几种奇怪的现象: 首先把源列表copy以后,把源列表第2个数值修改.那么没毛病. 如果源列表里,还包含了一个子列表:这也没毛病 如果我们这个时候修改子列表 ...

  10. 6 istio 配置 grafana

    1 验证prometheus  service 已经运行: $ kubectl -n istio-system get svc prometheus NAME CLUSTER-IP EXTERNAL- ...