Z-score

import numpy as np

def outliers_z_score(ys):
threshold = 3 mean_y = np.mean(ys)
stdev_y = np.std(ys)
z_scores = [(y - mean_y) / stdev_y for y in ys]
return np.where(np.abs(z_scores) > threshold)

Modified Z-score

import numpy as np

def outliers_modified_z_score(ys):
threshold = 3.5 median_y = np.median(ys)
median_absolute_deviation_y = np.median([np.abs(y - median_y) for y in ys])
modified_z_scores = [0.6745 * (y - median_y) / median_absolute_deviation_y
for y in ys]
return np.where(np.abs(modified_z_scores) > threshold)

IQR(interquartile range)

import numpy as np

def outliers_iqr(ys):
quartile_1, quartile_3 = np.percentile(ys, [25, 75])
iqr = quartile_3 - quartile_1
lower_bound = quartile_1 - (iqr * 1.5)
upper_bound = quartile_3 + (iqr * 1.5)
return np.where((ys > upper_bound) | (ys < lower_bound))

Conclusion

It is important to reiterate that these methods should not be used mechanically.
They should be used to explore the data – they let you know which points might be worth a closer look.
What to do with this information depends heavily on the situation.
Sometimes it is appropriate to exclude outliers from a dataset to make a model trained on that dataset more predictive.
Sometimes, however,
the presence of outliers is a warning sign that the real-world process generating the data is more complicated than expected. As an astute commenter on CrossValidated put it:
“Sometimes outliers are bad data, and should be excluded, such as typos.
Sometimes they are Wayne Gretzky or Michael Jordan, and should be kept.” Domain knowledge and practical wisdom are the only ways to tell the difference.

  

摘自:http://colingorrie.github.io/outlier-detection.html

Three ways to detect outliers的更多相关文章

  1. MATLAB remove outliers.

    Answer by Richard Willey on 9 Jan 2012 Hi Michael MATLAB doesn't provide a specific function to remo ...

  2. 异常值处理outlier

    python信用评分卡(附代码,博主录制) https://study.163.com/course/introduction.htm?courseId=1005214003&utm_camp ...

  3. 深度学习图像配准 Image Registration: From SIFT to Deep Learning

    Image Registration is a fundamental step in Computer Vision. In this article, we present OpenCV feat ...

  4. (转)利用libcurl和国内著名的两个物联网云端通讯的例程, ubuntu和openwrt下调试成功(四)

    1. libcurl 的参考文档如下 CURLOPT_HEADERFUNCTION Pass a pointer to a function that matches the following pr ...

  5. 如何查找物理cpu,cpu核心和逻辑cpu的数量

    环境 Red Hat Enterprise Linux 4 Red Hat Enterprise Linux 5 Red Hat Enterprise Linux 6 Red Hat Enterpri ...

  6. Gartner 2018 年WAF魔力象限报告:云WAF持续增长,Bot管理与API安全拥有未来

    Gartner 2018 年WAF魔力象限报告:云WAF持续增长,Bot管理与API安全拥有未来 来源 https://www.freebuf.com/articles/paper/184903.ht ...

  7. A Gentle Guide to Machine Learning

    A Gentle Guide to Machine Learning Machine Learning is a subfield within Artificial Intelligence tha ...

  8. [linux]linuxI/O测试的方法之dd

    参考http://www.thomas-krenn.com/en/wiki/Linux_I/O_Performance_Tests_using_dd Measuring Write Performan ...

  9. WGCNA构建基因共表达网络详细教程

    这篇文章更多的是对于混乱的中文资源的梳理,并补充了一些没有提到的重要参数,希望大家不会踩坑. 1. 简介 1.1 背景 WGCNA(weighted gene co-expression networ ...

随机推荐

  1. java图片上传及图片回显1

    目的:选择图片,进行图片回显之后将图片保存到服务器上(PS:没有使用任何插件,样式很丑) 实现方式: js+servlet+jsp的方式来实现 事先准备: 文件上传处理在浏览器中是以流的形式提交到服务 ...

  2. Json多层对象访问

    背景说明 本文主要记录演示,利用Gson工具,对多层的 Json 数据进行转换读取的示例.原始 Json 字符串格式化效果如下: 示例代码 import java.util.Iterator; imp ...

  3. EXTJS的使用

    最近一段时间一直使用Extjs作为前端,通过HTTP与.net后端进行交互,今天总结一下EXTJS,方便以后复习. 1.概念: ExtJS可以用来开发RIA也即富客户端的AJAX应用,是一个用java ...

  4. Linux查询进程和结束进程

    1.  ps -ef |grep redis ps:将某个进程显示出来-A 显示所有程序. -e 此参数的效果和指定"A"参数相同.-f 显示UID,PPIP,C与STIME栏位. ...

  5. yafu安装使用方法以及mismatched parens解决方法

    yafu用于自动整数因式分解,在RSA中,当p.q的取值差异过大或过于相近的时候,使用yafu可以快速的把n值分解出p.q值,原理是使用Fermat方法与Pollard rho方法等. 再摘录官方的一 ...

  6. 开放数据接口 API 简介与使用场景、调用方法

    此文章对开放数据接口 API 进行了功能介绍.使用场景介绍以及调用方法的说明,供用户在使用数据接口时参考之用. 在给大家分享的一系列软件开发视频课程中,以及在我们的社区微信群聊天中,都积极地鼓励大家开 ...

  7. .Net Core HttpClient 忽略https证书提醒

    在测试中经常会遇到请求一些https的url,但又没有本地证书,这时候可以用下面的方法忽略警告 var httpclientHandler = new HttpClientHandler(); htt ...

  8. Ocr答题辅助神器 OcrAnswerer4.x,通过百度OCR识别手机文字,支持屏幕窗口截图和ADB安卓截图,支持四十个直播App,可保存题库

    http://www.cnblogs.com/Charltsing/p/OcrAnswerer.html 联系qq:564955427 最新版为v4.1版,开放一定概率的八窗口体验功能,请截图体验(多 ...

  9. 《Effective C++》模板与泛型编程:条款32-条款40

    条款41:了解隐式接口和编译期多态 class支持显示接口和运行期多态 class的显示接口由函数的名签式构成(函数名称.参数类型.返回类型) class的多态通过virtual函数发生在运行期 te ...

  10. 第七章· Redis Cluster 核心技术

    Redis Cluster 分布式集群 Redis Cluster 安装部署 Redis Cluster 集群管理操作(核心)