MATLAB remove outliers.
Hi Michael
MATLAB doesn't provide a specific function to remove outliers. In general you have a couple different options to deal with outliers.
1. You can create an index that flags potential outliers and either delete them from your data set or substitute more plausible values
2. You can use robust techniques like robust regression which are less sensitive to the presence of outliers.
Your choice of strategies will depend a lot on your knowledge about the data set. For example, if you have a lot of data points that are coded with a value like -9999 these are probably error codes of some kind rather than actual numeric information.
I'm including some simple example code which shows a standard technique to detect outliers.
=====================
% Create a vector of X values
clear all
clc
hold off
X = 1:100;
X = X';
% Create a noise vector
noise = randn(100,1);
% Create a second noise value where sigma is much larger
noise2 = 10*randn(100,1);
% Substitute noise2 for noise1 at obs# (11, 31, 51, 71, 91)
% Many of these points will have an undue influence on the model
noise(11:20:91) = noise2(11:20:91);
% Specify Y = F(X)
Y = 3*X + 2 + noise;
% Cook's Distance for a given data point measures the extent to
% which a regression model would change if this data point
% were excluded from the regression. Cook's Distance is
% sometimes used to suggest whether a given data point might be an outlier.
% Use regstats to calculate Cook's Distance
stats = regstats(Y,X,'linear');
% if Cook's Distance > n/4 is a typical treshold that is used to suggest
% the presence of an outlier
potential_outlier = stats.cookd > 4/length(X);
% Display the index of potential outliers and graph the results
X(potential_outlier)
scatter(X,Y, 'b.')
hold on
scatter(X(potential_outlier),Y(potential_outlier), 'r.')
MATLAB remove outliers.的更多相关文章
- matlab中的containers.Map()
matlab中的containers.Map() 标签: matlabcontainers.Map容器map 2015-10-27 12:45 1517人阅读 评论(1) 收藏 举报 分类: Mat ...
- Taxi Trip Time Winners' Interview: 3rd place, BlueTaxi
Taxi Trip Time Winners' Interview: 3rd place, BlueTaxi This spring, Kaggle hosted two competitions w ...
- 异常值处理outlier
python信用评分卡(附代码,博主录制) https://study.163.com/course/introduction.htm?courseId=1005214003&utm_camp ...
- 壁虎书1 The Machine Learning Landscape
属性与特征: attribute: e.g., 'Mileage' feature: an attribute plus its value, e.g., 'Mileage = 15000' Note ...
- 第三课 创建函数 - 从EXCEL读取 - 导出到EXCEL - 异常值 - Lambda函数 - 切片和骰子数据
第 3 课 获取数据 - 我们的数据集将包含一个Excel文件,其中包含每天的客户数量.我们将学习如何对 excel 文件进行处理.准备数据 - 数据是有重复日期的不规则时间序列.我们将挑战数 ...
- Learning Spark中文版--第六章--Spark高级编程(2)
Working on a Per-Partition Basis(基于分区的操作) 以每个分区为基础处理数据使我们可以避免为每个数据项重做配置工作.如打开数据库连接或者创建随机数生成器这样的操作,我们 ...
- Matlab的标记分水岭分割算法
1 综述 Separating touching objects in an image is one of the more difficult image processing operation ...
- Matlab编程基础
平台:Win7 64 bit,Matlab R2014a(8.3) “Matlab”是“Matrix Laboratory” 的缩写,中文“矩阵实验室”,是强大的数学工具.本文侧重于Matlab的编程 ...
- Matlab 进阶学习记录
最近在看 Faster RCNN的Matlab code,发现很多matlab技巧,在此记录: 1. conf_proposal = proposal_config('image_means', ...
随机推荐
- BZOJ3856: Monster
题目:http://www.lydsy.com/JudgeOnline/problem.php?id=3856 题解:怎么乱搞一下都可以把 代码: #include<cstdio> #in ...
- 自己遇到的Android虚拟机出现的错误及解决方法【不断更新】
2012.11.9 第一个: [2012-11-09 13:15:14 - Tesa] Android Launch! [2012-11-09 13:15:14 - Tesa] The connect ...
- html5客户端跨域访问php服务端数据
客户端代码: var param = $.param( { feed:JSON.stringify({ content:'abcd' }) } ); $http({ url: 'http://61.1 ...
- this class is not key value coding-compliant for the key ##
setValue:forUndefinedKey:]: this class is not key value coding-compliant for the key ## 出现以上错误时很恶心,并 ...
- Make AngularJS $http service behave like jQuery.ajax()(转)
There is much confusion among newcomers to AngularJS as to why the $http service shorthand functions ...
- memcache的最佳实践方案。
基本问题 1.memcached的基本设置 1)启动Memcache的服务器端 # /usr/local/bin/memcached -d -m 10 -u root -l 192.168.0.200 ...
- poj 3694 Network
题意: 添加每条新连接后网络中桥的数目// 超时 先放着了 ,下次改//早上这代码超时了 下午改了,代码在下面#include <iostream> #include <algori ...
- 查看ORACLE执行计划的几种常用方法
SQL的执行计划实际代表了目标SQL在Oracle数据库内部的具体执行步骤,作为调优,只有知道了优化器选择的执行计划是否为当前情形下最优的执行计划,才能够知道下一步往什么方向. 执行计划的定义:执行目 ...
- SharePoint 2010 列表项事件接收器 ItemAdded 的使用方法
列表项事件处理器是继承于Microsoft.SharePoint.SPItemEventReceiver的类,Microsoft.SharePoint.SPItemEventReceiver类提供了许 ...
- 在英文 sql2005中 比较nvarchar 与 varchar的速度
declare @str1 varchar(max); declare @count int; ; print 'begin' begin set @str1 = @str1 + '*'; ; end ...