PP: Modeling extreme events in time series prediction

KDD: Knowledge Discovery and Data Mining (KDD)

Insititute: 复旦大学，中科大

Problem: time series prediction; modelling extreme events;

overlook the existence of extreme events, which result in weak performance when applying them to real time series.

为什么研究extreme events: Extreme events are rare and random, but do play a critical role in many real applications, such as the forecasting of financial crisis and natural disasters.

the weakness of deep learning methods roots in the conventional form of quadratic loss平方损失; --------> this paper use the extreme value theory极值理论 and develop a new form of loss for detecting the future occurrence of extreme events: extreme value loss.

普通预测： quadratic loss

极值预测：extreme value loss

Use memory network to memorize extreme events in historical records. EVL + memory network

Introduction:

time series prediction: classical research topic.

applications: climate prediction and stocks price monitoring;

Statistical methods: autoregressive moving average ARMA; nonlinear autoregressive exogenous NARX;

RNN (LSTM and GRU, gated recurrent unit); Compared with traditional methods, one of the major advantages of RNN structure is that it enables deep non-linear modeling of temporal patterns.

data imbalance and extreme events are harmful to deep learning models????; 值得验证

what are extreme events in time series: extremely small or large values of irregular and rare occurrences.

How to find extreme events? use certain thresholds to label extreme events

the randomness of extreme events have limited degrees of freedom (DOF)

end-to-end framewark.

underfitting and overfitting training problem;

Related work:

extreme events: 极大阈值 + 极小阈值

重尾分布

extreme value theory;

PROBLEMS CAUSED BY EXTREME EVENTS

conclusion: such a model would perform relatively poor if the true distribution of data in series is heavy-tailed.

underfit and overfit phenomenon

PREDICTING TIME-SERIES DATA WITH EXTREME EVENTS

Two factors: memorizing extreme events and modelling tail distribution;   memory network  to memorize the characteristic of extreme events; EVL

Memory network module:

1. Assumption: As pointed out by Ghil et al., extreme events in time-series data often show some form of temporal regularity [19]. 极值事件是有时间规律的，这是前提，如果没有这个前提，那么极值事件的研究是没有意义的。对于自然界的事物，如果没有规律性，那么无法进行建模。

2. windows sequence, wj -------- then use GRU to embed each window into feature space. wj as input,

Qi as the extreme events vector.  add attention mechanism as a part of the weight and update the output.

Extreme value loss:

Optimization: a direct thought is to combine the predicted outputs ot with the prediction of the occurrence of extreme events,

方差损失上增加了一个对于极值事件的惩罚项。

？？这只能算作loss function，怎么算作optimization呢？

Effectiveness of Time Series Prediction

针对两个真实数据库（climate and stock），一个伪造数据库上进行了实验，以rooted mean square error作为度量指标，在预测上约准确了0.01-0.08

但是看结果输出图，在极值上的预测结果确实好了。

Supplementary knowledge:

1. 张老师是战略能力很强，但是由于科研不在一线，导致战术可能会出现偏差。

2. 做交叉领域的文章时，i.可以做方法，ii.可以和领域结合，在领域里make sense, 有影响. 但如果四不像，两边都不会要。

3. 科研过程是一个严谨的流程体系，有一定的方法规律可循，不是瞎打一耙。

4. 其实真正重要的还是loss function怎么定，optimization 如何做，以及tailed distribution的一些现象。本质是数学问题，而非学习各种网络框架，最终还是要看deep learning 那本书和微积分。