Problem: time series forecasting Challenge: forecasting for non-stationary signals and multiple future steps prediction ?? how to deal with non-stationary datasets?? Introduction one-step prediction problem VS multi-step prediction; multi-step foreca…
Training (deep) Neural Networks Part: 1 Nowadays training deep learning models have become extremely easy with high-quality libraries such as Torch and Theano. These libraries are really helpful for rapidly prototyping deep learning models even witho…
http://handong1587.github.io/deep_learning/2015/10/09/training-dnn.html  //转载于 Training Deep Neural Networks  Published: 09 Oct 2015  Category: deep_learning Tutorials Popular Training Approaches of DNNs — A Quick Overview https://medium.com/@asjad/p…
Problem: multi-horizon probabilistic forecasting tasks; Propose an end-to-end framework for multi-horizon time series forecasting, with temporal attention mechanisms to capture latent patterns. Introduction: forecasting ----- understanding demands. t…
w强化算法和数学,来迎接机器学习.神经网络. http://cs.stanford.edu/people/karpathy/convnetjs/ ConvNetJS is a Javascript library for training Deep Learning models (Neural Networks) entirely in your browser. Open a tab and you're training. No software requirements, no comp…
Problem: high-dimensional time series forecasting ?? what is "high-dimensional" time series forecasting? one dimension for each individual time-series. n个time series为n维. A need for exploiting global pattern and coupling them with local calibrati…
目录 概 主要内容 Glorot X, Bengio Y. Understanding the difficulty of training deep feedforward neural networks[C]. international conference on artificial intelligence and statistics, 2010: 249-256. @article{glorot2010understanding, title={Understanding the…
目录 摘要 1.引言 2.BinaryConnect 2.1 +1 or -1 2.2确定性与随机性二值化 2.3 Propagations vs updates 2.4 Clipping 2.5 A few more tricks 2.6 Test-Time Inference 3 Benchmark results 3.1 Permutation-invariant MNIST 3.2 CIFAR-10 3.3 SVHN 4 Related works 5. Conclusion and f…
目录 摘要 引言 1.BinaryNet 符号函数 梯度计算和累积 通过离散化传播梯度 一些有用的成分 算法1 使用BinaryNet训练DNN 算法2 批量标准化转换(Ioffe和Szegedy,2015),适用于小批量激活x. 算法3 ADAM学习规则(Kingma&Ba,2014). 2.基准测试结果 MLP on MNIST ConvNet on CIFAR-10 ConvNet on SVHN 3.在运行时更快 第一层 4.相关工作 结论 参考资料 论文地址:https://arxiv…
1. 摘要 本文尝试解释为什么在深度的神经网络中随机初始化会让梯度下降表现很差,并且在此基础上来帮助设计更好的算法. 作者发现 sigmoid 函数不适合深度网络,在这种情况下,随机初始化参数会让较深的隐藏层陷入到饱和区域. 作者提出了一个新的参数初始化方法,称之为 Xavier 初始化,来帮助深度网络更快地收敛. 2. 激活函数的作用以及训练过程中的饱和现象 2.1. 三种激活函数 \[Tanh(x)=\frac{1-e^{-x}}{1+e^{-x}}\] \[Sigmoid(x)=\frac…