002-ImageNetClassificationDeep2017

ImageNet classification with deep convolutional neural networks #paper

---

1. paper-info

1.1 Metadata

• Author:: * Authors: [[Alex Krizhevsky]], [[Ilya Sutskever]], [[Geoffrey E. Hinton]]
• 作者机构::
• Keywords::
• Journal:: [[Communications of the ACM]]
• Date:: [[2017-05-24]]
• 状态:: #待读

1.2 Abstract

We trained a large, deep convolutional neural network to classify the 1.2 million high-resolution images in the ImageNet LSVRC-2010 contest into the 1000 different classes. On the test data, we achieved top-1 and top-5 error rates of 37.5% and 17.0% which is considerably better than the previous state-of-the-art. The neural network, which has 60 million parameters and 650,000 neurons, consists of five convolutional layers, some of which are followed by max-pooling layers, and three fully-connected layers with a final 1000-way softmax. To make training faster, we used non-saturating neurons and a very efficient GPU implementation of the convolution operation. To reduce overfitting in the fully-connected layers we employed a recently-developed regularization method called “dropout” that proved to be very effective. We also entered a variant of this model in the ILSVRC-2012 competition and achieved a winning top-5 test error rate of 15.3%, compared to 26.2% achieved by the second-best entry.

---

2. The Architecture

图 2.1 原始结构表示

源paper网络结构如图

图 2.2 AlexNet 结构图

5个卷积层加上3个全连接层，激活函数采用RuLU，输入为227x227x3的图片，输出为1000（1000类的图片标签），在第一个全连接层和第二个全连接层之间使用了dropout。

---

3. Analysis of some innovative points

• [[ReLU]]
  
  防止梯度衰减过快。

• [[dropout]]

• [[Data expension]]

  • Translational transformation (crop): 移动裁剪，由（256x256）-> （224x224）4个角和中间各一张，共5张。
  • Reflective transformation (flip):
  • Illumination and color shift:

• [[Overlap pooling]]
  
  汇聚层窗口大小s, 步幅z， 如果s=z就是传统的汇聚层，如果s>z，就得到了重叠汇聚层。在该算法中，使用overlap pooling可以减少过拟合。

• LRN local response normalization

---

4. Zotero links

• DOI: 10.1145/3065386

5. 参考文献

1. 网络结构

2. https://www.intefrankly.com/articles/Deep-Learning-Paper-Notes-I--The-AlexNet-Model-Explained/eda8465893b7

3. AlexNet 中的 LRN（Local Response Normalization） 是什么

4. 深度学习中Dropout原理解析