PaperReading20200219

CanChen ggchen@mail.ustc.edu.cn

---

This is my first day sharing my reading paper and I will try to paraphrase core ideas in these papers with very simple words. Every paper consists three parts, namely, motivation, method, and contribution. In each part, no more than 3 sentences will be used.

 

NAS-Bench-201

• Motivation:Network search algorithms are often quite expensive and different search spaces also make it difficult for us to compare these algorithms. In fact, we can treat network architectures as X and their corresponding accuracies as Y, and construct a standard dataset to solve this problem.
• Method: Using cell-based strategy, we only need to find a cell and insert it into the macro structure. In the paper, 4 nodes and 5 operations are used, which means we need to train 15625 cells. As last，the author just trained 15625 models on cifar10，cifar100 and sampled-ImageNet，and provided us with the corresponding training logs.
• Contribution: The paper is a ICLR paper and is not very novel（at least I think）. It shows us again: computing resources is very important. At least, it gives us a benchmark for NAS research and now we can use CPU to do NAS.

 

Peephole

• Motivation: Can we get the network's performance without training?
• Method: The author only considers sequential network architectures since we can treat the sequential network architectures as "a language". Then the author uses LSTM to deal with this problem like language modeling.
• Contribution: The work is kind of limited since it only deals with sequential network architecures while other structures such shortcut paths are in fact dominating this field.

 

Latency-aware

• Motivation: Current Darts algorithms do not take latency into consideration.
• Method: First, the author train a regression model that can predict a network's latency based on the network's structure. Then the author inserts this model into bi-level optimization equation as part of the loss function.
• Contribution: This work is an extension of Darts and can be very useful since latency is important in real scenarios.