caffe+GAN︱PPGN生成模型5则官方案例（caffe版）

一、效果与架构

PPGN 整合了对抗训练、cnn特征匹配、降噪自编码、Langevin采样；在NIPS2016得到了Ian Goodfellow的介绍.

PPGN生成的图像同类差异化大，可根据指定生成不同类别的图像、多类化，生成的图像清楚分辨率高。

PPGN可使用imagenet1000类分类网络生成特定类别的图像。

效果：

架构图：

详细解读：zdx3578（微信公众号）

二、案例

所需环境：caffe/python/GPU可以加速

确保你是在./caffe/python下运行settings.py

Install ImageMagick command-line interface on your system.

1、案例一：1_class_conditional_sampling.sh

1_class_conditional_sampling.sh: Sampling conditioning on the class “junco” (output unit #13 of the CaffeNet DNNtrained on ImageNet dataset). This script produces a sampling chain for a single given class.

Running ./1_class_conditional_sampling.sh 13 produces this result:

2、2_class_conditional_sampling_many.sh

Running ./2_class_conditional_sampling_many.sh <epsilon1> with different epsilon1 (multiplier for the image prior component) produces a chain with different styles of samples:

3、3_hidden_conditional_sampling.sh

Instead of conditioning on a class, it is possible to condition on a hidden neuron i.e. performing Multifaceted Feature Visualization or synthesizing a set of inputs that highly activate a given neuron to understand what features it has learned to detect.

Running ./3_hidden_conditional_sampling.sh 196 produces a set of images for a conv5 neuron #196 previously identified as a "face detector" in DeepVis toolbox:

4、4_hidden_conditional_sampling_placesCNN.sh

One can repeat the example above but with an arbitrary neuron in a different condition network. Here, we visualize the conv5 neuron #182 in the AlexNet DNN trained on MIT Places205 dataset. This neuron has been previously identified as a “food detector” in Zhou et al [2].

Running ./4_hidden_conditional_sampling_placesCNN.sh 182 produces this result:

5、5_caption_conditional_sampling.sh

We can also replace the image classifier network in previous examples with a pre-trained image captioning network to form a text-to-image model without even re-training anything. The image captioning model in this example is the LRCN model in Donahue et al (2015) [1].

You would need to use the Caffe provided here and update the path to Caffe accordingly in settings.py

The list of words supported are here

Running ./5_caption_conditional_sampling.sh a_church_steeple_that_has_a_clock_on_it produces this result: