深度学习--GAN学习笔记

生成模型

什么是生成模型？

GMM: 用来做聚类，（非监督学习）
NB（朴素贝叶斯）：（监督学习，可以用来做垃圾邮件分类）
Logistics 回归是生成模型吗？No！

生成模型与解决的任务之间没有必然的联系，关注的是样本本身。对于监督学习$p(x， y)$ , 非监督学习 $p(x,z)$ ，有些模型甚至仅用 $X$ , 成为 Autoregression model 。

GAN(生成式对抗网络)

工艺大师的目的：成为高水平，可以以假乱真的大师。($P_g \rightarrow P_d $)

如果用统计模型去刻画这个模型，如何去表示？模型中的生成器和判别器可以通过一些模型去表示。

GAN的流程：

Initialize the generator and dicriminator
In each traning iteration
- Step 1 : Fixed generator G , and update discriminator D.
- Step 2 : Fixed Discriminator D , and update generator G ;

Auto - Encoder

graph LR
A[Random Generator] -->C(N.N)
F(A vector as code) --> C(N.N Decoder)
C --> D[N.N Decoder] -->G[Image]

VAE

这里会对 $\sigma$ 进行限制：

\[\min ~ \sum_{i} ^{3} \left[ exp(\sigma_{i}) - (1+\sigma_{i}) + m_{i}^{2} \right]
\]

VAE存在着一些问题：它不能真的尝试去模拟真的图像

GAN——数学表示

数学符号：

$\{x_{i}\}_{i}^{N}$ 样本数据
$P_{data}：$ Population Density
$P_g(x;\theta_{g}):$ 生成模型的density；可以用 NN 进行逼近
$z \sim P_{Z}(Z)$ , Proposed Density，一般用来生成 $\tilde{x}_{i}$
$D(Z) = D(\tilde{x},\theta_{d}):$ 表示 Discrimination 识别是否是“国宝”的概率，越接近于1，越可能是国宝
$\tilde{x} = G(z;\theta_g)$

graph LR
A[Z] -->B(N.N)
B --> C{N.N}
F(P_data) --> C{N.N}
C -->|YES| D[国宝]
C -->|NO| E[赝品]

G[Z]-->H(N.N)-->I(tilde_x)

对于高专家来说：

如果 $x$ 是来自于 $P_{data}$ , 我们可以发现 $D(x;\theta_{d})$是较大的 $\Longrightarrow$ $\log(D(x;\theta_{d}))$ 是较大点的；
如果 $x$ 是来自于 $P_{generator}$ , 我们可以发现 $D(x;\theta_{d})$是较小的 $\Longrightarrow$ $1 - D(x;\theta_{d})$是较大点的$\Longrightarrow$ $\log(1 -D(G(Z))$是较大的；

所以对于高专家的目标函数为：

\[\max_{D} \rm E_{x \sim P_{data}} [~\log P(x)~] + E_{z \sim P_{z}} [~ \log (1 - D(Z))~]
\]

对于高级技师来说，想以假乱真：

如果 $x$ 是来自于 $P_{generative}$ , 我们可以高级技师希望 $D(x;\theta_{d})$是较大的 $\Longrightarrow$ $\log(1 -D(G(Z))$是较小的；

\[\min_{G}~ \rm E_{Z \sim P_{Z}} [~ \log( 1- D[G(Z)]) ]
\]

所以在这个体系中，总的目标为：

\[\min_{G} \max_{D} ~ \rm E_{x \sim P_{data}} [~\log P(x)~] + E_{z \sim P_{z}} [~ \log (1 - D(Z))~]
\]

小结：GAN的复杂度在于模型的学习，我们并没有直接面对$P_g (x , \theta_{g})$ ，而是用神经网络去逼近$P_g (x , \theta_{g})$ ,所以GAN是一种 Implict Density Model 。

GAN存在最优解吗？

一般来说，我们会从MLE (Maximum Likelihood Estimation) 角度去解目标函数，但在GAN模型中，我们可以不通过MLE，而通过神经网站逼近Prob Density。

f-divergence

Definition: P and Q are two distribution . $p(x)$ and $q(x)$ are the probability of sampling of sampling x.

\[D_{f}(P||Q) = \int_{x} q(x)f\left( \frac{p(x)}{q(x)}\right) dx
\]

Remark : $f$ function is convex , $f(1) = 0$ , $D_{f}({P||Q})$ evaluates the differences of P and Q.

if $p(x)$ = $q(x)$ for all $x$, $D_{f}({P||Q}) = 0$ ; if $p(x) = q(x)$ for some $x$ ,
\[D_{f}({P||Q})= \int_{x} q(x) f \left( \frac{p(x)}{q(x)}\right) dx \geq f \left[ \int_{x} q(x) \cdot \frac{p(x)}{q(x)} dx \right] = 0
\]

Fenchel Congjugate

every conex function $f$ have a Congjugate function $f^{*}$；$f^{*}$ also is a convex function . We can find that $(f^{*})^{*} = f$

\[f^{*}(t) = \max_{x \in dom (f)} {xt - f(x)} \Longleftrightarrow
f (x) = \max_{t \in dom (f^{*})} {xt - f^{*}(t)}
\]

if $f(x) = x \log x$ , $f^{*}(t) = e^{t-1}$

\[\begin{aligned}
D_{f}(P||Q) & = \int_{x} q(x)f\left( \frac{p(x)}{q(x)}\right) dx \\
& = \int_{x} q(x) \left[ \max_{t \in dom(f)}
\left( \frac{p(x)}{q(x)} t - f^{*} (t) \right) \right]dx \\
& \approx \max_{D} \int_{x} p(x)D(x)dx - q(x)f^{*}\left(D(x) \right) dx
\end{aligned}
\]

Where $t = D(x)$ ; D is a function , whose input is $x$ , Output is $t$

\[\begin{aligned}
D_{f}(P||Q) & \ge \int_{x} q(x) \left[
\left( \frac{p(x)}{q(x)} D(x) - f^{*} \left(D(x) \right) \right) \right]dx \\
& = \int_{x} p(x)D(x)dx - q(x)f^{*}\left(D(x) \right) dx
\end{aligned}
\]

GAN角度

这里出现了一个问题，我们并不知道 $P_{g}$ 和 $P_d$ 这两个分布，怎么去求解？在GAN角度中,我们可以通过从 $p_{g}$ 和 $p_d$ 中抽样的方法去求解生成器。

站在GAN的角度如何测量KL Div

GAN是从对抗学习的角度，因此我们要思考最优解存在吗？如果存在，$P_{g} \stackrel{？}{=} P_{d a t a}$

我们引用论文中的符号：

\[V(G,D) \equiv \rm E_{x \sim p(x)} \left[ \log (x)\right]
+ E_{x \sim p_{g}} \left[ \log \left( 1 - D\left(X \right )\right) \right]
\]

因此我们的目标函数可以写为：

\[\min_{G} \max_{D}~ V(G,D)
\]

Asumption:

$G(x)$ can be any fuction

求解步骤：

Step 1 : Given G , what is optimal $D^{*}$ maximizing

\[\begin{aligned}
\max_{D} ~ V(D,G) & = \int p_{data} \log \left(D\left(x \right)\right) dx +
\int p_{g} \log \left[ \left( 1- D(x) \right) \right] dx \\
& = \int \left[ p_{data} \log D(x) + p_{g} \log(1-D(x)) \right] dx
\end{aligned}
\]

FOC Condition：

\[\begin{aligned}
\frac{\partial} {\partial D} V(D,G) & = \frac{\partial }{\partial D}
\left [ \int \left[ p_{data} \log D + p_{g} \log(1-D(x)) \right] dx \right] \\
& = \int \frac{\partial }{\partial D}\left[ p_{data} \log D + p_{g} \log(1-D(x)) \right] dx \\
& = \int \left[ p_{data} \frac{1}{D(x)} + p_{g}
\frac{-1}{1-D(x)}\right] D^{\prime}(x)~ dx \stackrel{\Delta}{=}
0
\end{aligned}
\]

Remark: 这里定积分和求导可以互换顺序，$D(x) \in [0,1]$, 从FOC条件可以得到：

\[p_{d} \frac{1}{D(x)} = p_{g} \frac{1}{1-D(x)} ~ \Longrightarrow ~ p_d [1-D(x)] = p_{g}D(x)
~ \Longrightarrow ~ D^{*} (x) = \frac{p_{d}}{p_{d} + p_{g}}
\]

Step 2: 把 $D^{*}$ 带入目标函数，则有：

\[\begin{aligned}
\min_{G} \max_{D} V (G,D) & = \min_{G} V (G, D^{*}) \\
& = \min_{G} \rm E_{x \sim p_{d}} \log \left[
\frac{p_{d}}{p_{d} + p _{g}}\right]
+ E_{x \sim p_{g}} \log \left[ \frac{p_{g}}{p_{d}+p_{g}}\right]
\end{aligned}
\]

这个表达式与 KL Divergence 的形式非常像，可以用 KL来表示吗？p 或 q 为一个Prob Density 。

\[KL(p || q) = \rm E_{p} \left[ \log \frac{p}{q}\right] \ge 0
\]

答案是可以的，需要一些小技巧， $ 0 \le JSD < \le 2$ .

\[\begin{aligned}
\min_{G} \max_{D} V(G, D) & = \min_{G} \rm E_{x \sim p_{d}} \log \left[
\frac{p_{d}}{p_{d} + p _{G}}\right]
+ E_{x \sim p_{G}} \log \left[ \frac{p_{g}}{p_{d}+p_{G}}\right] \\
& = \min_{G} \rm E_{x \sim p_{d}} \log \left[
\frac{p_{d}}{\left( p_{d} + p _{G} \right) \2} \frac{1}{2} \right]
+ E_{x \sim p_{G}} \log \left[ \frac{p_{G}}{ \left (p_{d}+p_{G}\right) \2}
\frac{1}{2} \right] \\
& = \min_{G} KL[~ p_d || \frac{p_{d} + p_{G}}{2}~] +
KL[~p_g || \frac{p_{d} + p_{G}}{2}~] - \log(4) \\
& = 2JSD(P_{data} ||p_{G} ) - \log4
\end{aligned}
\]

什么时候等式成立？当且仅当 $p_{d} = p_{G} = \frac{p_d + p_{G} }{2}$ , 可以解出 $p_{d}^{*} = p_{G }^{*}$ = 0.5 , $G^{*} = 0.5$，这意味着什么？在最有的情况下，复制出来的工艺品，送给 Discriminator 已经无法识别真假。

Algorithm:

Given $G_{0}$
Find $D_{0}^{*}$ maxmizing $V(G_{0}, D)$
$\theta_{G} \leftarrow \eta ~ \partial V(D,G_{0}^{*}) / \partial \theta_{G} $ $\Longrightarrow$ obtain $G_{1}$

$V(G_{0}, D_{0}^{*})$ is the JS Divergence between $p_{data}(x)$ and $p_{G_{0}}$

Find $D_{1}^{*}$ maxmizing $V(G_{1}, D)$
\[\theta_{G} \leftarrow \eta ~ \partial V(D,G_{1}^{*}) / \partial \theta_{G} $$ obtain $G_{2}$

\]
......

In Practice

Given G , How to compute $\max V(G, D)$
- sample {$x^{1}, x^{2}, \ldots ,x^{m}$} from $p_{data}(x)$ , sample {$\tilde{x}^{1}, \tilde{x}^{2}, \ldots ,\tilde{x}^{m}$} from $p_{G}(x)$
  \[\max V = \frac{1}{m} \sum_{i}^{m} \log D(x^{i}) +
  \frac{1}{m} \sum_{i}^{m} \left[\log 1 - D( \tilde{x}^{i}) \right]
  \]
Remark： In binary classifier , D is the binary classifier with the sigmoid output . {$x^{1}, x^{2}, \ldots ,x^{m}$} from $p_{data}(x)$ can be viewed as positive examples , {$\tilde{x}^{1}, \tilde{x}^{2}, \ldots ,\tilde{x}^{m}$} from $p_{G}(x)$ can be viewed as negative examples. Our goal is to minimize the cross-entropy. this is equal to the $\max V$

Summary

Algorithm:

initialize $\theta_{d}$ for $D$ and $\theta_{g}$ for $G$
In each training iterations (k times) :
- sample m {$x^{1}, x^{2}, \ldots ,x^{m}$} from data distribution $p_{data}(x)$
- sample m noize {$z^{1}, z^{2}, \ldots ,z^{m}$} from prior distribution $p_{prior}(z)$
- obaining the generated data {$\tilde{x}^{1}, \tilde{x}^{2}, \ldots ,\tilde{x}^{m}$} , $\tilde{x}^{i} = G(z^{i})$
- update discriminator parameters $\theta_{d}$ to maxmize
  - $\max \tilde{V} = \frac{1}{m} \sum_{i}^{m} \log D(x^{i}) +
    \frac{1}{m} \sum_{i}^{m} \left[\log 1 - D( \tilde{x}^{i}) \right]$
  - $\theta_{d} \leftarrow \eta ~ \Delta \tilde{V}(\theta_{d})$
以上是测量 JSD，下面是最小化 JSD , 即 Learning Genarator , 但是不能 train 太多次。
- sample another m noise samples {$z^{1}, z^{2}, \ldots ,z^{m}$} from prior distribution $p_{prior}(z)$
- update discriminator parameters $\theta_{g}$ to minmize
  - $\min \tilde{V} = \frac{1}{m} \sum_{i}^{m} \log D(x^{i}) +
    \frac{1}{m} \sum_{i}^{m} \left[\log 1 - D \left(G \left( z^{i}\right) \right) \right]$
  - $\theta_{g} \leftarrow \eta ~ \Delta \tilde{V}(\theta_{g})$

Intution

W - GAN

WGAN

Considering one distribution P as a pile of earth , and another distribution Q as a target . The average distance the earther mover has to move the earth .

What's the moving plan ?

A "Moving plan " is matrix, the values of the element is the amount of earth from one position to another.

Average distance of a plan $\gamma$ :

\[B(\gamma) = \sum_{x_{p}, x_{q}} \gamma(x_{p}, x_{q})||x_{p} - x_{q} ||
\]

Earth Distance:

\[W(P, Q) = \min_{\gamma \in \prod } B(\gamma)
\]

Evaluate the wasserstein distance between $p_{data}$ and $p_{g}$; D has to be smooth enough .

\[ V(G,D) = \max_{D \in 1-Lipschitz } \{ E_{x \sim p_{data}}[D(x)] + E_{x \sim p_{G}} [D(x)]\}
\]

Without the constraint , the trainning of D will not converge.
Keep the D smmoth forces $D(x)$ become $\infin$ to $-\infin$
What's the Lipschitz ?

\[k - Lipschitz ~~ function: ~~||f(x_{1}) - f(x_{2}) \le k ||x_{1}-x_{2}||
\]

How to Solve ?

Weight Clipping : Force the parameters w between c and -c , After parameter update , if $w > c$ ，$w = c$; if $w < - c$ ,$w = -c$