StyleGAN
StyleGAN CVPR 2019
2018 Paper (arxiv)
CVPR 2019 Open Access
StyleGAN Github
StyleGAN2 Paper
StyleGAN2 Github
An architecture by Nvidia which allows controlling the "style" of the GAN output by applying adaptive instance normalization at different layers of the network.
StyleGAN2 improves upon this by...
Architecture
StyleGAN consists of a mapping network \(\displaystyle f\) and a synthesis network \(\displaystyle g\).
Mapping Network
The goal of the mapping network is to generate a latent vector \(\displaystyle w\).
This latent \(\displaystyle w\) is used by the synthesis network as input the each AdaIN block.
Before each AdaIN block, a learned affine transformation converts \(\displaystyle w\) into a "style" in the form of mean and standard deviation.
The mapping network \(\displaystyle f\) consists of 8 fully connected layers with leaky relu activations at each layer.
The input and output of this vector is an array of size 512.
Synthesis Network
The synthesis network is based on progressive growing (ProGAN).
It consists of 9 convolution blocks, one for each resolution from \(\displaystyle 4^2\) to \(\displaystyle 1024^2\).
Each block consists of upsample, 3x3 convolution, AdaIN, 3x3 convolution, AdaIN.
After each convolution layer, a gaussian noise with learned variance (block B in the figure) is added to the feature maps.
Adaptive Instance Normalization
Each AdaIN block takes as input the latent style \(\displaystyle w\) and the feature map \(\displaystyle x\).
An affine layer (fully connected with no activation, block A in the figure) converts the style to a mean \(\displaystyle y_{b,i}\) and standard deviation \(\displaystyle y_{s,i}\).
Then the feature map is shifted and scaled to have this mean and standard deviation.
- \(\displaystyle \operatorname{AdaIN}(\mathbf{x}_i, \mathbf{y}) = \mathbf{y}_{s,i}\frac{\mathbf{x}_i - \mu(\mathbf{x}_i)}{\sigma(\mathbf{x}_i)} + \mathbf{y}_{b,i}\)