SinGAN: Learning a Generative Model from a Single Natural Image
SinGAN
Paper
Website
Github Official PyTorch Implementation
SinGAN: Learning a Generative Model from a Single Natural Image
Authors: Tamar Rott Shaham (Technion), Tali Dekel (Google Research), Tomer Michaeli (Technion)
Basic Idea
Bootstrap patches of the original image and build GANs which can add fine details to blurry patches at different path sizes.
- Start by building a GAN to generate low-resolution versions of the original image
- Then upscale the image and build a GAN to add details to patches of your upscaled image
- Fix the parameters of the previous GAN. Upscale the outputs and repeat.
Architecture
They build \(\displaystyle N\) GANs.
Each GAN \(\displaystyle G_n\) adds details to patches of the image produced by GAN \(\displaystyle G_{n+1}\) below it.
The final GAN \(\displaystyle G_0\) adds only fine details.
Generator
Discriminator
Training and Loss Function
\(\displaystyle \min_{G_n} \max_{D_n} \mathcal{L}_{adv}(G_n, D_n) + \alpha \mathcal{L}_{rec}(G_n)\)
They use a combination of the standard GAN adversarial loss and a reconstruction loss.
Reconstruction Loss
\(\displaystyle \mathcal{L}_{rec} = \Vert G_n(0,(\bar{x}^{rec}_{n+1}\uparrow^r) - x_n \Vert ^2\)
The reconstruction loss ensures that the original image can be built by the GAN.
Rather than inputting noise to the generators, they input
\(\displaystyle \{z_N^{rec}, z_{N-1}^{rec}, ..., z_0^{rec}\} = \{z^*, 0, ..., 0\}\)
where the initial noise \(\displaystyle z^*\) is drawn once and then fixed during the rest of the training.