Generative adversarial network: Difference between revisions
| Line 48: | Line 48: | ||
** Trained using a frame discriminator and a motion discriminator. (Each generator are trained with both discriminators) | ** Trained using a frame discriminator and a motion discriminator. (Each generator are trained with both discriminators) | ||
===Video Compression=== | ===Image and Video Compression=== | ||
* [https://arxiv.org/pdf/1912.10653.pdf | * [http://openaccess.thecvf.com/content_ICCV_2019/html/Agustsson_Generative_Adversarial_Networks_for_Extreme_Learned_Image_Compression_ICCV_2019_paper.html Image Compression] | ||
* [https://arxiv.org/pdf/1912.10653.pdf Video compression via colorization] | |||
** Colorize with GAN. Only transmit luminance (Y of YUV) | ** Colorize with GAN. Only transmit luminance (Y of YUV) | ||
** The paper claims 72% BDBR reduction compared to HM 16.0. | ** The paper claims 72% BDBR reduction compared to HM 16.0. | ||
Revision as of 16:49, 7 January 2020
GANs are generative adversarial networks. They were developed by Ian Goodfellow.
Goal: Learn to generate examples from the same distribution as your training set.
Basis Structure
GANs consist of a generator and a discriminator.
For iteration i
For iteration j
Update Discriminator
Update Generator
Variations
Conditional GAN
Paper
Feed data y to both generator and discriminator
Wasserstein GAN
Paper
Medium post
This new WGAN-GP loss function improves the stability of training.
Normally, the discriminator is trained with a cross-entropy with sigmoid loss function.
The WGAN proposes using Wasserstein distance which is implemented by removing the cross-entropy+sigmoid
and clipping (clamp) the weights on the discriminator to a range \(\displaystyle [-c, c]\).
However, weight clipping leads to other issues which limit the critic.
Instead of clipping, WGAN-GP proposes gradient penalty to enforce 1-Lipschitz .
Applications
CycleGan
InfoGAN
SinGAN
Paper
Website
Github Official PyTorch Implementation
SinGAN: Learning a Generative Model from a Single Natural Image
MoCoGAN
Paper
MoCoGAN: Decomposing Motion and Content for Video Generation
Video Prediction
- Dual Motion GAN (Liang et al. 2017)
- Have a frame generator and a motion generator
- Combine the outputs of both generators using a fusing layer
- Trained using a frame discriminator and a motion discriminator. (Each generator are trained with both discriminators)
Image and Video Compression
- Video compression via colorization
- Colorize with GAN. Only transmit luminance (Y of YUV)
- The paper claims 72% BDBR reduction compared to HM 16.0.