5,321
edits
Generative adversarial network: Difference between revisions
Generative adversarial network (view source)
Revision as of 15:28, 17 January 2020
, 17 January 2020→Important Papers
| Line 56: | Line 56: | ||
==Important Papers== | ==Important Papers== | ||
===Latent Space Exploration=== | |||
* [https://arxiv.org/abs/1907.07171 On the "steerability" of generative adversarial networks] | |||
** Exploring which directions in the latent space control high-level features such as camera position, object rotation, object hue,... | |||
====Inversion==== | |||
How to go from an image to a latent space vector | |||
* [https://arxiv.org/abs/1904.03189 Image2StyleGAN] | |||
** Mostly showing off applications using StyleGAN: morphing, style transfer, expression transfer | |||
** Invert StyleGAN to get style vectors <math>w</math> but with a different style vector per layer. | |||
** Able to get StyleGAN trained on faces to output cats, dogs, cars, ... | |||
** Followup Papers: [https://arxiv.org/abs/1911.11544 Image2StyleGAN++] adds Activation Tensor Manipulation | |||
===Activation Tensor Manipulation=== | |||
* [https://arxiv.org/abs/1811.10597 GAN Dissection: Visualizing and Understanding Generative Adversarial Networks] | * [https://arxiv.org/abs/1811.10597 GAN Dissection: Visualizing and Understanding Generative Adversarial Networks] | ||
** Basically, | ** Authors: David Bau | ||
** Basically, individual "units" or channels of the intermediate representations correspond to some features like windows or trees in the output | |||
** Dissection: Identifying which units correspond to features can be done by visualizing each channel as a heatmap. Then threshold the heatmap so each value is binary 0/1. Calculate the IOU between the heatmap and the segmented feature in the generated picture. | ** Dissection: Identifying which units correspond to features can be done by visualizing each channel as a heatmap. Then threshold the heatmap so each value is binary 0/1. Calculate the IOU between the heatmap and the segmented feature in the generated picture. | ||
** Intervention: By zeroing out channels, you can remove windows or trees from the generated image. Alternatively you can add windows or trees at specific locations by activating the neurons at that location of the corresponding window/tree channel. | ** Intervention: By zeroing out channels, you can remove windows or trees from the generated image. Alternatively you can add windows or trees at specific locations by activating the neurons at that location of the corresponding window/tree channel. | ||
** This is fairly specific to CNN architectures where there is a locality correspondence between the intermediate representations and the output image. | ** This is fairly specific to CNN architectures where there is a locality correspondence between the intermediate representations and the output image. | ||
** Followup Papers: [https://dl.acm.org/doi/abs/10.1145/3306346.3323023 Semantic photo manipulation] | |||
==Resources== | ==Resources== | ||
* [https://github.com/soumith/ganhacks Tricks for Training GANs] | * [https://github.com/soumith/ganhacks Tricks for Training GANs] | ||