Neural scene representation and rendering (Science 2018)

Authors: S. M. Ali Eslami*,†, View ORCID ProfileDanilo Jimenez Rezende†, View ORCID ProfileFrederic Besse, View ORCID ProfileFabio Viola, View ORCID ProfileAri S. Morcos, Marta Garnelo, View ORCID ProfileAvraham Ruderman, View ORCID ProfileAndrei A. Rusu, Ivo Danihelka, Karol Gregor, David P. Reichert, Lars Buesing, View ORCID ProfileTheophane Weber, Oriol Vinyals, Dan Rosenbaum, Neil Rabinowitz, Helen King, Chloe Hillier, View ORCID ProfileMatt Botvinick, Daan Wierstra, Koray Kavukcuoglu, Demis Hassabis
Affiliations: Deepmind

Science URL

Generative Query Network

We have K images for scene i: \(\mathbf{x}^k_i\) taken from 2D viewpoints \(\mathbf{v}^k_i\)
A representation network creates a neural representation \(\mathbf{r}\).
The generation network predicts the scene from viewpoints \(\mathbf{v}^q\) and random noise \(\mathbf{z}\).