SynSin: End-to-end View Synthesis from a Single Image

SynSin: End-to-end View Synthesis from a Single Image (CVPR 2020)

Authors: Olivia Wiles, Georgia Gkioxari, Richard Szeliski, Justin Johnson
Affiliations: University of Oxford, Facebook AI Research, Facebook, University of Michigan

• Website
• CVF Mirror Arxiv mirror
• Supp

Method

Figure 2 from SynSin Paper

1. First a depth map and a set of features are generated for each pixel using depth network \(d\) and feature network \(f\).
2. The depths are used to create a 3D point cloud of features \(P\).
3. Features are repositioned using the transformation matrix T.
4. Repositioned features are rendered using a neural point cloud renderer.
5. Rendered features are passed through a refinement network \(g\).