Image-based rendering

Image-based rendering focuses on rendering scenes from existing captured or rasterized images, typically from a new viewpoint.
Recent research allows adding new objects, performing relighting, and other AR effects.

Implicit Representations

Light Fields

Main article: Light field

Lightfields aim to capture the radiance of light rays within the scene.

NeRF

Main article: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

NeRF preprocesses unstructured light fields into a neural network (MLP) representation which predicts/interpolates unknown light rays based on the known light rays in the scene.

Resources

• yenchenlin/awesome-NeRF
• NeRF explosion 2020

Layered Representations

Some notable people here are Noah Snavely and Richard Tucker.
Representations here vary from implicit (MPI, MSI) to explicit (LDI, Point Clouds).

Multi-plane Image (MPI)

• Stereo Magnification (SIGGRAPH 2018)
• DeepView (CVPR 2019)

Layered Depth Image (LDI)

• One-shot 3D photography
• Casual 3D Photography

Multi-sphere Image (MSI)

• Matryodshka (ECCV 2020) - Renders 6-dof video from ODS videos.

Point Clouds

• SynSin

Classical Reconstruction

Reconstruction aims to recreate the 3D scene from a set of input images.
Techniques include structure from motion, multi-view stereo.
This type of reconstruction is also studied in the field of photogrammetry.