Image-based rendering

Image-based rendering focuses on rendering scenes from existing captured or rasterized images, typically from a new viewpoint.
Recent research allows adding new objects, performing relighting, and other AR effects.

Implicit Representations

Light Fields

Main article: Light field

Lightfields aim to capture the radiance of light rays within the scene.

Light Field Networks

This is an implicit representation similar to NeRF.
However, you directly predict colors from light rays instead of performing volume rendering.

NeRF

Main article: NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis

NeRF preprocesses unstructured light fields into a neural network (MLP) representation which predicts radiance at different points during volume rendering.

Resources

• yenchenlin/awesome-NeRF
• NeRF explosion 2020

Layered Representations

Some notable people here are Noah Snavely and Richard Tucker.
Representations here vary from implicit (MPI, MSI) to explicit (LDI, Point Clouds).

Multi-plane Image (MPI)

• Stereo Magnification (SIGGRAPH 2018)
• DeepView (CVPR 2019)

Layered Depth Image (LDI)

• One-shot 3D photography
• Casual 3D Photography

Multi-sphere Image (MSI)

• Matryodshka (ECCV 2020) - Renders 6-dof video from ODS videos.

Point Clouds

• SynSin

Classical Reconstruction

Reconstruction aims to recreate the 3D scene from a set of input images.
Techniques include structure from motion, multi-view stereo.
This type of reconstruction is also studied in the field of photogrammetry.