Image-based rendering: Difference between revisions
No edit summary |
No edit summary |
||
| Line 10: | Line 10: | ||
NeRF preprocesses unstructured light fields into a neural network (MLP) representation which predicts/interpolates unknown light rays based on the known light rays in the scene. | NeRF preprocesses unstructured light fields into a neural network (MLP) representation which predicts/interpolates unknown light rays based on the known light rays in the scene. | ||
==Layered Representations== | |||
Some notable people here are [https://scholar.google.com/citations?user=Db4BCX8AAAAJ&hl=en&oi=ao Noah Snavely] and [https://scholar.google.com/citations?user=IkpNZAoAAAAJ&hl=en&oi=sra Richard Tucker]. | |||
Representations here vary from very implicit (MPI, MSI) to somewhat explicit (LDI, Point Clouds). | |||
==Reconstruction== | ===Multi-plane Image (MPI)=== | ||
* [https://arxiv.org/abs/1805.09817 Stereo Magnification (SIGGRAPH 2018)] | |||
* [https://openaccess.thecvf.com/content_CVPR_2019/html/Flynn_DeepView_View_Synthesis_With_Learned_Gradient_Descent_CVPR_2019_paper.html DeepView (CVPR 2019)] | |||
===Layered Depth Image (LDI)=== | |||
* [https://facebookresearch.github.io/one_shot_3d_photography/ One-shot 3D photography] | |||
* Casual 3D Photography | |||
===Multi-sphere Image (MSI)=== | |||
* [http://visual.cs.brown.edu/projects/matryodshka-webpage/ Matryodshka (ECCV 2020)] - Renders 6-dof video from ODS videos. | |||
===Point Clouds=== | |||
* [https://www.robots.ox.ac.uk/~ow/synsin.html SynSin] | |||
==Classical Reconstruction== | |||
Reconstruction aims to recreate the 3D scene from a set of input images. | Reconstruction aims to recreate the 3D scene from a set of input images. | ||
Techniques include structure from motion, multi-view stereo. | Techniques include structure from motion, multi-view stereo. | ||
This is also known as photogrammetry. | This is also known as photogrammetry. | ||
Revision as of 20:09, 15 January 2021
Image-based rendering focuses on rendering scenes from existing captured or rasterized images, typically from a new viewpoint.
Recent research allows adding new objects, performing relighting, and other AR effects.
Light Fields
Lightfields aim to capture the radiance of light rays within the scene.
NeRF
NeRF preprocesses unstructured light fields into a neural network (MLP) representation which predicts/interpolates unknown light rays based on the known light rays in the scene.
Layered Representations
Some notable people here are Noah Snavely and Richard Tucker.
Representations here vary from very implicit (MPI, MSI) to somewhat explicit (LDI, Point Clouds).
Multi-plane Image (MPI)
Layered Depth Image (LDI)
- One-shot 3D photography
- Casual 3D Photography
Multi-sphere Image (MSI)
- Matryodshka (ECCV 2020) - Renders 6-dof video from ODS videos.
Point Clouds
Classical Reconstruction
Reconstruction aims to recreate the 3D scene from a set of input images.
Techniques include structure from motion, multi-view stereo.
This is also known as photogrammetry.