5,350
edits
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis: Difference between revisions
NeRF: Representing Scenes as Neural Radiance Fields for View Synthesis (view source)
Revision as of 21:44, 8 February 2021
, 8 February 2021no edit summary
(→Method) |
No edit summary |
||
| Line 1: | Line 1: | ||
NeRF Representing Scenes as Neural Radiance Fields for View Synthesis (ECCV 2020) | NeRF Representing Scenes as Neural Radiance Fields for View Synthesis (ECCV 2020) | ||
Authors: | * Authors: Ben Mildenhall, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, Ren Ng | ||
* Affiliations: UC Berkeley, Google Research, UC San Diego | |||
* [https://www.matthewtancik.com/nerf Webpage] | * [https://www.matthewtancik.com/nerf Webpage] | ||
The main idea is to do volume rendering through an MLP.<br> | |||
As volume rendering is differentiable, the MLP will learn color and density at each position.<br> | |||
Training is supervised on multiple posed static images (e.g. 25 images). Poses can be obtained using COLMAP. | |||
Successful execution relies on two tricks: | |||
* A positional encoding. | |||
* Hierarchical volume sampling | |||
==Method== | ==Method== | ||
===Volume Rendering=== | |||
===Positional Encoding=== | |||
This is further explored in a followup paper<ref name="tancik2020fourier"></ref>. | |||
===Hierarchical volume sampling=== | |||
This idea is to uses a course network to influence sampling of the fine network. | |||
==Architecture== | ==Architecture== | ||
They use a standard MLP with extra inputs.<br> | |||
Their MLP has 9 layers with 256-dim features and one layer with 128-dim features (~600k parameters?).<br> | |||
See their paper and code for more details. | |||
==Experiments== | ==Experiments== | ||
They compare with their past work on Local Light Field Fusion as well as a few other view synthesis papers. | |||
They also publish a dataset available [https://drive.google.com/drive/folders/128yBriW1IG_3NJ5Rp7APSTZsJqdJdfc1 on their Google Drive]. | |||
==Followup Work== | ==Followup Work== | ||
See [https://github.com/yenchenlin/awesome-NeRF yenchenlin/awesome-NeRF] | See [https://github.com/yenchenlin/awesome-NeRF yenchenlin/awesome-NeRF] | ||
===Positional Encoding=== | |||
* [https://arxiv.org/abs/2006.10739 Fourier Features Let Networks Learn High Frequency Functions in Low Dimensional Domains] | |||
===Videos=== | ===Videos=== | ||
* [https://arxiv.org/abs/2011.12948 Deformable Neural Radiance Fields] | * [https://arxiv.org/abs/2011.12948 Deformable Neural Radiance Fields] | ||
* [https://arxiv.org/pdf/2011.12950.pdfSpace-time Neural Irradiance Fields for Free-Viewpoint Video] | * [https://arxiv.org/pdf/2011.12950.pdfSpace-time Neural Irradiance Fields for Free-Viewpoint Video] | ||
==References== | |||
{{reflist| | |||
<ref name="tancik2020fourier">Tancik, M., Srinivasan, P. P., Mildenhall, B., Fridovich-Keil, S., Raghavan, N., Singhal, U., ... & Ng, R. (2020). Fourier features let networks learn high frequency functions in low dimensional domains. [https://arxiv.org/abs/2006.10739 Arxiv]</ref> | |||
}} | |||