Light Field Networks: Neural Scene Representations with Single-Evaluation Rendering

Authors:Vincent Sitzmann, Semon Rezchikov, William T. Freeman, Joshua B. Tenenbaum, Fredo Durand
Affiliations: MIT Links https://arxiv.org/abs/2106.02634

Summary

Background

See NeRF and SIREN.

Method

The idea here is to use light field rendering instead of volume rendering or SDF ray marching.
In this case, the input to the network is still a single point and direction but it represents the entire ray rather than the radiance at a particular point.
Thus, it is not necessary to sample across the entire ray and composite the samples.

Plucker coordinates

They use Plucker coordinates to encode rays instead of directly inputting the (point, direction) representation or using a two-plane parameterization.
The benefit is that Plucker coordinates are invariance to the selected point and can represent the entire 360 set of rays.
\(\displaystyle \mathbf{r} = (\mathbf{d},\mathbf{m}) \in \mathbb{R}^6\) where \(\displaystyle \mathbf{m}=\mathbf{p} \times \mathbf{d}\)