Light Field Duality: Concept and Applications: Difference between revisions

No edit summary
No edit summary
 
(9 intermediate revisions by the same user not shown)
Line 1: Line 1:
Light Field Duality: Concept and Applications
Light Field Duality: Concept and Applications (VRST 2002)


Authors: George Chen, Li Hong, Kim Ng, Peter McGuinness, Christian Hofsetz, Yang Liu, Nelson Max
Authors: George Chen, Li Hong, Kim Ng, Peter McGuinness, Christian Hofsetz, Yang Liu, Nelson Max
Affiliations: STMicroelectronics, UC Davis
Affiliations: STMicroelectronics, UC Davis


* [https://dl.acm.org/doi/pdf/10.1145/585740.585743 Paper (escholorship mirror)]
* [https://dl.acm.org/doi/abs/10.1145/585740.585743 ACM DL] [https://dl.acm.org/doi/pdf/10.1145/585740.585743 ACL DL PDF] [https://escholarship.org/content/qt0fp8x522/qt0fp8x522.pdf escholorship.org mirror]


==Background==
==Background==
Line 25: Line 25:
See the figure to the right.
See the figure to the right.


With some algebra, this constraint can be written as either:
With some algebra, this constraint can be written as either:
<math>
<math>
\begin{bmatrix}
\begin{bmatrix}
Line 40: Line 40:
\end{bmatrix}
\end{bmatrix}
</math>   
</math>   
Or: 
<math>
<math>
\begin{bmatrix}
\begin{bmatrix}
Line 53: Line 54:
</math>   
</math>   
Given a point, this constraint defines the hyperline in the 4D light field space.
Given a point, this constraint defines the hyperline in the 4D light field space.
Henceforth, a hyperline will be represented as <math>(a, b, c, d)</math> where hyperpoints on said line satisfy: 
<math>
\begin{bmatrix}
a & 0 & b & 0\\
0 & a & 0 & b
\end{bmatrix}
\begin{bmatrix}
s \\ t \\ u \\ v
\end{bmatrix}
=
\begin{bmatrix}
c \\ d
\end{bmatrix}
</math>


The paper has the following remarks:
The paper has the following remarks:
* Points in one space are lines in the other.
* Points in one space are lines in the other.
* (?) A bundle of lines in word space correspond to a set of hyperlinear points in the light field.
* A bundle of lines in world space correspond to a set of hyperlinear points in the light field.
* (?) A set of colinear points in world space corresponds to a bundle of hyperlines in the light field.
* A set of colinear points in world space corresponds to a bundle of hyperlines in the light field.


In the light field space, you get:
In the light field space, you get:
Line 64: Line 80:


==CHL Light Field Rendering==
==CHL Light Field Rendering==
Each camera <math>i</math> is a point in world coordinates which corresponds to a hyperline <math>\mathbf{l}_i=(a_i, b_i, c_i, d_i)</math> in the light field. 
If we want to render from a new viewpoint, for each render pixel, we have a target ray which corresponds a point on the light field <math>\mathbf{r} = (s_r, t_r, u_r, v_r)</math>. 
For each camera/hyperline <math>i</math>, you can find the ray/hyperpoint which minimizes some distance function: 
<math>
\begin{aligned}
d^2 &= \Vert (s_i,t_i,u_i,v_i) - (s_r,t_r,u_r,v_r) \Vert^2_2\\
&= (s_i - s_r)^2 + (t_i - t_r)^2 + (u_i - u_r)^2 + (v_i - v_r)^2
\end{aligned}
</math>. 
There is a closed form solution provided in the paper. 
Then simply blend with weights as inverse distance.
===Dynamic focal plane===
You can add a slope loss to the distance function to make sure rays point in the same direction: 
<math>
\begin{aligned}
d^2 &= \Vert (s_i,t_i,u_i,v_i) - (s_r,t_r,u_r,v_r) \Vert^2_2 +\\
&\hspace{5mm}\beta\Vert (s_i-u_i, t_i-v_i) - (s_r-u_r, t_r-v_r) \Vert^2_2
\end{aligned}
</math>.
If the virtual camera is not at the same position as the source cameras, then the lines will not be parallel and will intersect. 
<math>\beta</math> can be used to control the depth at which they intersect.


==GHL Light Field Rendering==
==GHL Light Field Rendering==
Rendering lightfields using geometry hyperlines is equivalent to rendering point clouds. 
For each virtual ray, you can compute the dual <math>(s_r, t_r, u_r, v_r)</math>. 
Then for each GHL, you can compute the optimal ray closest to the virtual ray. 
Then blend over selected GHLs.
The paper has details on addressing issues with holes and opacity using clustering.