Essential Matrix
An essential matrix, denoted \(\displaystyle \mathbf{E}\), is a \(\displaystyle 3 \times 3\) matrix relating camera parameters.
You can compute the essential matrix based on features matches between two images.
Using the essential matrix, you can extract the relative rotation and translation between two cameras.
Given feature points \(\displaystyle \mathbf{x}\) and \(\displaystyle \mathbf{x'}\) from two images, the essential matrix satisfies the equation \(\displaystyle \mathbf{x}'^T \mathbf{E} \mathbf{x} = 0\)
Background and Derivation
Much of this section is from [1]
A pinhole camera with \(\displaystyle 3 \times 4\) projection matrix \(\displaystyle P = K(R | -RT)\) takes points \(\displaystyle \mathbf{x} = (x, y, z)^T\) and projects them to \(\displaystyle \mathbf{u} = (u, v, w)^T = \mathbf{R}(\mathbf{x} - \mathbf{t})\).
We now consider two cameras:
Camera 1 is at the origin of world space (or it's object space) \(\displaystyle P = (I | 0)\).
Camera 2 is displaced with some rotation \(\displaystyle R\) and translation \(\displaystyle R\), \(\displaystyle P' = (R | -RT)\).
Any point \(\displaystyle \mathbf{u} = (u,v,w)^T\) in camera 1 is represented by an epipolar line in camera 2.
Under camera 2, the position of camera 1 is \(\displaystyle -RT\) and \(\displaystyle P' (u,v,w,0)^T = R\mathbf{u}\) is somewhere on this epipolar line.
Thus the line can be calculated by taking the cross product between the camera origin and \(\displaystyle \mathbf{u}\).
\[
(p,q,r)^T = RT \times R\mathbf{u} = R(T \times \mathbf{u}) = R[T]_{\times} \mathbf{u}
\]
Now the line is represented by \(\displaystyle \{(u',v',w') \mid pu' + qv' + rw' = 0\}\), i.e. all points orthogonal to \(\displaystyle (p,q,r)^T\).
Given a vector \(\displaystyle \mathbf{t}\), the matrix form of its cross product is:
\(\displaystyle
[\mathbf{t}]_{\times} =
\begin{bmatrix}
0 & -t_z & t_y\\
t_z & 0 & -t_x\\
-t_y & t_x & 0
\end{bmatrix}
\)
- \(\displaystyle [\mathbf{t}]_{\times} \mathbf{u} = \mathbf{t} \times \mathbf{u}\)
- This matrix is skew-symmetric. I.e. \(\displaystyle [\mathbf{t}]^T_{\times} = -[\mathbf{t}]_{\times}\)
Given feature points \(\displaystyle \mathbf{x}\) and \(\displaystyle \mathbf{x'}\) from two images, we can relate them with a rotation \(\displaystyle \mathbf{R}\) and a translation \(\displaystyle \mathbf{t}\) such that \(\displaystyle (\mathbf{R}\mathbf{x} + \mathbf{t}) = x'\).
Properties
- A \(\displaystyle 3 \times 3\) matrix is an essential matrix iff two of its singular values are equal and the third value is \(\displaystyle 0\)
See Bartoli and Olsen[2].
Calculating the Essential Matrix from two images
Planar Images
Spherical Images
Here we assume an equirectangular projection.