Essential Matrix
An essential matrix, denoted \(\displaystyle \mathbf{E}\), is a \(\displaystyle 3 \times 3\) matrix relating camera parameters.
You can compute the essential matrix based on features matches between two images.
Using the essential matrix, you can extract the relative rotation and translation between two cameras.
Given feature points \(\displaystyle \mathbf{x}\) and \(\displaystyle \mathbf{x'}\) from two images, the essential matrix satisfies the equation \(\displaystyle \mathbf{x}'^T \mathbf{E} \mathbf{x} = 0\)
Background and Derivation
Much of this section is from [1]
A pinhole camera with \(\displaystyle 3 \times 4\) projection matrix \(\displaystyle P = K(R | -RT)\) takes points \(\displaystyle \mathbf{x} = (x, y, z)^T\) and projects them to \(\displaystyle \mathbf{u} = (u, v, w)^T = \mathbf{R}(\mathbf{x} - \mathbf{t})\).
We now consider two cameras:
Camera 1 is at the origin of world space (or it's object space) \(\displaystyle P = (I | 0)\).
Camera 2 is displaced with some rotation \(\displaystyle R\) and translation \(\displaystyle R\), \(\displaystyle P' = (R | -RT)\).
Any point \(\displaystyle \mathbf{u} = (u,v,w)^T\) in camera 1 is represented by an epipolar line in camera 2.
Under camera 2, the position of camera 1 is \(\displaystyle -RT\) and \(\displaystyle P' (u,v,w,0)^T = R\mathbf{u}\) is somewhere on this epipolar line.
Thus the line can be calculated by taking the cross product between the camera origin and \(\displaystyle \mathbf{u}\).
\[
(p,q,r)^T = RT \times R\mathbf{u} = R(T \times \mathbf{u}) = R[T]_{\times} \mathbf{u}
\]
Now the line is represented by \(\displaystyle \{(u',v',w') \mid pu' + qv' + rw' = 0\}\), i.e. all points orthogonal to \(\displaystyle (p,q,r)^T\).
Given a vector \(\displaystyle \mathbf{t}\), the matrix form of its cross product is:
\(\displaystyle
[\mathbf{t}]_{\times} =
\begin{bmatrix}
0 & -t_z & t_y\\
t_z & 0 & -t_x\\
-t_y & t_x & 0
\end{bmatrix}
\)
- \(\displaystyle [\mathbf{t}]_{\times} \mathbf{u} = \mathbf{t} \times \mathbf{u}\)
- This matrix is skew-symmetric. I.e. \(\displaystyle [\mathbf{t}]^T_{\times} = -[\mathbf{t}]_{\times}\)
Now if \(\displaystyle \mathbf{u}'\) is a feature point from camera 2 matching point \(\displaystyle \mathbf{u}\), then it must lie on this epipolar line. Thus \(\displaystyle \mathbf{u}' \in \{(u',v',w') \mid pu' + qv' + rw' = 0\} \implies \mathbf{u}'^T R[T]_{\times} \mathbf{u} = 0\). Now \(\displaystyle Q = R[T]_{\times}\) is the essential matrix.
Given 8 or more correspondence points between camera 1 and camera 2, you can solve for \(\displaystyle Q\) using the Wikipedia: Eight-point algorithm
Properties
- A \(\displaystyle 3 \times 3\) matrix is an essential matrix iff two of its singular values are equal and the third value is \(\displaystyle 0\)
See Bartoli and Olsen[2].
Calculating the Essential Matrix from two images
Planar Images
Spherical Images
Here we assume an equirectangular projection.
Determining rotation \(\displaystyle \mathbf{R}\) and translation \(\displaystyle \mathbf{t}\)
3D points
See Wikipedia: Essential_matrix