Essential Matrix

An essential matrix, denoted \(\displaystyle \mathbf{E}\), is a \(\displaystyle 3 \times 3\) matrix relating camera parameters.
You can compute the essential matrix based on features matches between two images.
Using the essential matrix, you can extract the relative rotation and translation between two cameras.

Given feature points \(\displaystyle \mathbf{x}\) and \(\displaystyle \mathbf{x'}\) from two images, the essential matrix satisfies the equation \(\displaystyle \mathbf{x}'^T \mathbf{E} \mathbf{x} = 0\)

Much of this is from An Investigation of the Essential Matrix] by Richard Hartley

Background and Derivation

Wikipedia: Epipolar Geometriy

A pinhole camera with \(\displaystyle 3 \times 4\) projection matrix \(\displaystyle P = K(R | -RT)\) takes points \(\displaystyle \mathbf{x} = (x, y, z)^T\) and projects them to \(\displaystyle \mathbf{u} = (u, v, w)^T = \mathbf{R}(\mathbf{x} - \mathbf{t})\).

We now consider two cameras: Camera 1 is at the origin of world space (or it's object space) \(\displaystyle P = (I | 0)\). Camera 2 is displaced with some rotation \(\displaystyle R\) and translation \(\displaystyle R\), \(\displaystyle P' = (R | -RT)\).
Any point \(\displaystyle \mathbf{u} = (u,v,w)^T\) in camera 1 is represented by an epipolar line in camera 2.
Under camera 2, the position of camera 1 is \(\displaystyle -RT\) and \(\displaystyle P' (u,v,w,0)^T = R\mathbf{u}\) is somewhere on this epipolar line. Thus the line can be calculated by taking the cross product between the camera origin and \(\displaystyle \mathbf{u}\). \[ (p,q,r)^T = RT \times R\mathbf{u} = R(T \times \mathbf{u}) = R[T]_{\times} \mathbf{u} \] Now the line is represented by \(\displaystyle \{(u',v',w') \mid pu' + qv' + rw' = 0\}\), i.e. all points orthogonal to \(\displaystyle (p,q,r)^T\).

Given a vector \(\displaystyle \mathbf{t}\), the matrix form of its cross product is:
\(\displaystyle [\mathbf{t}]_{\times} = \begin{bmatrix} 0 & -t_z & t_y\\ t_z & 0 & -t_x\\ -t_y & t_x & 0 \end{bmatrix} \)

• \(\displaystyle [\mathbf{t}]_{\times} \mathbf{u} = \mathbf{t} \times \mathbf{u}\)
• This matrix is skew-symmetric. I.e. \(\displaystyle [\mathbf{t}]^T_{\times} = -[\mathbf{t}]_{\times}\)

Now if \(\displaystyle \mathbf{u}'\) is a feature point from camera 2 matching point \(\displaystyle \mathbf{u}\) from camera 1, then it must lie on this epipolar line.
Thus \(\displaystyle \mathbf{u}' \in \{(u',v',w') \mid pu' + qv' + rw' = 0\} \implies \mathbf{u}'^T R[T]_{\times} \mathbf{u} = 0\). Now \(\displaystyle Q = R[T]_{\times}\) is the essential matrix.

Given 8 or more correspondence points between camera 1 and camera 2, you can solve for \(\displaystyle Q\) using the Wikipedia: Eight-point algorithm

Properties

• A \(\displaystyle 3 \times 3\) matrix is an essential matrix iff two of its singular values are equal and the third value is \(\displaystyle 0\)

Proof

See Bartoli and Olsen.

Calculating the Essential Matrix from two images

Main article: Wikipedia:Eight-point algorithm

Spherical Images

Here we assume an equirectangular projection.

3D points

See Wikipedia: Essential_matrix

Resources

• Wikipedia: Essential_matrix