Live Intrinsic Video

Live Intrinsic Video (SIGGRAPH 2016)

Authors: Abhimitra Meka, Michael Zollhöfer, Christian Richardt, Christian Theobalt
Affiliations: Max Planck Institute for Informatics, Intel Visual Computing Institute

• Website

The goal of this paper is to decompose a standard RGB video stream \(\displaystyle I \in \mathbb{R}^3\) into a reflectance component in \(\displaystyle \mathbb{R}^3\) and a shading component in \(\displaystyle \mathbb{R}\): \(\displaystyle \mathbf{I} = \mathbf{R} \times S\).
This is an underconstrained problem since for each pixel you have 4 unknowns and 3 equations.
The paper forms an energy function, i.e. set of heuristics, with \(\displaystyle l_2\) and \(\displaystyle l_p\) norms.
Then it proposes an iterative optimizer which can minimize its energy function in real time (e.g. 36 ms for \(\displaystyle 1024 \times 576\)).

Method

They define an energy function which is defined per pixel \(\displaystyle x\):
\(\displaystyle E(x) = E_{data}(x) + E_{priors}(x) + E_{\text{non-local}}(x) +E_{clustering}(x)\)