Foveated Rendering

From David's Wiki
\( \newcommand{\P}[]{\unicode{xB6}} \newcommand{\AA}[]{\unicode{x212B}} \newcommand{\empty}[]{\emptyset} \newcommand{\O}[]{\emptyset} \newcommand{\Alpha}[]{Α} \newcommand{\Beta}[]{Β} \newcommand{\Epsilon}[]{Ε} \newcommand{\Iota}[]{Ι} \newcommand{\Kappa}[]{Κ} \newcommand{\Rho}[]{Ρ} \newcommand{\Tau}[]{Τ} \newcommand{\Zeta}[]{Ζ} \newcommand{\Mu}[]{\unicode{x039C}} \newcommand{\Chi}[]{Χ} \newcommand{\Eta}[]{\unicode{x0397}} \newcommand{\Nu}[]{\unicode{x039D}} \newcommand{\Omicron}[]{\unicode{x039F}} \DeclareMathOperator{\sgn}{sgn} \def\oiint{\mathop{\vcenter{\mathchoice{\huge\unicode{x222F}\,}{\unicode{x222F}}{\unicode{x222F}}{\unicode{x222F}}}\,}\nolimits} \def\oiiint{\mathop{\vcenter{\mathchoice{\huge\unicode{x2230}\,}{\unicode{x2230}}{\unicode{x2230}}{\unicode{x2230}}}\,}\nolimits} \)

Foveated rendering is a method to render fewer pixels than are shown to the user.
The underlying idea is that the human retina is very high-resolution only around the gaze point where the user is focusing. The remainder, or peripheral region, is low-resolution. Therefore more rendering power should be focused around the gaze region.
Typically this can be done in one of several ways:

  • Tiling
    • Approach 1: Render the whole image in a low resolution (in terms of density) and then render the foveal region in a higher resolution. Then place the high-resolution foveal region over the low-resolution image.
    • Approach 2: Tile the output into separate regions. Render different regions at different resolutions.
  • Geometric subsampling
    • Render a subset of pixels to be shown. The sampling is denser near the foveal region and sparser in the periphery.
    • Typically uses a log-polar transformation where y-axis is polar (0 to \(\displaystyle 2\pi\)) and the x-axis is log distance from the focus center.
    • Requires some form of interpolation to reconstruct the missing pixels (e.g. bilinear or using a GAN)

Resources