Image quality assessment

From David's Wiki
Jump to navigation Jump to search
\( \newcommand{\P}[]{\unicode{xB6}} \newcommand{\AA}[]{\unicode{x212B}} \newcommand{\empty}[]{\emptyset} \newcommand{\O}[]{\emptyset} \newcommand{\Alpha}[]{Α} \newcommand{\Beta}[]{Β} \newcommand{\Epsilon}[]{Ε} \newcommand{\Iota}[]{Ι} \newcommand{\Kappa}[]{Κ} \newcommand{\Rho}[]{Ρ} \newcommand{\Tau}[]{Τ} \newcommand{\Zeta}[]{Ζ} \newcommand{\Mu}[]{\unicode{x039C}} \newcommand{\Chi}[]{Χ} \newcommand{\Eta}[]{\unicode{x0397}} \newcommand{\Nu}[]{\unicode{x039D}} \newcommand{\Omicron}[]{\unicode{x039F}} \DeclareMathOperator{\sgn}{sgn} \def\oiint{\mathop{\vcenter{\mathchoice{\huge\unicode{x222F}\,}{\unicode{x222F}}{\unicode{x222F}}{\unicode{x222F}}}\,}\nolimits} \def\oiiint{\mathop{\vcenter{\mathchoice{\huge\unicode{x2230}\,}{\unicode{x2230}}{\unicode{x2230}}{\unicode{x2230}}}\,}\nolimits} \)

Methods for Image quality assessment

The standard metrics are mean-squared error, peak signal to noise ratio (psnr), and structural similarity (ssim).

Standard Images and Video

\(\DeclareMathOperator{\mean}{mean}\)

  • MSE
    \(\displaystyle \begin{align}MSE = \mean((I_1 - I_2)^2)\end{align}\)
  • PSNR
    \(\displaystyle \begin{align}PSNR=10\log_{10}(\frac{R^2}{MSE})\end{align}\)
    • where R^2 is the maximum fluctuation (e.g. 1.0 for [0-1] float images, 255 for uint8).
  • SSIM

For video, simply average MSE over all frames and then compute the PSNR over the averaged MSE.
Typically, for a yuv420 video, people only calculate MSE or PSNR for the luminance (Y) channel due to chroma subsampling.

FFmpeg can compute PSNR and SSIM.

Foveated Quality Assessment

  • Lee et al.[1] propose Foveated signal to noise ratio (FSNR) which measures the signal to noise ratio in a curvilinear space. However they do not provide the exact equations to compute the curvilienar space.

Spherical Quality Assessment

  • WS-PSNR is a standard PSNR calculation where the mean squared error is weighted by the size of each pixel.
    \(\displaystyle \begin{align} WMSE &= \frac{1}{\sum_{i,j}w(i,j)} \sum_{i,j} (I_1(i,j)-I_2(i,j))^2 * w(i,j)\\ WS\_PSNR &= 10\log_{10}(\frac{R^2}{WMSE})\\ \end{align} \)
    • The weights are \(w(i,j) = \sin(\frac{j+0.5}{N}\pi) = \cos(\frac{j+0.5-N/2}{N}\pi)\).
  • Yu et al.[2] propose Spherical PSNR (S-PSNR). In S-PSNR, points are randomly sampled on a sphere and back projected to the reference and reconstructed images. In practice, these randomly sampled points need to be saved for reproducibility. They use 655262 points which are available on their repo.


Samsung has a 360tools program which can compute WS-PSNR, S_PSNR. However, it requires raw .yuv video.

Resources

References

  1. Sanghoon Lee, M.S. Pattichis, A.C. Bovik, (2002) Foveated video quality assessment IEEE Multimedia 2002.
  2. Matt Yu, Haricharan Lakshman, Bernd Girod (2015) A Framework to Evaluate Omnidirectional Video Coding Schemes ISMAR 2015.