Diffusion Models: Difference between revisions
| Line 17: | Line 17: | ||
The forward diffusion can be sampled for any <math>t</math> using:<br> | The forward diffusion can be sampled for any <math>t</math> using:<br> | ||
<math>\mathbf{x}_{t} = \sqrt{\bar\alpha_t} \mathbf{x}_0 - \sqrt{1-\bar\alpha_t} \boldsymbol{\epsilon}</math> where <math>\bar\alpha_t = \prod_{s=1}^{t}(1-\beta{s})</math> | <math>\mathbf{x}_{t} = \sqrt{\bar\alpha_t} \mathbf{x}_0 - \sqrt{1-\bar\alpha_t} \boldsymbol{\epsilon}</math> where <math>\bar\alpha_t = \prod_{s=1}^{t}(1-\beta{s})</math> | ||
The loss function is based on the mean of the posterior.<br> | |||
If we estimate <math>\mu_\theta(x_t, t)</math> as <math>\frac{1}{\sqrt{\alpha_t}} \left( x_t - \frac{\beta_t}{\sqrt{1-\bar\alpha_t}} \boldsymbol{\epsilon}_\theta (\mathbf{x}_t, t) \right)</math>, then the loss function simplifies to:<br> | |||
<math>E \left[ \frac{\beta^2_t}{2\sigma^2_t \alpha (1-\bar\alpha_t)} \Vert \boldsymbol{\epsilon} - \boldsymbol{\epsilon}_\theta( \sqrt{\bar\alpha_t} \mathbf{x}_0 - \sqrt{1-\bar\alpha_t} \boldsymbol{\epsilon}, t) \Vert^2 \right]</math> | |||
===Super-resolution=== | ===Super-resolution=== | ||