5,323
edits
Unsupervised Learning: Difference between revisions
→Variational Bound
(→M-Step) |
|||
| Line 176: | Line 176: | ||
\begin{aligned} | \begin{aligned} | ||
KL(Q_i(z) \Vert P(z|X)) &= E_{Q} \left[ \log(\frac{Q_i(z)}{P(z|X)}) \right]\\ | KL(Q_i(z) \Vert P(z|X)) &= E_{Q} \left[ \log(\frac{Q_i(z)}{P(z|X)}) \right]\\ | ||
&=E_Q(\log(\frac{Q_i(z) P(X^{(i)})}{P( | &=E_Q(\log(\frac{Q_i(z) P(X^{(i)})}{P(X|z)P(z)})\\ | ||
&=E_Q(\log(\frac{Q_i(z)}{P(z)})) + \log(P(x^{(i)})) - E_Q(\log(P(X|z))\\ | &=E_Q(\log(\frac{Q_i(z)}{P(z)})) + \log(P(x^{(i)})) - E_Q(\log(P(X|z))\\ | ||
&=KL(Q_i(z) \Vert P(z)) + \log(P(x^{(i)}) - E_Q(\log(P(X|z)) | &=KL(Q_i(z) \Vert P(z)) + \log(P(x^{(i)}) - E_Q(\log(P(X|z)) | ||