5,321
edits
Unsupervised Learning: Difference between revisions
→EM Algorithm
| Line 77: | Line 77: | ||
</math><br> | </math><br> | ||
<math> | <math> | ||
\implies \log[E_{Q}(\frac{Pr(x^{(i)}, z^{(i)}; \theta)}{Q})] \geq E_{Q}[\log(\frac{Pr(X^{(i)}, q^{(i)}; \theta)}{Q^{(i)}(j)}] | \implies \log\left[E_{Q}\left(\frac{Pr(x^{(i)}, z^{(i)}; \theta)}{Q}\right)\right] \geq E_{Q} \left[ \log \left(\frac{Pr(X^{(i)}, q^{(i)}; \theta)}{Q^{(i)}(j)} \right)\right] | ||
</math><br> | </math><br> | ||
; E-Step | ; E-Step | ||
We will fix <math>\theta</math> and optimize wrt <math>Q</math> | We will fix <math>\theta</math> and optimize wrt <math>Q</math>.<br> | ||
Jensen's inequality holds with equality iff either the function is linear or if the random variable is degenerate.<br> | Jensen's inequality holds with equality iff either the function is linear or if the random variable is degenerate.<br> | ||
Since log is not linear, we will assume <math>\frac{P(x^{(i)}, z^{(i)}=j; \theta) | Since log is not linear, we will assume <math>\frac{P(x^{(i)}, z^{(i)}=j; \theta) | ||