5,322
edits
Machine Learning: Difference between revisions
→Cross Entropy
(→SVM) |
|||
| Line 23: | Line 23: | ||
The cross entropy loss is | The cross entropy loss is | ||
* <math>J(\theta) = \sum [(y^{(i)})\log(h_\theta(x)) + (1-y^{(i)})\log(1-h_\theta(x))]</math> | * <math>J(\theta) = \sum [(y^{(i)})\log(h_\theta(x)) + (1-y^{(i)})\log(1-h_\theta(x))]</math> | ||
;Notes | ;Notes | ||
* This is the sum of the log probabilities of picking the correct class (i.e. p if y=1 or 1-p if y=0). | |||
* If our model is <math>g(\theta^Tx^{(i)})</math> where <math>g(x)</math> is the sigmoid function <math>\frac{e^x}{1+e^x}</math> then this is convex | * If our model is <math>g(\theta^Tx^{(i)})</math> where <math>g(x)</math> is the sigmoid function <math>\frac{e^x}{1+e^x}</math> then this is convex | ||