5,332
edits
Deep Learning: Difference between revisions
→Linear Regression
| Line 503: | Line 503: | ||
===Linear Regression=== | ===Linear Regression=== | ||
Assume we have a dataset: | Assume we have a dataset:<br> | ||
<math>\{(x_i, y_i)\}_{i=1}^{n}</math> | <math>\{(x_i, y_i)\}_{i=1}^{n}</math> | ||
<math>y_i \in \mathbb{R}</math> | <math>y_i \in \mathbb{R}</math><br> | ||
<math>x_i \in \mathbb{R}^d</math> | <math>x_i \in \mathbb{R}^d</math><br> | ||
<math>f(w, x) = w^t x</math> | <math>f(w, x) = w^t x</math> | ||
<math>L(w) = \frac{1}{2} \sum_{i=1}^{n}(y_i - f(w, x_i))^2</math> | <math>L(w) = \frac{1}{2} \sum_{i=1}^{n}(y_i - f(w, x_i))^2</math><br> | ||
<math>\min_{W} L(w)</math> | <math>\min_{W} L(w)</math><br> | ||
GD: <math>w(t+1) = w(t) - \eta_{t} \nabla L(w_t)</math> where our gradient is: | GD: <math>w(t+1) = w(t) - \eta_{t} \nabla L(w_t)</math> where our gradient is:<br> | ||
<math>\sum_{i=1}^{n}(y_i - f(w, x_i)) \nabla_{w} f(w_t, x_i) = \sum_{i=1}^{n}(y_i - f(w, x_i)) x_i</math> | <math>\sum_{i=1}^{n}(y_i - f(w, x_i)) \nabla_{w} f(w_t, x_i) = \sum_{i=1}^{n}(y_i - f(w, x_i)) x_i</math> | ||