5,337
edits
Deep Learning: Difference between revisions
→Optimization
| Line 38: | Line 38: | ||
# Sample some batch <math display="inline">B</math> | # Sample some batch <math display="inline">B</math> | ||
# <math display="inline">w^{(t+1)} = w^{(t)} - \eta \frac{1}{|B|} \sum_{i \in B} \nabla_{W} l(f_{W}(x_i), y_i)</math> | # <math display="inline">w^{(t+1)} = w^{(t)} - \eta \frac{1}{|B|} \sum_{i \in B} \nabla_{W} l(f_{W}(x_i), y_i)</math> | ||
Optimizers/Solvers: | |||
* Momentum | |||
* RMSProp | |||
* Adam | |||
==Misc== | ==Misc== | ||