5,332
edits
Deep Learning: Difference between revisions
→Theorem 4.1 (Uniform conditioning implies PL* condition)
| Line 93: | Line 93: | ||
\begin{aligned} | \begin{aligned} | ||
\frac{1}{2}\Vert \nabla f(w) \Vert^2 &= \frac{1}{2}\Vert (F(w)-y)^T \nabla F(w)\Vert^2\\ | \frac{1}{2}\Vert \nabla f(w) \Vert^2 &= \frac{1}{2}\Vert (F(w)-y)^T \nabla F(w)\Vert^2\\ | ||
&=\frac{1}{2}(F(w) | &=\frac{1}{2}(F(w)-y)^T \nabla F(w) \nabla F(w)^T (F(w)-y)\\ | ||
&\geq \frac{1}{2} \lambda_{\min}(K(w)) \Vert F(w)-y\Vert ^2\\ | &\geq \frac{1}{2} \lambda_{\min}(K(w)) \Vert F(w)-y\Vert ^2\\ | ||
&= \lambda_{\min}(K(w)) L(w)\\ | &= \lambda_{\min}(K(w)) L(w)\\ | ||