5,337
edits
Debugging ML Models: Difference between revisions
→NaNs and Infs
| Line 82: | Line 82: | ||
** For Tensorflow see [https://www.tensorflow.org/api_docs/python/tf/clip_by_norm tf.clip_by_norm] and [https://www.tensorflow.org/api_docs/python/tf/clip_by_value tf.clip_by_value]. | ** For Tensorflow see [https://www.tensorflow.org/api_docs/python/tf/clip_by_norm tf.clip_by_norm] and [https://www.tensorflow.org/api_docs/python/tf/clip_by_value tf.clip_by_value]. | ||
* Using a safe divide which forces the denominator to have values with abs > EPS. | * Using a safe divide which forces the denominator to have values with abs > EPS. | ||
** Note that this can cutoff gradients. | |||
==Soft Operations== | ==Soft Operations== | ||