Debugging ML Models: Difference between revisions

Debugging ML Models (view source)

47 bytes added , 3 August 2020

5,337

edits

@@ Line 8: / Line 8: @@
 ** Check that your loss is implemented correctly and taken against the correct ground truth image.
 * Dump all inputs and outputs into [[TensorBoard]]. You may have an unexpected input or output somewhere.
-* If it looks like it is underfitting (e.g. if the training output and validation output are both blurry):
-** Train for 4x as long until the training loss and validation loss both flatten.
-** Increase or decrease the learning rate one magnitude.
-** Make sure the batch size is a multiple of 2. Try increasing it to get more stable gradient updates or decreasing it to get faster iterations.
 * Try disabling any tricks you have like dropout.
 * Make sure there is no activation on the final layer.
+==Underfitting==
+If it looks like it is underfitting (e.g. if the training output and validation output are both blurry), then you can try the following.
+* Train for 4x as long until the training loss and validation loss both flatten.
+* Increase or decrease the learning rate one magnitude.
+* Make sure the batch size is a multiple of 2. Try increasing it to get more stable gradient updates or decreasing it to get faster iterations.