5,323
edits
PyTorch: Difference between revisions
→Optimizations
| Line 108: | Line 108: | ||
Note that [https://en.wikipedia.org/wiki/Bfloat16_floating-point_format <code>bfloat16</code>] is different from IEEE float16. bfloat16 has fewer mantissa bits (8 exp, 7 mantissa) and is used by Google's TPUs. In contrast, float16 has 5 exp and 10 mantissa bits. | Note that [https://en.wikipedia.org/wiki/Bfloat16_floating-point_format <code>bfloat16</code>] is different from IEEE float16. bfloat16 has fewer mantissa bits (8 exp, 7 mantissa) and is used by Google's TPUs. In contrast, float16 has 5 exp and 10 mantissa bits. | ||
===torch.multinomial=== | |||
If you need to sample with a lot of categories and with replacement, it may be faster to use `torch.cumsum` to build a CDF and `torch.searchsorted`. | |||
==Classification== | ==Classification== | ||