5,323
edits
PyTorch: Difference between revisions
→Reducing memory usage
(→Memory) |
|||
| Line 81: | Line 81: | ||
When possible, use functions which return new views of existing tensors rather than making duplicates of tensors: | When possible, use functions which return new views of existing tensors rather than making duplicates of tensors: | ||
* [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.permute <code>permute</code>] | * [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.permute <code>permute</code>] | ||
* [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.expand <code>expand</code>] instead of <code>repeat</code> | * [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.expand <code>expand</code>] instead of [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.repeat <code>repeat</code>] | ||
* [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.view <code>view</code>] | * [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.view <code>view</code>] | ||
Note that <code>permute</code> does not change the underlying data. | Note that <code>permute</code> does not change the underlying data. | ||
This can result in a minor performance hit if you repeatedly use a contiguous tensor with a channels last tensor. | This can result in a minor performance hit which PyTorch will warn you about if you repeatedly use a contiguous tensor with a channels last tensor. | ||
To address this, call [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.contiguous <code>contiguous</code>] on the tensor with the new memory format. | To address this, call [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.contiguous <code>contiguous</code>] on the tensor with the new memory format. | ||