PyTorch: Difference between revisions

From David's Wiki
Line 48: Line 48:
Reducing memory usage
Reducing memory usage
* Save loss using [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.item <code>.item()</code>] which returns a standard Python number
* Save loss using [https://pytorch.org/docs/stable/tensors.html#torch.Tensor.item <code>.item()</code>] which returns a standard Python number
* For non-scalar items, use <code>my_var.detach().cpu().numpy()</code>
* <code>detach()</code> deletes the item from the autograd edge
* <code>cpu()</code> copies the tensor to the CPU
* <code>numpy()</code> returns a numpy view of the tensor
==TensorBoard==
{{main | TensorBoard}}
See [https://pytorch.org/docs/stable/tensorboard.html PyTorch Docs: Tensorboard]
<syntaxhighlight lang="python">
from torch.utils.tensorboard import SummaryWriter
writer = SummaryWriter(log_dir="./runs")
writer.add_scalar("train_loss", loss_np, step)
# Optionally flush e.g. at checkpoints
writer.flush()
# Close the writer (will flush)
writer.close()
</syntaxhighlight>

Revision as of 14:01, 29 July 2020

PyTorch is a popular machine learning library developed by Facebook

Installation

See PyTorch Getting Started

# If using conda, python 3.5+, and CUDA 10.0 (+ compatible cudnn)
conda install pytorch torchvision cudatoolkit=10.0 -c pytorch

Getting Started

import torch
import torch.nn as nn

# Training
for epoch in range(epochs):
  running_loss = 0.0
    for i, data in enumerate(trainloader, 0):
        # get the inputs; data is a list of [inputs, labels]
        inputs, labels = data

        # zero the parameter gradients
        optimizer.zero_grad()

        # forward + backward + optimize
        outputs = net(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

Importing Data

See Data Loading Tutorial

Usage

torch.nn.functional

PyTorch Documentation

F.grid_sample

Doc
This function allows you to perform interpolation on your input tensor.
It is very useful for resizing images or warping images.

Memory Usage

Reducing memory usage

  • Save loss using .item() which returns a standard Python number
  • For non-scalar items, use my_var.detach().cpu().numpy()
  • detach() deletes the item from the autograd edge
  • cpu() copies the tensor to the CPU
  • numpy() returns a numpy view of the tensor

TensorBoard

See PyTorch Docs: Tensorboard

from torch.utils.tensorboard import SummaryWriter
writer = SummaryWriter(log_dir="./runs")

writer.add_scalar("train_loss", loss_np, step)

# Optionally flush e.g. at checkpoints
writer.flush()

# Close the writer (will flush)
writer.close()