Docker (software)

Docker is a service for creating and managing linux containers.
Containers are the application layer of an OS and whatever software you're trying to run.

The container itself contains the code to be run along with the environment.
Anything which needs state is mounted as a volume to the container.

Installation

Ubuntu

Reference

Install Script

# Uninstall old docker
sudo apt-get remove docker docker-engine docker.io containerd runc

# Update repos
sudo apt update

# Install prerequisites
sudo apt-get install \
    apt-transport-https \
    ca-certificates \
    curl \
    gnupg-agent \
    software-properties-common

# Add official gpg key
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -

# Add docker repo
sudo add-apt-repository \
   "deb [arch=amd64] https://download.docker.com/linux/ubuntu \
   $(lsb_release -cs) \
   stable"

# Install
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io

Windows

Upgrade Windows to 2004 or newer
Install and enable WSL2
Install Docker Desktop

Guides

Get Started

Dockerfile

How to write a dockerfile

CLI Usage

Images

For the most part, you don't need to worry about images as docker run and docker-compose will download and build images for you as needed.

# List images.
docker image ls

# Prune unused images.
docker image prune -a

# Copy image.
docker tag $SOURCE $TARGET
docker push $TARGET

Notes

Pruning with docker system prune will also delete images.
Omitting -a will only prune dangling (untagged) images.

Containers

docker container ls

Run

docker run <container>

-p hostport:containerport to do port forwarding
- To restrict listing to localhost use -p 127.0.0.1:80:80
-it to be interactive with a pseudo-tty

stats

docker stats Returns information about container cpu usage, memory usage, network usage, and disk usage.

Networking

The default mode for networking is bridge. You should leave this for most of your containers.

bridge

In bridge mode, the docker service acts as a NAT and gives each container a separate local IP along with the docker host.
On linux, you can type ip a to see the ip address of the docker0 network interface.
On my server, it is 172.17.0.1/16.

To access services running on the host (such as MySQL or Postgres), you will need to make these services listen on this network interface and allow it through your firewall. I suggest using qoomon/docker-host which can redirect network traffic to the host.

When using docker-compose, services can access each other using their service name as the hostname. However, the port needs to be exposed in the compose file.

host

In this mode, docker processes have full access to your network. This can cause port conflicts if you are not careful. Furthermore, your docker service will have full access to your localhost. I do not recommend using this mode for most things.

GPUs

See docker guide

Setup

Go to nvidia-container-runtime and add the repo
Install nvidia-container-runtime

Run

Add --gpus all to your docker run command.

compose

See issue.

    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: 1
              capabilities: [gpu]

Windows

Notes on using docker with windows

Git bash paths

Reference
When mounting paths using git bash, you need to prepend a / to $(pwd)

docker-compose

Docker compose allows you to specify multiple docker services into a single docker-compose.yml file and run them all together.
You can also use it to setup docker commands instead of listing options in a shell script.

# Create a folder for your service and cd into it
# Make the docker-compose file.

# Run (i.e. build, create, and start)
docker-compose up -d

# Stop
docker-compose down

# Upgrade
docker-compose pull # Optional, reduces downtime
docker-compose up --force-recreate --build -d
docker image prune -f

Note that docker-compose restart will just restart existing containers. It will not recreate them.

Compose file reference

Compose file

See Compose file specification.

Previously, the Compose file (docker-compose.yml) required a version. Version 2 and version 3 had different options and not all options from version 2 were available in version 3. However, as of docker-compose v1.27+, you should no longer specify a version and options from both versions are supported.

Example docker-compose.yml

services:
  web:
    image: registry.gitlab.davidl.me/dli7319/davidl_me:latest
    restart: unless-stopped

Accessing the Host

Sometimes you may have services running on the host which you want to access from a container.
See docker-host for a container which can access the host.

Add the following to your docker compose to expose port 8201 to other containers:

  docker-host:
    image: qoomon/docker-host
    cap_add:
      - NET_ADMIN
      - NET_RAW

You do not need to add expose.

By default, networks are allocated with ip ranges:

172.17.0.0/12 with size /16
192.168.0.0/16 with size /20

If you want this to be more consistent, you can change it as follows: Set the following in /etc/docker/daemon.json:

{
  "default-address-pools":[
    {"base":"172.16.0.0/12","size":24}
  ]
}

Then restart your docker: sudo systemctl restart docker and prune networks docker network prune.
Next, in your firewall, allow connections to your localhost from 172.16.0.0/12.

ufw allow from 172.16.0.0/12 to any comment "from_docker"

Registries

The official Docker registry is Docker Hub.
However, Docker Hug has rate limits of 100 pulls per 6 hours.

Alternative public registries:

AWS ECR Gallery has a mirror for all official docker containers.

Caching

If you want your builds to be fast on CICD, you have to setup caching.

In particular you should:

Enable buildkit by setting the environment variable DOCKER_BUILDKIT=1
Use --cache-from in your build.
Setup external caching, e.g. with --build-arg BUILDKIT_INLINE_CACHE=1.
- Use --arg BUILDKIT_INLINE_CACHE=1 if using docker buildx build.
For multistage builds, cache each stage in your container registry.

Resources

Useful Services

https://containrrr.dev/watchtower/ is a tool which will automatically update your docker containers when new images are published. It also has an http endpoint to trigger checks manually e.g. from CI/CD.

My Images

I have a few custom container images below:

Resources

freeCodeCamp.org Docker Tutorial for Beginners Video