Skip to main content

Install Docker with NVIDIA Support

This guide covers installing Docker and the NVIDIA Container Toolkit on Ubuntu 26.04. For NVIDIA driver and CUDA installation, see the Install CUDA guide.

Install Docker​

0. Uninstall old versions​

Before installing, remove any conflicting packages:

sudo apt remove $(dpkg --get-selections docker.io docker-compose docker-compose-v2 docker-doc podman-docker containerd runc | cut -f1)

1. Set up Docker's apt repository​

# Install dependencies
sudo apt update
sudo apt install ca-certificates curl

# Add Docker's official GPG key
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc

# Add the repository to apt sources
sudo tee /etc/apt/sources.list.d/docker.sources << EOF
Types: deb
URIs: https://download.docker.com/linux/ubuntu
Suites: $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}")
Components: stable
Signed-By: /etc/apt/keyrings/docker.asc
EOF

2. Install the Docker packages​

sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin

3. Manage Docker as a non-root user​

sudo usermod -aG docker $USER
newgrp docker

4. Verify the installation​

docker run hello-world

Install NVIDIA Container Toolkit​

5. Install the toolkit​

sudo apt-get install nvidia-container-toolkit

6. Configure the Docker runtime​

Edit the Docker daemon configuration file:

sudo vim /etc/docker/daemon.json

Make sure the following contents are present:

{
  "runtimes": {
    "nvidia": {
      "args": [],
      "path": "nvidia-container-runtime"
    }
  },
  "data-root": "/home/docker"
}

7. Change the containerd root​

sudo vim /etc/containerd/config.toml

Find the line:

#root = "/var/lib/containerd"

Uncomment it and change it to:

root = "/home/docker/containerd"

8. Restart Docker​

sudo systemctl restart docker

Validation​

Pick an NVIDIA image from Docker Hub and verify GPU access:

docker run --gpus all nvidia/cuda:13.2.0-base-ubuntu26.04 nvidia-smi
tip

For more advanced Docker usage including GUI applications, X11 forwarding, and SLURM integration, see the Docker Usage guide.