SSD + Docker Setup

Once you have your Jetson set up by flashing the latest Jetson Linux (L4T) BSP on it or by flashing the SD card with the whole JetPack image, before embarking on testing out all the great generative AI applications using jetson-containers, you want to make sure you have a huge storage space for all the containers and the models you will download.

This guide shows how you can install SSD on your Jetson and set it up for Docker.

SSD

Physical Installation

1. Unplug power and any peripherals from the Jetson developer kit.

2. Physically install an NVMe SSD card on the carrier board of your Jetson developer kit, making sure to properly seat the connector and secure with the screw.

3. Reconnect any peripherals, and then reconnect the power supply to turn on the Jetson developer kit.

4. Once the system is up, verify that your Jetson identifies a new memory controller on PCI bus:

lspci

The output should look like:

0007:01:00.0 Non-Volatile memory controller: Marvell Technology Group Ltd. Device 1322 (rev 02)

Format and Set Up Auto-mount

1. Run lsblk to find the device name:

lsblk

The output should look like:

NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0          7:0    0    16M  1 loop
mmcblk1      179:0    0  59.5G  0 disk
├─mmcblk1p1  179:1    0    58G  0 part /
├─mmcblk1p2  179:2    0   128M  0 part
├─mmcblk1p3  179:3    0   768K  0 part
├─mmcblk1p4  179:4    0  31.6M  0 part
├─mmcblk1p5  179:5    0   128M  0 part
├─mmcblk1p6  179:6    0   768K  0 part
├─mmcblk1p7  179:7    0  31.6M  0 part
├─mmcblk1p8  179:8    0    80M  0 part
├─mmcblk1p9  179:9    0   512K  0 part
├─mmcblk1p10 179:10   0    64M  0 part
├─mmcblk1p11 179:11   0    80M  0 part
├─mmcblk1p12 179:12   0   512K  0 part
├─mmcblk1p13 179:13   0    64M  0 part
└─mmcblk1p14 179:14   0 879.5M  0 part
zram0        251:0    0   1.8G  0 disk [SWAP]
zram1        251:1    0   1.8G  0 disk [SWAP]
zram2        251:2    0   1.8G  0 disk [SWAP]
zram3        251:3    0   1.8G  0 disk [SWAP]
nvme0n1      259:0    0 238.5G  0 disk

Identify the device corresponding to your SSD. In this case, it is nvme0n1.

2. Format the SSD, create a mount point, and mount it:

sudo mkfs.ext4 /dev/nvme0n1

You can choose any name for the mount point directory. We use /ssd here, but in jetson-containers’ setup.md documentation, /mnt is used.

sudo mkdir /ssd
sudo mount /dev/nvme0n1 /ssd

3. Set up auto-mount to ensure the mount persists after boot:

First, identify the UUID for your SSD:

lsblk -f

Then, add a new entry to the fstab file:

sudo vi /etc/fstab

Insert the following line, replacing the UUID with the value found from lsblk -f:

UUID=************-****-****-****-******** /ssd/ ext4 defaults 0 2

4. Change ownership of the /ssd directory:

sudo chown ${USER}:${USER} /ssd

Docker

Install nvidia-container Package

sudo apt update
sudo apt install -y nvidia-container

sudo apt update
sudo apt install -y nvidia-container curl
curl https://get.docker.com | sh && sudo systemctl --now enable docker
sudo nvidia-ctk runtime configure --runtime=docker

Configure Docker

1. Restart Docker service and add your user to the docker group:

sudo systemctl restart docker
sudo usermod -aG docker $USER
newgrp docker

2. Add default runtime in /etc/docker/daemon.json:

sudo apt install -y jq
sudo jq '. + {"default-runtime": "nvidia"}' /etc/docker/daemon.json | \
  sudo tee /etc/docker/daemon.json.tmp && \
  sudo mv /etc/docker/daemon.json.tmp /etc/docker/daemon.json

sudo vi /etc/docker/daemon.json

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia"
}

3. Restart Docker:

sudo systemctl daemon-reload && sudo systemctl restart docker

Migrate Docker Directory to SSD

Now that the SSD is installed and available on your device, you can use the extra storage capacity to hold the storage-demanding Docker directory.

1. Stop the Docker service:

sudo systemctl stop docker

2. Move the existing Docker folder:

sudo du -csh /var/lib/docker/ && \
    sudo mkdir /ssd/docker && \
    sudo rsync -axPS /var/lib/docker/ /ssd/docker/ && \
    sudo du -csh /ssd/docker/

3. Edit /etc/docker/daemon.json:

sudo vi /etc/docker/daemon.json

Add the "data-root" line:

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia",
    "data-root": "/ssd/docker"
}

4. Rename the old Docker data directory:

sudo mv /var/lib/docker /var/lib/docker.old

5. Restart the docker daemon:

sudo systemctl daemon-reload && \
    sudo systemctl restart docker && \
    sudo journalctl -u docker

Test Docker on SSD

watch -n1 df

docker pull ubuntu:22.04

docker image ls

Final Verification

Reboot your Jetson and verify the following:

# Check SSD is recognized
sudo blkid | grep nvme

# Check disk space
df -h

# Check Docker root directory
docker info | grep Root

# List Docker directory on SSD
sudo ls -l /ssd/docker/

# Check Docker size
sudo du -chs /ssd/docker/

# Verify nvidia runtime
docker info | grep -e "Runtime" -e "Root"

Expected output should show:

• ✅ SSD mounted at /ssd
• ✅ Docker Root Dir: /ssd/docker
• ✅ Default Runtime: nvidia

✅ Your Jetson is Now Set Up!

Your Jetson is now configured with SSD storage and Docker optimized for AI workloads.

Next Steps

• Introduction to GenAI - Learn about running LLMs and VLMs on Jetson
• Ollama - Quick and easy local LLM deployment
• Supported Models - Browse models optimized for Jetson