Tips - SSD + Docker

Once you have your Jetson set up by flashing the latest Jetson Linux (L4T) BSP on it or by flashing the SD card with the whole JetPack image, before embarking on testing out all the great generative AI application using jetson-containers, you want to make sure you have a huge storage space for all the containers and the models you will download.

We are going to show how you can install SSD on your Jetson, and set it up for Docker.

SSD

Physical installation

Unplug power and any peripherals from the Jetson developer kit.
Physically install an NVMe SSD card on the carrier board of your Jetson developer kit, making sure to properly seat the connector and secure with the screw.
Reconnect any peripherals, and then reconnect the power supply to turn on the Jetson developer kit.
Once the system is up, verify that your Jetson identifies a new memory controller on PCI bus:
```
lspci
```
The output should look like the following:
```
0007:01:00.0 Non-Volatile memory controller: Marvell Technology Group Ltd. Device 1322 (rev 02)
```

Format and set up auto-mount

Run lsblk to find the device name.

lsblk

The output should look like the following:

NAME         MAJ:MIN RM   SIZE RO TYPE MOUNTPOINT
loop0          7:0    0    16M  1 loop 
mmcblk1      179:0    0  59.5G  0 disk 
├─mmcblk1p1  179:1    0    58G  0 part /
├─mmcblk1p2  179:2    0   128M  0 part 
├─mmcblk1p3  179:3    0   768K  0 part 
├─mmcblk1p4  179:4    0  31.6M  0 part 
├─mmcblk1p5  179:5    0   128M  0 part 
├─mmcblk1p6  179:6    0   768K  0 part 
├─mmcblk1p7  179:7    0  31.6M  0 part 
├─mmcblk1p8  179:8    0    80M  0 part 
├─mmcblk1p9  179:9    0   512K  0 part 
├─mmcblk1p10 179:10   0    64M  0 part 
├─mmcblk1p11 179:11   0    80M  0 part 
├─mmcblk1p12 179:12   0   512K  0 part 
├─mmcblk1p13 179:13   0    64M  0 part 
└─mmcblk1p14 179:14   0 879.5M  0 part 
zram0        251:0    0   1.8G  0 disk [SWAP]
zram1        251:1    0   1.8G  0 disk [SWAP]
zram2        251:2    0   1.8G  0 disk [SWAP]
zram3        251:3    0   1.8G  0 disk [SWAP]
nvme0n1      259:0    0 238.5G  0 disk

Identify the device corresponding to your SSD. In this case, it is nvme0n1.

Format the SSD, create a mount point, and mount it to the filesystem.
```
sudo mkfs.ext4 /dev/nvme0n1
```
You can choose any name for the mount point directory. We use /ssd here, but in jetson-containers' setup.md documentation, /mnt is used.
```
sudo mkdir /ssd
```
```
sudo mount /dev/nvme0n1 /ssd
```
In order to ensure that the mount persists after boot, add an entry to the fstab file:

First, identify the UUID for your SSD:
```
lsblk -f
```
Then, add a new entry to the fstab file:
```
sudo vi /etc/fstab
```
Insert the following line, replacing the UUID with the value found from lsblk -f:
```
UUID=************-****-****-****-******** /ssd/ ext4 defaults 0 2
```
Finally, change the ownership of the /ssd directory.
```
sudo chown ${USER}:${USER} /ssd
```

Docker

Install nvidia-container package.

Note: If you used an NVIDIA-supplied SD card image to flash your SD card, all necessary JetPack components (including nvidia-containers) and Docker are already pre-installed, so this step can be skipped.
```
sudo apt update
sudo apt install -y nvidia-container
```
JetPack 6.0 DP users

If you flash Jetson Linux (L4T) R36.2 (JetPack 6.0 DP) on your Jetson using SDK Manager, and install nvidia-container using apt, on JetPack 6.0 it no longer automatically installs Docker.

Therefore, you need to run the following to manually install Docker and set it up.
```
sudo apt update
sudo apt install -y nvidia-container curl
curl https://get.docker.com | sh && sudo systemctl --now enable docker
sudo nvidia-ctk runtime configure --runtime=docker
```
Restart the Docker service and add your user to the docker group, so that you don't need to use the command with sudo.
```
sudo systemctl restart docker
sudo usermod -aG docker $USER
newgrp docker
```

Add default runtime in /etc/docker/daemon.json

sudo vi /etc/docker/daemon.json

Insert the "default-runtime": "nvidia" line as following:

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia"
}

Restart Docker

sudo systemctl daemon-reload && sudo systemctl restart docker

Migrate Docker directory to SSD

Now that the SSD is installed and available on your device, you can use the extra storage capacity to hold the storage-demanding Docker directory.

Stop the Docker service.
```
sudo systemctl stop docker
```

Move the existing Docker folder

sudo du -csh /var/lib/docker/ && \
    sudo mkdir /ssd/docker && \
    sudo rsync -axPS /var/lib/docker/ /ssd/docker/ && \
    sudo du -csh  /ssd/docker/

Edit /etc/docker/daemon.json

sudo vi /etc/docker/daemon.json

Insert "data-root" line like the following.

{
    "runtimes": {
        "nvidia": {
            "path": "nvidia-container-runtime",
            "runtimeArgs": []
        }
    },
    "default-runtime": "nvidia",
    "data-root": "/ssd/docker"
}

Rename the old Docker data directory

sudo mv /var/lib/docker /var/lib/docker.old

Restart the docker daemon

sudo systemctl daemon-reload && \
    sudo systemctl restart docker && \
    sudo journalctl -u docker

Test Docker on SSD

[Terminal 1] First, open a terminal to monitor the disk usage while pulling a Docker image.
```
watch -n1 df 
```
[Terminal 2] Next, open a new terminal and start Docker pull.
```
docker pull nvcr.io/nvidia/l4t-base:r35.2.1
```

[Terminal 1] Observe that the disk usage on /ssd goes up as the container image is downloaded and extracted.

~$ docker image ls
REPOSITORY                  TAG       IMAGE ID       CREATED        SIZE
nvcr.io/nvidia/l4t-base     r35.2.1   dc07eb476a1d   7 months ago   713MB

Final Verification

Reboot your Jetson, and verify that you observe the following:

~$ sudo blkid | grep nvme
/dev/nvme0n1: UUID="9fc06de1-7cf3-43e2-928a-53a9c03fc5d8" TYPE="ext4"

~$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/mmcblk1p1  116G   18G   94G  16% /
none            3.5G     0  3.5G   0% /dev
tmpfs           3.6G  108K  3.6G   1% /dev/shm
tmpfs           734M   35M  699M   5% /run
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
tmpfs           3.6G     0  3.6G   0% /sys/fs/cgroup
tmpfs           734M   88K  734M   1% /run/user/1000
/dev/nvme0n1    458G  824M  434G   1% /ssd

~$ docker info | grep Root
 Docker Root Dir: /ssd/docker

~$ sudo ls -l /ssd/docker/
total 44
drwx--x--x  4 root root 4096 Mar 22 11:44 buildkit
drwx--x---  2 root root 4096 Mar 22 11:44 containers
drwx------  3 root root 4096 Mar 22 11:44 image
drwxr-x---  3 root root 4096 Mar 22 11:44 network
drwx--x--- 13 root root 4096 Mar 22 16:20 overlay2
drwx------  4 root root 4096 Mar 22 11:44 plugins
drwx------  2 root root 4096 Mar 22 16:19 runtimes
drwx------  2 root root 4096 Mar 22 11:44 swarm
drwx------  2 root root 4096 Mar 22 16:20 tmp
drwx------  2 root root 4096 Mar 22 11:44 trust
drwx-----x  2 root root 4096 Mar 22 16:19 volumes

~$ sudo du -chs /ssd/docker/
752M    /ssd/docker/
752M    total

~$ docker info | grep -e "Runtime" -e "Root"
 Runtimes: io.containerd.runtime.v1.linux nvidia runc io.containerd.runc.v2
 Default Runtime: nvidia
 Docker Root Dir: /ssd/docker

Your Jetson is now set up with the SSD!