Reachy Mini Jetson Assistant

The Reachy Mini Jetson Assistant is a fully local voice and vision robot assistant for Reachy Mini Lite powered by NVIDIA Jetson. It listens through the robot microphone, captures camera frames, reasons with a vision-language model, speaks with text-to-speech, tracks the person in front of it, and drives expressive head, body, and antenna movements.

The interesting part is not only that the entire AI pipeline runs locally. It is that the complete stack can run on a Jetson Orin Nano 8GB with proper memory optimization:

[Mic] -> [Silero VAD] -> [faster-whisper STT] --+
[USB Camera] -> [Frame Ring Buffer] ------------+-> [VLM stream] -> [TTS stream] -> [Speaker + Robot]
                                                +-> [Web UI via WebSocket]

Jetson Device Skills are applicable across Jetson devices. In this tutorial, we walk through how those skills were used to deploy a conversational AI application on Jetson Orin Nano 8GB and validate the memory decisions that make the pipeline practical on the smallest Orin Nano target.

What You Will Build

You will run a local Reachy Mini assistant with:

• Vision-language reasoning with Cosmos-Reason2-2B GGUF Q4_K_M served by llama.cpp
• Speech-to-text with faster-whisper and CUDA-enabled CTranslate2
• Text-to-speech with Kokoro ONNX and CUDA ONNX Runtime
• Voice activity detection with Silero VAD
• Face detection and tracking with OpenCV YuNet
• Reachy Mini motion control with face tracking and TTS-synchronized gestures
• Browser UI with live camera, conversation state, and telemetry

Why Memory Optimization Matters

The NVIDIA Technical Blog post Maximizing Memory Efficiency to Run Bigger Models on NVIDIA Jetson presents the Reachy Mini assistant as a stack-wide transition from an Orin NX 16GB-class configuration to a complete Orin Nano 8GB deployment.

The Orin Nano has a real deployment budget of about 7.6GB usable unified memory, and this application runs several memory-hungry components at the same time. The point is not to optimize one layer. The system fits because every layer is chosen for the budget.

How Jetson Device Skills Helped

Jetson Device Skills provided an agent-guided workflow for bringing up the application on a live Jetson. We used them to inspect the Orin Nano 8GB system, identify memory pressure, choose a low-memory VLM runtime, and verify the result after tuning. The skills produced the evidence and recommendations; the Reachy application used those findings to choose the runtime, model format, headless deployment, and memory-conscious application settings.

If you are customizing the Jetson image before flashing, also look at Jetson BSP Skills. Those skills focus on BSP and image-build workflows, while the Jetson Device Skills used here operate after the device has booted and help inspect, tune, and validate a live Jetson deployment.

The table below keeps the skills story simple: what the skills showed, what we chose, and what changed.

Prepare Reachy Mini USB Access

Connect Reachy Mini Lite to the Jetson over USB, then add udev rules so the SDK can access the serial ports without root:

echo 'SUBSYSTEM=="tty", ATTRS{idVendor}=="2e8a", ATTRS{idProduct}=="000a", MODE="0666", SYMLINK+="reachy_mini"' \
  | sudo tee /etc/udev/rules.d/99-reachy-mini.rules
sudo udevadm control --reload-rules && sudo udevadm trigger

Add your user to the dialout group and reboot:

sudo usermod -aG dialout $USER
sudo reboot

After reboot, verify the device is visible:

ls -la /dev/ttyACM*

Add NVMe Swap

sudo fallocate -l 8G /mnt/nvme/swapfile
sudo chmod 600 /mnt/nvme/swapfile
sudo mkswap /mnt/nvme/swapfile
sudo swapon /mnt/nvme/swapfile

echo '/mnt/nvme/swapfile none swap sw 0 0' | sudo tee -a /etc/fstab

Install the Assistant

Install the system dependencies used by audio, Python virtual environments, and CUDA-enabled inference libraries:

sudo apt-get update
sudo apt-get install -y \
  python3.10-venv \
  portaudio19-dev \
  libasound2-dev \
  pulseaudio-utils \
  libcudnn9-dev-cuda-12

Clone the repository:

git clone https://github.com/NVIDIA-AI-IOT/reachy-mini-jetson-assistant
cd reachy-mini-jetson-assistant

Create and activate a Python 3.10 virtual environment:

python3.10 -m venv venv
source venv/bin/activate
pip install --upgrade pip wheel
pip install -r requirements.txt

Install the Jetson GPU build of ONNX Runtime:

pip install onnxruntime-gpu --extra-index-url https://pypi.jetson-ai-lab.io/jp6/cu126

Install the Reachy Mini SDK:

pip install reachy-mini

Pin NumPy for Jetson ONNX Runtime compatibility:

pip install "numpy==1.26.4"

Build CTranslate2 with CUDA

The pip ctranslate2 package is CPU-only on Jetson. Build CTranslate2 from source for GPU-accelerated faster-whisper STT:

pip install pybind11

cd ~
git clone --depth 1 https://github.com/OpenNMT/CTranslate2.git
cd CTranslate2
git submodule update --init --recursive

mkdir build && cd build
export PATH=/usr/local/cuda/bin:$PATH
export CUDA_HOME=/usr/local/cuda
cmake .. -DWITH_CUDA=ON -DWITH_CUDNN=ON -DCMAKE_BUILD_TYPE=Release \
         -DCUDA_ARCH_LIST="8.7" -DOPENMP_RUNTIME=NONE -DWITH_MKL=OFF

make -j$(nproc)
cmake --install . --prefix ~/.local

export LD_LIBRARY_PATH=~/.local/lib:$LD_LIBRARY_PATH
cd ../python
pip install .

Persist the library path in your virtual environment activation script:

echo 'export LD_LIBRARY_PATH=$HOME/.local/lib:$LD_LIBRARY_PATH' >> ~/reachy-mini-jetson-assistant/venv/bin/activate

Verify the Runtime

Activate the environment and check the key GPU providers:

source venv/bin/activate
python3 -c "
import ctranslate2; print('CTranslate2 CUDA devices:', ctranslate2.get_cuda_device_count())
import onnxruntime; print('ONNX providers:', onnxruntime.get_available_providers())
from reachy_mini import ReachyMini; print('Reachy Mini SDK: OK')
import faster_whisper; print('faster-whisper: OK')
import kokoro_onnx; print('kokoro-onnx: OK')
"

Expected output includes:

CTranslate2 CUDA devices: 1
ONNX providers: ['CUDAExecutionProvider', 'CPUExecutionProvider']
Reachy Mini SDK: OK
faster-whisper: OK
kokoro-onnx: OK

Start the VLM Server

In the first terminal, start Cosmos-Reason2-2B with llama.cpp:

NP=1 ./run_llama_cpp.sh Kbenkhaled/Cosmos-Reason2-2B-GGUF:Q4_K_M

Wait until the server reports that it is listening on http://0.0.0.0:8080.

Run Web Vision Chat

In a second terminal, start the full assistant:

source venv/bin/activate
python3 run_web_vision_chat.py

Open the web UI from a browser on the same network:

http://<jetson-ip>:8090

You should see the live camera feed, conversation log, push-to-talk control, active settings, and system telemetry.

Test Robot Movement

source venv/bin/activate
python3 scripts/test_reachy_movement.py

Troubleshooting

pip uninstall onnxruntime
pip install onnxruntime-gpu --extra-index-url https://pypi.jetson-ai-lab.io/jp6/cu126

export LD_LIBRARY_PATH=$HOME/.local/lib:$LD_LIBRARY_PATH

docker ps
docker logs assistant-llm

lsof -ti :8090 | xargs kill -9

ls /dev/video*
fuser -k /dev/video0

Next Steps

After the assistant is running, experiment with:

• Different VLM models that fit your Jetson memory budget
• Additional Reachy Mini gestures and movement profiles under config/settings.yaml