# Jetson AI Lab

> Jetson AI Lab is NVIDIA's open-source resource for deploying generative AI on NVIDIA Jetson edge devices (Orin Nano, AGX Orin, Thor). It provides tutorials, model deployment guides, and benchmarks for running LLMs, VLMs, and VLA models locally on Jetson hardware.

Jetson AI Lab covers the full workflow: initial device setup, inference engine installation (vLLM, Ollama, llama.cpp, TensorRT), model deployment, fine-tuning, and benchmarking. All content targets on-device edge AI — no cloud required.

- Source code: https://github.com/NVIDIA-AI-IOT/jetson-ai-lab
- Full documentation: https://www.jetson-ai-lab.com/llms-full.txt

## Getting Started

- [Getting Started with Jetson](https://www.jetson-ai-lab.com/tutorials/getting-started-with-jetson/): Official developer kit user guides, remote SSH setup with VS Code or Cursor, and Jetson agentic skills for AI-assisted development
- [SSD + Docker Setup](https://www.jetson-ai-lab.com/tutorials/ssd-docker-setup/): Set up NVMe SSD storage and configure Docker on Jetson for optimal performance with AI containers
- [RAM Optimization](https://www.jetson-ai-lab.com/tutorials/ram-optimization/): Optimize system RAM by disabling desktop GUI, unnecessary services, and mounting swap for large model workloads

## Inference Engines & Fundamentals

- [Introduction to GenAI on Jetson](https://www.jetson-ai-lab.com/tutorials/genai-on-jetson-llms-vlms/): Practical intro to running LLMs and VLMs on Jetson using Ollama for experimentation and vLLM for production performance
- [Ollama on Jetson](https://www.jetson-ai-lab.com/tutorials/ollama/): Install and run Ollama for easy local LLM deployment, covering native installation, Docker containers, and Open WebUI
- [GenAI Benchmarking](https://www.jetson-ai-lab.com/tutorials/genai-benchmarking/): Benchmark LLMs and VLMs on Jetson using vLLM — measure throughput, latency, TTFT, and key performance metrics

## Model Optimization

- [Fine-tune LLMs on Jetson](https://www.jetson-ai-lab.com/tutorials/finetune-on-jetson/): Fine-tune large language models directly on Jetson using PyTorch and Hugging Face TRL — Full SFT (4B), LoRA (9B), and QLoRA (27B)
- [TensorRT Edge-LLM on Jetson](https://www.jetson-ai-lab.com/tutorials/tensorrt-edge-llm/): Use NVIDIA TensorRT Edge-LLM for quantization, ONNX export, engine builds, and pure C++ on-device inference

## Applications

- [Multi-Modal AI Studio on Jetson](https://www.jetson-ai-lab.com/tutorials/multi-modal-ai-studio/): Run a conversational AI pipeline on Jetson Thor with on-device ASR, LLM/VLM, and TTS
- [Live VLM WebUI](https://www.jetson-ai-lab.com/tutorials/live-vlm-webui/): Real-time Vision Language Model interface with WebRTC webcam streaming, OpenAI-compatible API, and interactive prompt editor
- [OpenClaw on Jetson](https://www.jetson-ai-lab.com/tutorials/openclaw/): Fully local AI personal assistant on Jetson with OpenClaw and WhatsApp, no cloud APIs needed
- [NanoOWL](https://www.jetson-ai-lab.com/tutorials/nanoowl/): OWL-ViT optimized for real-time open-vocabulary object detection on Jetson with TensorRT

## Vision Language Models (VLMs)

- [Gemma 4 on Jetson](https://www.jetson-ai-lab.com/tutorials/gemma4-on-jetson/): Run Google Gemma 4 models (E2B, E4B, 26B-A4B, 31B) on Jetson with vLLM or llama.cpp — reasoning, tool calling, and audio
- [Cosmos Reason2 on Jetson](https://www.jetson-ai-lab.com/tutorials/cosmos-reason2-vlm/): Run NVIDIA Cosmos Reason2 (2B/8B) with vLLM and Live VLM WebUI for real-time vision inference

## Vision-Language-Action (VLA) Models

- [OpenPi on Jetson Thor](https://www.jetson-ai-lab.com/tutorials/openpi_on_thor/): Deploy Physical Intelligence's OpenPi VLA model on Jetson Thor with TensorRT NVFP4 quantization for low-latency robotics inference

## Workshops

- [GTC 2026 Workshop](https://www.jetson-ai-lab.com/tutorials/gtc26/): 100-minute hands-on workshop — deploy AI microservices, run VLMs, and build conversational AI pipelines on Jetson Thor
- [GTC DC 2025 Workshop](https://www.jetson-ai-lab.com/tutorials/workshop-gtc-dc-2025/): Inference optimization on Jetson Thor with vLLM — production-grade LLM serving, quantization (FP16/FP8/FP4), speculative decoding
- [Hackathon Guide](https://www.jetson-ai-lab.com/tutorials/hackathon-guide/): Setup tips, project ideas, and resources for building AI projects on Jetson at hackathons

## Supported Models

The full model catalog with deployment commands is at https://www.jetson-ai-lab.com/models/

### NVIDIA Models
- [Nemotron3 Nano 4B](https://www.jetson-ai-lab.com/models/nemotron3-nano-4b/): NVIDIA's compact 4B Nano model with day-0 llama.cpp support on Jetson Orin and Thor
- [Nemotron3 Nano 30B-A3B](https://www.jetson-ai-lab.com/models/nemotron-3-nano-30b-a3b/): NVIDIA's flagship hybrid MoE reasoning model with 30B total / 3.5B active parameters
- [Nemotron Nano 9B v2](https://www.jetson-ai-lab.com/models/nemotron-nano-9b-v2/): NVIDIA's efficient 9B hybrid architecture model with Mamba-2 and attention layers
- [Nemotron 3 Super 120B-A12B](https://www.jetson-ai-lab.com/models/nemotron-3-super/): NVIDIA's large hybrid Mixture-of-Experts reasoning model — 120B total / 12B active — NVFP4 for Blackwell/Thor.
- [Nemotron Nano 12B VL](https://www.jetson-ai-lab.com/models/nemotron-nano-12b-vl/): NVIDIA's vision-language model for image understanding and multimodal reasoning
- [Nemotron 3 Nano Omni](https://www.jetson-ai-lab.com/models/nemotron-3-nano-omni/): NVIDIA's multimodal reasoning model with language, vision, audio, and video understanding — 30B total / 3B active MoE, available in NVFP4, FP8, and BF16.
- [Cosmos Reason 1 7B](https://www.jetson-ai-lab.com/models/cosmos-reason1-7b/): NVIDIA's 7B parameter reasoning vision-language model designed for physical AI and robotics applications
- [Cosmos Reason 2 2B](https://www.jetson-ai-lab.com/models/cosmos-reason2-2b/): NVIDIA's compact 2B parameter vision-language model with built-in chain-of-thought reasoning for edge deployment
- [Cosmos Reason 2 8B](https://www.jetson-ai-lab.com/models/cosmos-reason2-8b/): NVIDIA's 8B parameter vision-language model with advanced chain-of-thought reasoning capabilities

### Google Gemma Models
- [FunctionGemma](https://www.jetson-ai-lab.com/models/functiongemma/): Google's specialized function calling model built on Gemma 3 270M, optimized for tool use
- [Gemma 3 270M](https://www.jetson-ai-lab.com/models/gemma3-270m/): Google's ultra-compact 270 million parameter model for lightweight edge deployments
- [Gemma 3 1B](https://www.jetson-ai-lab.com/models/gemma3-1b/): Google's efficient 1 billion parameter model balancing capability and resource usage
- [Gemma 3 4B](https://www.jetson-ai-lab.com/models/gemma3-4b/): Google's versatile 4 billion parameter model - the default Gemma 3 variant
- [Gemma 3 12B](https://www.jetson-ai-lab.com/models/gemma3-12b/): Google's powerful 12 billion parameter model for advanced reasoning tasks
- [Gemma 3 27B](https://www.jetson-ai-lab.com/models/gemma3-27b/): Google's flagship 27 billion parameter model delivering state-of-the-art Gemma performance
- [Gemma 4 E2B](https://www.jetson-ai-lab.com/models/gemma4-e2b/): Google's compact frontier Gemma 4 model for efficient multimodal and agentic workloads
- [Gemma 4 12B](https://www.jetson-ai-lab.com/models/gemma4-12b/): Google's mid-size dense Gemma 4 model — strong general reasoning and multimodal understanding for Jetson Thor and AGX Orin
- [Gemma 4 E4B](https://www.jetson-ai-lab.com/models/gemma4-e4b/): Google's Gemma 4 E4B variant with Q4_K_M GGUF support on Jetson through llama.cpp
- [Gemma 4 26B-A4B](https://www.jetson-ai-lab.com/models/gemma4-26b-a4b/): Google's 26B MoE frontier Gemma 4 model for fast high-end reasoning and multimodal workflows
- [Gemma 4 31B](https://www.jetson-ai-lab.com/models/gemma4-31b/): Google's Gemma 4 31B variant with Q4_K_M GGUF support on Jetson through llama.cpp

### Meta Llama Models
- [Llama 3.2 3B](https://www.jetson-ai-lab.com/models/llama3-2-3b/): Meta's compact 3 billion parameter model, ideal for resource-constrained Jetson deployments
- [Llama 3.1 8B](https://www.jetson-ai-lab.com/models/llama3-1-8b/): Meta's efficient 8 billion parameter instruction-tuned language model optimized for Jetson
- [Llama 3.1 70B](https://www.jetson-ai-lab.com/models/llama3-1-70b/): Meta's flagship 70 billion parameter model delivering state-of-the-art performance on Jetson Thor

### Alibaba Qwen Models
- [Qwen3 4B](https://www.jetson-ai-lab.com/models/qwen3-4b/): Alibaba's efficient 4 billion parameter instruction-tuned language model
- [Qwen3 8B](https://www.jetson-ai-lab.com/models/qwen3-8b/): Alibaba's powerful 8 billion parameter instruction-tuned language model
- [Qwen3 30B-A3B (MoE)](https://www.jetson-ai-lab.com/models/qwen3-30b-a3b/): Alibaba's Mixture-of-Experts model with 30B total / 3B active parameters
- [Qwen3 32B](https://www.jetson-ai-lab.com/models/qwen3-32b/): Alibaba's flagship 32 billion parameter language model for advanced reasoning
- [Qwen3 VL 4B](https://www.jetson-ai-lab.com/models/qwen3-vl-4b/): Alibaba's 4 billion parameter vision-language model for multimodal understanding
- [Qwen3 VL 8B](https://www.jetson-ai-lab.com/models/qwen3-vl-8b/): Alibaba's 8 billion parameter vision-language model for advanced multimodal understanding
- [Qwen3.5 35B-A3B (MoE)](https://www.jetson-ai-lab.com/models/qwen3-5-35b-a3b/): Alibaba's latest Mixture-of-Experts model with 35B total / 3B active parameters, featuring native tool calling and MTP speculative decoding
- [Qwen3.5 27B](https://www.jetson-ai-lab.com/models/qwen3-5-27b/): Alibaba's dense 27 billion parameter language model with native tool calling and MTP speculative decoding
- [Qwen3.5 9B](https://www.jetson-ai-lab.com/models/qwen3-5-9b/): Alibaba's dense Qwen3.5 9B vision-language model with Jetson-specific checkpoints for Orin and Thor
- [Qwen3.5 4B](https://www.jetson-ai-lab.com/models/qwen3-5-4b/): Alibaba's efficient Qwen3.5 4B vision-language model tuned for practical multimodal deployment
- [Qwen3.5 0.8B](https://www.jetson-ai-lab.com/models/qwen3-5-0-8b/): Alibaba's compact Qwen3.5 vision-language model for lightweight multimodal deployment
- [Qwen3.6 35B-A3B (MoE)](https://www.jetson-ai-lab.com/models/qwen3-6-35b-a3b/): Alibaba's latest Mixture-of-Experts model with 35B total / 3B active parameters, featuring native tool calling and MTP speculative decoding
- [Qwen3.6 27B](https://www.jetson-ai-lab.com/models/qwen3-6-27b/): Alibaba's dense 27 billion parameter language model with native tool calling and MTP speculative decoding

### OpenAI Models
- [GPT OSS 20B](https://www.jetson-ai-lab.com/models/gpt-oss-20b/): OpenAI's open-source 20 billion parameter language model
- [GPT OSS 120B](https://www.jetson-ai-lab.com/models/gpt-oss-120b/): OpenAI's open-source 120 billion parameter language model for Jetson Thor

### Mistral Models
- [Ministral 3 3B Instruct](https://www.jetson-ai-lab.com/models/ministral3-3b-instruct/): Mistral AI's compact 3 billion parameter instruction-tuned model
- [Ministral 3 8B Instruct](https://www.jetson-ai-lab.com/models/ministral3-8b-instruct/): Mistral AI's versatile 8 billion parameter instruction-tuned model
- [Ministral 3 14B Instruct](https://www.jetson-ai-lab.com/models/ministral3-14b-instruct/): Mistral AI's powerful 14 billion parameter instruction-tuned model
- [Ministral 3 3B Reasoning](https://www.jetson-ai-lab.com/models/ministral3-3b-reasoning/): Mistral AI's compact 3 billion parameter model optimized for reasoning tasks
- [Ministral 3 8B Reasoning](https://www.jetson-ai-lab.com/models/ministral3-8b-reasoning/): Mistral AI's versatile 8 billion parameter model optimized for reasoning tasks
- [Ministral 3 14B Reasoning](https://www.jetson-ai-lab.com/models/ministral3-14b-reasoning/): Mistral AI's powerful 14 billion parameter model optimized for complex reasoning

### MiniMax Models
- [MiniMax M2.7](https://www.jetson-ai-lab.com/models/minimax-m2-7/): MiniMax's 230B agentic MoE flagship for software engineering and self-evolving agent harnesses with llama.cpp at 4-bit

### Other Models
- [Cosmos3 Edge](https://www.jetson-ai-lab.com/models/cosmos3-edge/): NVIDIA's edge-optimized omnimodal world model (4B) — multimodal reasoning, video generation, and robot action policies on Jetson.
- [Cosmos3 Nano](https://www.jetson-ai-lab.com/models/cosmos3-nano/): NVIDIA's compact vision-language reasoning model (16B) with chain-of-thought over text, image, and video — NVFP4 for Blackwell/Thor.

## Other Pages

- [Model Catalog](https://www.jetson-ai-lab.com/models/): Browse all supported models with one-click deployment commands for Jetson Orin and Thor
- [Community Projects](https://www.jetson-ai-lab.com/community/): Community-contributed projects, demos, and integrations built on Jetson
- [Research](https://www.jetson-ai-lab.com/research/): Academic research and papers related to Jetson edge AI