Gemma 4 26B-A4B
Google's 26B MoE frontier Gemma 4 model for fast high-end reasoning and multimodal workflows
Serve the model
Pick your Jetson, then copy the command and paste it into a terminal on the Jetson. New to Jetson? Setup guide
Model Details
If you want to learn more about the Gemma 4 family and the different ways to run it on Jetson, check out the Gemma 4 on Jetson tutorial .
Gemma 4 26B-A4B is a larger Gemma 4 variant that can be served on Jetson with llama.cpp. Google presents this model as the latency-optimized high-end option in the family: a Mixture-of-Experts model that targets much better throughput than a dense model of similar total size.
- Long-context agents with tool use
- Local coding copilots and repository Q&A on higher-memory Jetson systems
- Document and chart understanding workloads
- Research-style assistants that need stronger reasoning than the edge-sized models
Inputs and Outputs
Input: Text and image
Output: Text
Supported Platforms
- Jetson AGX Orin
- Jetson Thor
Inference Engine
This model is configured to run on Jetson with vLLM and llama.cpp.
Official Highlights
- Google’s model card describes 26B-A4B as a Mixture-of-Experts model with 25.2B total parameters and 3.8B active parameters during inference.
- It supports 256K context, text/image input, native function calling, and the same long-context reasoning features shared by the rest of Gemma 4.
- Google explicitly notes that the model runs much faster than its total parameter count suggests because only a subset of experts are active per token.
- In Google’s benchmark table, 26B-A4B tracks close to 31B dense on many reasoning and coding tasks while keeping a stronger latency profile.
Performance on Jetson
- 43.2 vLLM on AGX Thor: 43.2 tok/s 32 llama.cpp on AGX Thor: 32 tok/sAGX Thor128 GB
- 25.7 vLLM on AGX Orin: 25.7 tok/s 18.1 llama.cpp on AGX Orin: 18.1 tok/sAGX Orin64 GB
Output tokens/s
- 122 vLLM on AGX Thor: 122 tok/s 52.6 llama.cpp on AGX Thor: 52.6 tok/sAGX Thor128 GB
- 66.6 vLLM on AGX Orin: 66.6 tok/s 31 llama.cpp on AGX Orin: 31 tok/sAGX Orin64 GB
Output tokens/s
- 78.4 MTP vLLM with MTP on AGX Thor: 78.4 tok/sAGX Thor128 GB
- 52.1 MTP vLLM with MTP on AGX Orin: 52.1 tok/sAGX Orin64 GB
Output tokens/s
- 285 MTP vLLM with MTP on AGX Thor: 285 tok/sAGX Thor128 GB
- 179 MTP vLLM with MTP on AGX Orin: 179 tok/sAGX Orin64 GB
Output tokens/s