Multimodal

Gemma 4 26B-A4B

Google's 26B MoE frontier Gemma 4 model for fast high-end reasoning and multimodal workflows

Parameters 3.8B active (25.8B total, MoE)
Inputs
Text Image Video
Outputs
Text
Context Length 256K
License Apache 2.0
Precision
NVFP4 AWQ-4bit Q4_0 GGUF

Serve the model

Pick your Jetson, then copy the command and paste it into a terminal on the Jetson. New to Jetson? Setup guide

Run on your Jetson
Engine
Speculative decoding
Weights
First, one-time setup

Model Details

If you want to learn more about the Gemma 4 family and the different ways to run it on Jetson, check out the Gemma 4 on Jetson tutorial .

Gemma 4 26B-A4B is a larger Gemma 4 variant that can be served on Jetson with llama.cpp. Google presents this model as the latency-optimized high-end option in the family: a Mixture-of-Experts model that targets much better throughput than a dense model of similar total size.

  • Long-context agents with tool use
  • Local coding copilots and repository Q&A on higher-memory Jetson systems
  • Document and chart understanding workloads
  • Research-style assistants that need stronger reasoning than the edge-sized models

Inputs and Outputs

Input: Text and image

Output: Text

Supported Platforms

  • Jetson AGX Orin
  • Jetson Thor

Inference Engine

This model is configured to run on Jetson with vLLM and llama.cpp.

Official Highlights

  • Google’s model card describes 26B-A4B as a Mixture-of-Experts model with 25.2B total parameters and 3.8B active parameters during inference.
  • It supports 256K context, text/image input, native function calling, and the same long-context reasoning features shared by the rest of Gemma 4.
  • Google explicitly notes that the model runs much faster than its total parameter count suggests because only a subset of experts are active per token.
  • In Google’s benchmark table, 26B-A4B tracks close to 31B dense on many reasoning and coding tasks while keeping a stronger latency profile.

Performance on Jetson

2K input · 128 output tokens
Concurrent requests
vLLMllama.cpp
  • 43.2 vLLM on AGX Thor: 43.2 tok/s 32 llama.cpp on AGX Thor: 32 tok/s
    AGX Thor128 GB
  • 25.7 vLLM on AGX Orin: 25.7 tok/s 18.1 llama.cpp on AGX Orin: 18.1 tok/s
    AGX Orin64 GB

Output tokens/s