Multimodal

Gemma 4 31B

Google's flagship Gemma 4 model with NVFP4, W4A16, and GGUF deployment paths on Jetson

Parameters 31B
Inputs
Text Image Video
Outputs
Text
Context Length 256K
License Apache 2.0
Precision
W4A16 QAT Q4_0 GGUF

Serve the model

Pick your Jetson, then copy the command and paste it into a terminal on the Jetson. New to Jetson? Setup guide

Run on your Jetson
Engine
Speculative decoding
Weights
First, one-time setup

Model Details

If you want to learn more about the Gemma 4 family and the different ways to run it on Jetson, check out the Gemma 4 on Jetson tutorial .

Gemma 4 31B is the largest model in the current Gemma 4 set here, and it can be served on Jetson with llama.cpp. In Google’s launch post, 31B is the flagship dense model in the family, aimed at the best possible raw quality for local reasoning, coding, and agentic workflows.

  • Highest-quality local reasoning and coding on Jetson Thor or well-provisioned AGX Orin setups
  • Long-context assistants over large documents or repositories
  • Multimodal analysis of screenshots, charts, forms, and PDFs
  • Advanced agent systems where answer quality matters more than minimum latency

Inputs and Outputs

Input: Text and image

Output: Text

Supported Platforms

  • Jetson AGX Orin
  • Jetson Thor

Inference Engine

This model is configured to run on Jetson with vLLM and llama.cpp.

Official Highlights

  • Google’s model card describes 31B as a dense multimodal model with 30.7B parameters, 256K context, and text/image input.
  • The Gemma 4 launch post positions 31B as the top-quality model in the family and states that it ranked #3 among open models on the Arena AI text leaderboard at launch.
  • In Google’s published benchmark table, 31B is the strongest Gemma 4 variant across the major reasoning, coding, and multimodal rows shown in the card.
  • Google also calls out 31B as a strong foundation for fine-tuning when quality matters more than latency.

Performance on Jetson

2K input · 128 output tokens
Concurrent requests
vLLMllama.cpp
  • 8.8 vLLM on AGX Thor: 8.8 tok/s 7.1 llama.cpp on AGX Thor: 7.1 tok/s
    AGX Thor128 GB
  • 5.7 vLLM on AGX Orin: 5.7 tok/s 4.2 llama.cpp on AGX Orin: 4.2 tok/s
    AGX Orin64 GB

Output tokens/s