Gemma 4 31B
Google's flagship Gemma 4 model with NVFP4, W4A16, and GGUF deployment paths on Jetson
Serve the model
Pick your Jetson, then copy the command and paste it into a terminal on the Jetson. New to Jetson? Setup guide
Model Details
If you want to learn more about the Gemma 4 family and the different ways to run it on Jetson, check out the Gemma 4 on Jetson tutorial .
Gemma 4 31B is the largest model in the current Gemma 4 set here, and it can be served on Jetson with llama.cpp. In Google’s launch post, 31B is the flagship dense model in the family, aimed at the best possible raw quality for local reasoning, coding, and agentic workflows.
- Highest-quality local reasoning and coding on Jetson Thor or well-provisioned AGX Orin setups
- Long-context assistants over large documents or repositories
- Multimodal analysis of screenshots, charts, forms, and PDFs
- Advanced agent systems where answer quality matters more than minimum latency
Inputs and Outputs
Input: Text and image
Output: Text
Supported Platforms
- Jetson AGX Orin
- Jetson Thor
Inference Engine
This model is configured to run on Jetson with vLLM and llama.cpp.
Official Highlights
- Google’s model card describes 31B as a dense multimodal model with 30.7B parameters, 256K context, and text/image input.
- The Gemma 4 launch post positions 31B as the top-quality model in the family and states that it ranked #3 among open models on the Arena AI text leaderboard at launch.
- In Google’s published benchmark table, 31B is the strongest Gemma 4 variant across the major reasoning, coding, and multimodal rows shown in the card.
- Google also calls out 31B as a strong foundation for fine-tuning when quality matters more than latency.
Performance on Jetson
- 8.8 vLLM on AGX Thor: 8.8 tok/s 7.1 llama.cpp on AGX Thor: 7.1 tok/sAGX Thor128 GB
- 5.7 vLLM on AGX Orin: 5.7 tok/s 4.2 llama.cpp on AGX Orin: 4.2 tok/sAGX Orin64 GB
Output tokens/s
- 23.5 vLLM on AGX Thor: 23.5 tok/s 13.6 llama.cpp on AGX Thor: 13.6 tok/sAGX Thor128 GB
- 14.8 vLLM on AGX Orin: 14.8 tok/s 8.5 llama.cpp on AGX Orin: 8.5 tok/sAGX Orin64 GB
Output tokens/s
- 21.2 MTP vLLM with MTP on AGX Thor: 21.2 tok/sAGX Thor128 GB
- 13.8 MTP vLLM with MTP on AGX Orin: 13.8 tok/sAGX Orin64 GB
Output tokens/s
- 88.9 MTP vLLM with MTP on AGX Thor: 88.9 tok/sAGX Thor128 GB
- 55.1 MTP vLLM with MTP on AGX Orin: 55.1 tok/sAGX Orin64 GB
Output tokens/s