Multimodal
Qwen3.5 4B
Alibaba's efficient Qwen3.5 4B vision-language model tuned for practical multimodal deployment
Parameters 4B
Inputs
Text Image Video
Outputs
Text
Context Length 256K
License Apache 2.0
Precision
NVFP4 W4A16
Serve the model
Pick your Jetson, then copy the command and paste it into a terminal on the Jetson. New to Jetson? Setup guide
Run on your Jetson
Engine
Speculative decoding
Weights
Model Details
View on HuggingFace
Qwen3.5 4B offers a balanced point in the Qwen3.5 family for local multimodal instruction following, visual understanding, and agent-style workloads on Jetson.
Inputs and Outputs
Input: Text and images
Output: Text
Intended Use Cases
- Visual question answering: Multimodal prompting with image inputs
- Image understanding: Captioning, scene analysis, and grounded responses
- Tool calling: Structured tool use with vLLM
- Multilingual tasks: Translation and multilingual prompting
Additional Resources
- Original Model - Base Qwen3.5 4B checkpoint
- W4A16 Checkpoint - Jetson Orin checkpoint
- NVFP4 Checkpoint - Jetson Thor checkpoint
Performance on Jetson
2K input · 128 output tokensWriting and summarization · up to 256 output tokens
Concurrent requests
vLLMllama.cpp
- 56.1 vLLM on AGX Thor: 56.1 tok/sAGX Thor128 GB
- 27.9 vLLM on AGX Orin: 27.9 tok/sAGX Orin64 GB
- 10.2 llama.cpp on Orin Nano: 10.2 tok/sOrin Nano8 GB
Output tokens/s
vLLM
- 239 vLLM on AGX Thor: 239 tok/sAGX Thor128 GB
- 92.1 vLLM on AGX Orin: 92.1 tok/sAGX Orin64 GB
Output tokens/s
vLLMllama.cpp
- 76 MTP vLLM with MTP on AGX Thor: 76 tok/s 110 DFlash2 vLLM with DFlash2 on AGX Thor: 110 tok/sAGX Thor128 GB
- 47.5 MTP vLLM with MTP on AGX Orin: 47.5 tok/s 66.5 DFlash2 vLLM with DFlash2 on AGX Orin: 66.5 tok/sAGX Orin64 GB
- 21.6 MTP llama.cpp with MTP on Orin Nano: 21.6 tok/sOrin Nano8 GB
Output tokens/s
vLLM
- 414 MTP vLLM with MTP on AGX Thor: 414 tok/s 375 DFlash2 vLLM with DFlash2 on AGX Thor: 375 tok/sAGX Thor128 GB
- 220 MTP vLLM with MTP on AGX Orin: 220 tok/s 188 DFlash2 vLLM with DFlash2 on AGX Orin: 188 tok/sAGX Orin64 GB
Output tokens/s