New Text

Nemotron 3.5 Lightning

NVIDIA's fast open-weight model for responsive local agents, reasoning, coding, and tool use. It delivers performance comparable to the much larger Nemotron 3 Super.

Command to Run on Jetson Model Details
Parameters 30B total / 3B active
Modalities
Text
Context Length 1M
License OpenMDW 1.1
Precision
NVFP4

Serve the model

Start server

Choose module, then engine and optional parameters on the left, then copy the serve command by clicking the button on the right.

Command

·

Call the model over Web API

Copy a client command below and paste it into your terminal to make a Web API request to the model you just served. 

curl -s http://${JETSON_HOST}:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nemotron35",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Model Details

HuggingFace View on HuggingFace

NVIDIA Nemotron 3.5 Lightning is a fast, open-weight 30B Mixture-of-Experts model that activates only 3B parameters per token. It brings performance close to the much larger Nemotron 3 Super while remaining practical for local coding assistants, research agents, tool-calling workflows, and other always-on applications.

The model supports context lengths of up to one million tokens. Reasoning can be enabled or disabled, and a configurable reasoning budget lets users control how much reasoning the model performs before answering.

The best performance we saw with this model averaged 115 tokens/sec on Jetson AGX Thor and 89 tokens/sec on Jetson AGX Orin across multi-step agentic workloads involving reasoning and tool calls.

Nemotron 3.5 Lightning supports several speculative decoding options, including built-in Multi-Token Prediction, DSpark, and DFlash. We tested the available approaches and found DSpark with vLLM delivered the best performance on both Jetson AGX Thor and Jetson AGX Orin.

Inputs and Outputs

Input: Text

Output: Text

Supported Platforms

  • Jetson AGX Orin 64GB
  • Jetson AGX Thor T4000
  • Jetson AGX Thor T5000 Developer Kit

Why Nemotron 3.5 Lightning on Jetson

  • Super-class capability: Performance close to Nemotron 3 Super in a model that is much faster and more practical to run locally.
  • Responsive local agents: Strong performance for multi-step reasoning, tool calls, coding assistants, and research workflows.
  • Controllable reasoning: Enable or disable reasoning and set a reasoning budget to balance quality, latency, and token usage.
  • Long-context support: Work with context lengths of up to one million tokens when memory allows.
  • Fast speculative decoding: Choose from MTP, DSpark, and DFlash, with DSpark providing the best results in our Jetson testing.

Speculative Decoding on Jetson

Nemotron 3.5 Lightning includes Multi-Token Prediction and is released with dedicated DSpark and DFlash checkpoints. The commands on this page use DSpark with five speculative tokens, which was the fastest configuration in our testing on both supported Jetson platforms.

The server exposes an OpenAI-compatible API on port 8000 with reasoning parsing, automatic tool selection, and the Qwen3 Coder tool-call parser enabled.

Additional Resources