New Text

Nemotron 3.5 Bolt E4B 4GB

NVIDIA's compact conversational model for local assistants, multi-turn tool use, and interactive characters. Run text chat on Jetson with llama.cpp.

Command to Run on Jetson Model Details

Parameters 8B total / 4B effective

Modalities

Text

Context Length 64K

License OpenMDW-1.1

Precision

Q4_K_M GGUF

Serve the model

Start server

Choose module, then engine and optional parameters on the left, then copy the serve command by clicking the button on the right.

Command

Call the model over Web API

Copy a client command below and paste it into your terminal to make a Web API request to the model you just served.

llama.cpp server (OpenAI-compatible API)

After llama-server is running with --network host, call it from another machine on the LAN (set ${JETSON_HOST} or use the field). Default port is often 8080 unless you set --port.

curl -s http://${JETSON_HOST}:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

from openai import OpenAI

client = OpenAI(
    base_url="http://${JETSON_HOST}:8000/v1",
    api_key="not-needed",  # vLLM / llama.cpp typically do not enforce a key
)

completion = client.chat.completions.create(
    model="nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(completion.choices[0].message.content)

Model Details

View on HuggingFace

Get started with Nemotron 3.5 Bolt on Jetson

Nemotron 3.5 Bolt E4B 4GB is NVIDIA’s compact open language model built on Gemma-4-E4B-it. It is designed for local assistants, interactive characters, and robots that hold conversations and use tools to carry out tasks. It takes text as input and generates text as output.

NVIDIA’s additional training focuses on:

Using tools across multiple turns to complete a task.
Following instructions throughout a conversation.
Grounding replies in the user’s input, conversation history, and tool results.

The model supports reasoning and non-reasoning modes through enable_thinking, plus an optional conversational_mode for interactive dialogue. The model card includes API examples for these options.

The Q4_K_M GGUF release targets an approximately 4 GB model memory footprint. Total memory use also depends on context length and runtime overhead. This guide serves text chat with llama.cpp.

Run on Jetson

Select your Orin or Thor module above and copy the command. Docker pulls the latest container for your platform, and llama.cpp downloads Nemotron-3.5-Bolt-E4B-Q4_K_M.gguf from nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB. The download is cached in ~/.cache/huggingface/hub for subsequent runs.

Once the server is ready, open http://<JETSON_IP>:8000 for the chat UI, or use the API examples above.

Read the NVIDIA research overview for the model’s design and results, or the official model card for deployment examples.