New Multimodal

Muse Glimmer 30B

Meta's local agentic model with reasoning, tool use, image understanding, and DFlash speculative decoding

Command to Run on Jetson Model Details
Parameters 30B
Modalities
Text Image
Context Length 131K
License Apache 2.0
Precision
K-Quant (~4-bit) GGUF

Serve the model

Start server

Choose module, then engine and optional parameters on the left, then copy the serve command by clicking the button on the right.

Command

·

Call the model over Web API

Copy a client command below and paste it into your terminal to make a Web API request to the model you just served.

llama.cpp server (OpenAI-compatible API)

After llama-server is running with --network host, call it from another machine on the LAN (set ${JETSON_HOST} or use the field). Default port is often 8080 unless you set --port.

curl -s http://${JETSON_HOST}:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "my_model",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Model Details

HuggingFace View on HuggingFace

Muse Glimmer 30B is Meta’s compact agentic model for running long-horizon AI workflows locally. It combines multi-step reasoning, reliable tool calling, failure recovery, and multilingual support with image understanding through a dedicated perception encoder.

The official 17GB K-quant build is designed for a 24GB memory envelope. It can run on Jetson AGX Orin and Jetson Thor with llama.cpp, leaving room for the vision projector, KV cache, and the included DFlash speculative-decoding model.

Inputs and Outputs

Input: Text and image

Output: Text

Supported Platforms

  • Jetson AGX Orin 64GB
  • Jetson AGX Thor T4000
  • Jetson AGX Thor T5000 Developer Kit

Why Muse Glimmer on Jetson

  • Local agents: Plan, invoke tools, recover from failures, and complete multi-step tasks without relying on a cloud model.
  • Agentic coding: Work through repository-scale coding and debugging tasks with controllable reasoning effort.
  • Multimodal understanding: Analyze screenshots, charts, forms, and documents with the automatically downloaded mmproj-kquant.gguf perception encoder.
  • Faster generation: --spec-type draft-dflash automatically downloads and enables dflash-kquant.gguf for speculative decoding without changing output quality.
  • Jetson performance: Reach up to 36 tokens/s on Jetson Thor and 25 tokens/s on Jetson AGX Orin with DFlash speculative decoding enabled.

Inference Engine

This model uses the latest Jetson Orin or Jetson Thor llama.cpp container. The server downloads these official Meta artifacts from Hugging Face on first launch:

  • muse-glimmer-30B-kquant-17gb.gguf — 17GB language-model weights
  • mmproj-kquant.gguf — perception encoder for image input
  • dflash-kquant.gguf — DFlash speculative-decoding model

The command uses Meta’s recommended sampling defaults: temperature 1.0, top-p 0.95, and top-k 64.

Reasoning Strength

Muse Glimmer supports low, medium, high, and xhigh reasoning strengths. Set the desired level in the system prompt, for example:

Reasoning strength: high.

Use high or xhigh for complex agentic and coding tasks. Lower levels trade some depth for faster responses.

Additional Resources