Text
GPT OSS 20B
OpenAI's open-source 20 billion parameter language model
Parameters 21B total / 3.6B activated
Inputs
Text
Outputs
Text
Context Length 128K
License Apache 2.0
Precision
W4A16
Serve the model
Pick your Jetson, then copy the command and paste it into a terminal on the Jetson. New to Jetson? Setup guide
Run on your Jetson
Engine
Speculative decoding
Weights
Model Details
OpenAI GPT OSS 20B is OpenAI’s open-source 20 billion parameter language model. This model requires tiktoken encodings to be downloaded before serving.
Running with vLLM
Step 1: Download Tiktoken Encodings
mkdir -p $HOME/.cache/tiktoken
wget -q https://openaipublic.blob.core.windows.net/encodings/cl100k_base.tiktoken \
-O $HOME/.cache/tiktoken/cl100k_base.tiktoken
wget -q https://openaipublic.blob.core.windows.net/encodings/o200k_base.tiktoken \
-O $HOME/.cache/tiktoken/o200k_base.tiktoken
Step 2: Serve
sudo docker run -it --rm --pull always --runtime=nvidia --network host \
-v $HOME/.cache/huggingface:/root/.cache/huggingface \
-v $HOME/.cache/tiktoken:/etc/encodings \
-e TIKTOKEN_ENCODINGS_BASE=/etc/encodings \
ghcr.io/nvidia-ai-iot/vllm:latest-jetson-orin \
vllm serve openai/gpt-oss-20b --gpu-memory-utilization 0.8
GPT OSS Family
| Model | Parameters | Memory | Minimum Jetson |
|---|---|---|---|
| GPT OSS 20B | 20B | 16GB RAM | AGX Orin |
| GPT OSS 120B | 120B | 64GB RAM | Thor |