Benchmarks

Large Language Models (LLM)

For running LLM benchmarks, see the MLC container documentation.

Small Language Models (SLM)

Small language models are generally defined as having fewer than 7B parameters (Llama-7B shown for reference)
For more data and info about running these models, see the SLM tutorial and MLC container documentation.

Vision Language Models (VLM)

This measures the end-to-end pipeline performance for continuous streaming like with Live Llava.
For more data and info about running these models, see the NanoVLM tutorial.

Vision Transformers (ViT)

VIT performance data from [1] [2] [3]

Stable Diffusion

Riva

For running Riva benchmarks, see ASR Performance and TTS Performance.

Vector Database

For running vector database benchmarks, see the NanoDB container documentation.