Skip to content

Tutorial - MiniGPT-4

Give your locally running LLM an access to vision, by running MiniGPT-4 on Jetson!

What you need

  1. One of the following Jetson devices:

    Jetson AGX Orin (64GB) Jetson AGX Orin (32GB) Jetson Orin NX (16GB)

  2. Running one of the following versions of JetPack :

    JetPack 5 (L4T r35.x) JetPack 6 (L4T r36.x)

  3. NVMe SSD highly recommended for storage speed and space

  4. Clone and setup jetson-containers : 

    git clone https://github.com/dusty-nv/jetson-containers
bash jetson-containers/install.sh

Start minigpt4 container with models

To start the MiniGPT4 container and webserver with the recommended models, run this command: 

jetson-containers run $(autotag minigpt4) /bin/bash -c 'cd /opt/minigpt4.cpp/minigpt4 && python3 webui.py \
  $(huggingface-downloader --type=dataset maknee/minigpt4-13b-ggml/minigpt4-13B-f16.bin) \
  $(huggingface-downloader --type=dataset maknee/ggml-vicuna-v0-quantized/ggml-vicuna-13B-v0-q5_k.bin)'

Then, open your web browser and access http://<IP_ADDRESS>:7860 .

Results