Tutorial - MiniGPT-4

Give your locally running LLM an access to vision, by running MiniGPT-4 on Jetson!

What you need

1. One of the following Jetson devices:

   Jetson AGX Orin (64GB) Jetson AGX Orin (32GB) Jetson Orin NX (16GB) Jetson Orin Nano (8GB)

2. Running one of the following versions of JetPack:

   JetPack 5 (L4T r35.x) JetPack 6 (L4T r36.x)

3. Sufficient storage space (preferably with NVMe SSD).

   • 5.8GB for container image
   • Space for pre-quantized MiniGPT-4 model

4. Clone and setup jetson-containers:

   git clone https://github.com/dusty-nv/jetson-containers
   bash jetson-containers/install.sh
   

Start minigpt4 container with models

To start the MiniGPT4 container and webserver with the recommended models, run this command:

jetson-containers run $(autotag minigpt4) /bin/bash -c 'cd /opt/minigpt4.cpp/minigpt4 && python3 webui.py \
  $(huggingface-downloader --type=dataset maknee/minigpt4-13b-ggml/minigpt4-13B-f16.bin) \
  $(huggingface-downloader --type=dataset maknee/ggml-vicuna-v0-quantized/ggml-vicuna-13B-v0-q5_k.bin)'

Then, open your web browser and access http://<IP_ADDRESS>:7860.

Results