It runs on a GPU with 24GB of VRAM (e.g., RTX 4090).

It's using Qwen2-VL-7B. If there's any better LLMs, please let me know!

# to start chatbot
./start-chatbot.sh

# to stop docker containers
docker compose stop

If something is wrong, check the docker logs.

docker logs <container name>

Requirements

• NVIDIA Container Toolkit
• Docker