Part of LAN BAT, open home infrastructure you can read, fork and borrow from.
This repository contains configuration and test suites for LLMs hosted on RunPod serverless endpoints, organized by purpose.
Each LLM purpose has its own directory under models/:
models/coding-agent- Qwen3.8-27B (official FP8)models/home-assistant- Qwen3-8B (AWQ)
-
Required Tools
curlpython3sqlite3npm(for opencode)
-
RunPod API key — stored in
~/.config/envman/(seedocs/opencode-setup.md)
-
Clone Repository
git clone <repository-url> cd runpod-llm-fleet
-
Install OpenCode + fleet config
npm install -g @opencode/cli echo 'export RUNPOD_API_KEY="rpa_..."' > ~/.config/envman/RUNPOD.env ./scripts/setup-opencode.sh ./scripts/validate-opencode.sh
Full documentation:
docs/opencode-setup.md
See docs/runpod-setup.md for standing up a new RunPod
serverless endpoint from scratch (template + endpoint creation via the REST API) —
needed when adding a new purpose to the fleet or rebuilding an existing endpoint.
Navigate to a specific model directory and run the test suite:
cd models/coding-agent
./run_tests.sh| Purpose | Directory | Current Model | Consumer | Scaling |
|---|---|---|---|---|
| Coding assistant backend | models/coding-agent |
Qwen3.8-27B (FP8) | opencode | Scale-to-zero |
| Home Assistant conversation agent | models/home-assistant |
Qwen3-8B (AWQ) | Home Assistant | Scale-to-zero |
export RUNPOD_API_KEY=<key>
./scripts/deploy-fleet.sh all # deploy both + install opencode config + test
./scripts/deploy-fleet.sh coding # coding endpoint only
./scripts/deploy-fleet.sh verify # smoke-test without redeploying- The RunPod endpoints are specific to this setup and may require different API keys
- All model test suites cost small amounts of RunPod GPU time
- OpenCode reads the API key from
~/.config/envman/RUNPOD.key(not shell env) — seedocs/opencode-setup.md - Shell scripts (
deploy-fleet.sh,run_tests.sh) useRUNPOD_API_KEYfrom the environment or~/.config/envman/RUNPOD.env - For debugging agent behavior, check
~/.local/share/opencode/opencode.db