Add LM Studio backend (Ollama -> LM Studio) with dynamic model picker - #7
Open
frostybittn wants to merge 8 commits into
Open
Add LM Studio backend (Ollama -> LM Studio) with dynamic model picker#7frostybittn wants to merge 8 commits into
frostybittn wants to merge 8 commits into
Conversation
Adds LM Studio as a switchable local LLM backend alongside Ollama, using LM Studio's native /api/v0 REST API. The avatar's brain, the command-center model picker, and the raw-LLM lab all run on LM Studio; the dashboard auto-discovers the entire LM Studio model library. Selected via LLM_BACKEND (lmstudio is the new default; ollama reverts to the original path). TTS stays Kokoro -- LM Studio has no audio API. Orchestrator: - providers/lmstudio.py: native /api/v0/chat/completions client (streaming + tool_calls verified), a thin subclass of OllamaProvider (identical wire format) - system/lmstudio.py: model discovery + residency via /api/v0/models - catalog kind=lmstudio + sync_dynamic_brains(); dispatcher model_override - config LLM_BACKEND/LMSTUDIO_URL/LMSTUDIO_DEFAULT_MODEL; backend-aware routes Frontend: - dashboard loaded-state chip; /api/lmstudio/ proxy (nginx + vite) - Lab 1 + shared helpers converted to LM Studio native API; lab labels updated Docs/config: - docker-compose default LLM_BACKEND=lmstudio; docs/lmstudio-runbook.md - CLAUDE.md, README, wiki swap-model FAQ Tests: - tests/test_lmstudio.py (16 new); fixed 3 pre-existing stale qwen3-4b tests - full suite 171 passing Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Pull request overview
Note
Copilot was unable to run its full agentic suite in this review.
Adds LM Studio as a first-class local LLM backend (native /api/v0), including dynamic model discovery/residency and UI + workshop updates so models can be swapped live via the dashboard.
Changes:
- Introduces LM Studio backend integration in orchestrator (provider, discovery/residency, dynamic catalog brains, “auto” model resolution).
- Updates orchestrator routes and tests to support
LLM_BACKENDswitching and new default brain/model identifiers. - Updates frontend dashboard + lab pages and adds reverse-proxy passthrough for direct LM Studio calls; adds runbook/docs.
Reviewed changes
Copilot reviewed 28 out of 28 changed files in this pull request and generated 6 comments.
Show a summary per file
| File | Description |
|---|---|
| wiki/faqs/swap-model.md | Updates FAQ to describe dashboard-based live model switching (LM Studio). |
| services/orchestrator/tests/test_routes_state.py | Adjusts state route test expectation for updated default brain id. |
| services/orchestrator/tests/test_routes_catalog.py | Updates catalog tests for updated qwen brain id and pull hint strings. |
| services/orchestrator/tests/test_lmstudio.py | Adds full test suite for LM Studio backend/provider/catalog/route behavior. |
| services/orchestrator/orchestrator/system/lmstudio.py | Adds LM Studio discovery + residency adapter against /api/v0/models. |
| services/orchestrator/orchestrator/routes/models.py | Makes /v1/models proxy follow the active local backend (Ollama vs LM Studio). |
| services/orchestrator/orchestrator/routes/chat.py | Adds fallback to default brain if persisted brain missing; resolves LM Studio “auto” model at request time. |
| services/orchestrator/orchestrator/routes/catalog.py | Merges LM Studio library as dynamic brains and annotates availability/loaded state. |
| services/orchestrator/orchestrator/providers/ollama.py | Makes chat endpoint path configurable for subclasses via _CHAT_PATH. |
| services/orchestrator/orchestrator/providers/lmstudio.py | Adds LMStudioProvider subclass targeting native /api/v0/chat/completions. |
| services/orchestrator/orchestrator/providers/dispatcher.py | Dispatches lmstudio brains and supports request-time model_override. |
| services/orchestrator/orchestrator/main.py | Wires LM Studio backend, promotes LM Studio default brain, and resets stale persisted brain. |
| services/orchestrator/orchestrator/config.py | Adds llm_backend, lmstudio_url, and lmstudio_default_model settings. |
| services/orchestrator/orchestrator/catalog.py | Adds lmstudio brain kind + dynamic brain syncing for LM Studio discovery. |
| frontend/vite.config.js | Adds dev proxy passthrough route for /api/lmstudio to reach host LM Studio. |
| frontend/src/dashboard/components/ControlsPanel.js | Adds LM Studio “loaded” chip logic using residency snapshot loaded list. |
| frontend/public/lab/index.html | Updates Lab 1 description to LM Studio-based playground. |
| frontend/public/lab/_shared.js | Adds LM Studio preflight + model listing via native /api/v0/models. |
| frontend/public/lab/06-pipeline.html | Updates pipeline UI label to “LLM (LM Studio)”. |
| frontend/public/lab/04-orchestrator.html | Updates orchestrator lab copy to refer to LM Studio backend. |
| frontend/public/lab/01-llm.html | Switches Lab 1 to native LM Studio endpoints + model picker + stats logging. |
| frontend/nginx.conf | Adds nginx passthrough /api/lmstudio/ proxy to host LM Studio for labs. |
| docs/lmstudio-runbook.md | Adds detailed runbook for enabling/using LM Studio backend. |
| docker-compose.yml | Sets LM Studio as default orchestrator backend via env vars. |
| configs/catalog.yml | Adds static LM Studio brains (auto + pinned model) and documents dynamic discovery. |
| README.md | Updates prerequisites to support Ollama or LM Studio (LM Studio default). |
| CLAUDE.md | Updates architecture notes and documents Plan #11 LM Studio backend behavior. |
| .env.example | Adds example env vars for selecting/configuring LM Studio backend. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| settings.lmstudio_url | ||
| if settings.llm_backend == "lmstudio" | ||
| else settings.ollama_url | ||
| ) |
Comment on lines
+80
to
+89
| loaded = [ | ||
| { | ||
| "model": (m.get("id") or m.get("key") or ""), | ||
| "size_bytes": 0, | ||
| "size_vram_bytes": 0, | ||
| "residency": "loaded", | ||
| } | ||
| for m in data | ||
| if m.get("state") == "loaded" | ||
| ] |
Comment on lines
+137
to
+147
| if settings.llm_backend == "lmstudio": | ||
| try: | ||
| current = catalog.brain(app.state.state_store.get_state()["brain"]) | ||
| needs_reset = current.kind != "lmstudio" | ||
| except Exception: | ||
| needs_reset = True | ||
| if needs_reset: | ||
| try: | ||
| app.state.state_store.set_state("brain", "lmstudio-auto") | ||
| except Exception as e: # pragma: no cover - defensive | ||
| log.warning("could not set LM Studio default brain: %s", e) |
Comment on lines
+9
to
+19
| @router.get("/v1/models") | ||
| async def list_models(request: Request) -> JSONResponse: | ||
| backend = request.app.state.settings.ollama_url | ||
| settings = request.app.state.settings | ||
| # Proxy whichever local backend is active. Both Ollama and LM Studio expose | ||
| # an OpenAI-compatible /v1/models ({data:[{id}, ...]}), which is the shape | ||
| # the frontend preflight + Lab pages expect. | ||
| backend = ( | ||
| settings.lmstudio_url | ||
| if settings.llm_backend == "lmstudio" | ||
| else settings.ollama_url | ||
| ) |
|
|
||
| ## Easiest: pick from the dashboard (LM Studio) | ||
|
|
||
| When `LLM_BACKEND=lmstudio` (the default), the orchestrator auto-discovers **every model in your LM Studio library** and lists them in the dashboard's **Brain** dropdown (loaded models first, with a green "loaded" chip). Just pick one — selecting a model and chatting JIT-loads it in LM Studio. The **LM Studio (auto)** brain always follows whatever model you currently have loaded. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Trim trailing slash from backend URL.
Clarified the default setting for LLM_BACKEND in the dashboard instructions.
…t brain Boot-time migration now uses the _promote_brain_default() return value and resets the active brain to "lmstudio-auto" only when it exists in the catalog; otherwise falls back to the validated catalog default. Never writes a brain id the catalog doesn't know (e.g. if a user edits configs/catalog.yml). Adds a regression test for the missing-auto-brain case. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…eava-workshop-public into lmstudio-backend
Owner
|
Thanks for this — the implementation looks solid (clean subclass of OllamaProvider, dedicated tests, runbook included). One blocker before we can merge: please flip Keeping LM Studio as an opt-in via |
Addresses upstream review feedback (Lucasmind): keep all the additive LM Studio support (native /api/v0 provider, dynamic discovery, dashboard picker, lab toggle) but restore Ollama as the docker-compose default so the upstream workshop's pedagogy is preserved. Deploy defaults flipped: - docker-compose.yml: LLM_BACKEND default lmstudio -> ollama (override in .env) - .env.example: LLM_BACKEND=ollama with reordered comments Prose reframed (Ollama default, LM Studio opt-in): - CLAUDE.md: architecture diagram reordered, Plan #11 line - README.md: LLM-backend bullets reordered - docs/lmstudio-runbook.md: opening + section 1 (Activate) + section 4 (Switch back) + the per-deploy overrides table - wiki/faqs/swap-model.md: dashboard-picker section - configs/catalog.yml: LM Studio brain block moved AFTER Ollama brains; header reads "(Plan #11, opt-in via LLM_BACKEND=lmstudio)"; lmstudio-qwen3-4b label no longer says "fast default" - services/orchestrator/orchestrator/config.py: corrected the stale comment that still claimed docker-compose sets LLM_BACKEND=lmstudio Lab pages re-labeled: - frontend/public/lab/index.html: Lab 1 card (Ollama first, LM Studio via toggle) - frontend/public/lab/04-orchestrator.html: "Ollama by default; switchable to LM Studio via LLM_BACKEND" - frontend/public/lab/06-pipeline.html: stage label "LLM (Ollama / LM Studio)" Lab 1 (raw LLM playground) gains a backend toggle: - New Backend <select> at the top of the controls grid; Ollama selected by default - BACKENDS map parameterizes the 4 things that actually differ (endpoint, model-list source, preflight key, default model); everything else (SSE parser, sampling knobs, chip timing, native-stats logging) stays backend-agnostic because both backends speak the same OpenAI-shaped wire - refreshModels() updates the step-tag textContent + model dropdown reactively when the backend changes - syncBackend() is async + awaits refreshModels() so a rapid toggle + Run click can't race ahead of the model-list fetch (caught by adversarial review) - $(go).onclick captures the backend at click time so a mid-request toggle doesn't move the fetch goalposts Drive-by fix discovered while running pytest on this branch: - services/orchestrator/orchestrator/system/lmstudio.py: corrected over-indented block in LMStudioBackend.query() (lines 80-94) that caused an IndentationError on import, blocking pytest collection of test_lmstudio.py and test_routes_personality.py. Introduced by bbeec6d "Refactor model loading logic for clarity"; logic was correct, only the indentation was off by one space. Full suite 172/172 passes after the fix. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Author
|
I flipped it back to |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds LM Studio as a switchable local LLM backend alongside Ollama, using LM Studio's native /api/v0 REST API. The avatar's brain, the command-center model picker, and the raw-LLM lab all run on LM Studio; the dashboard auto-discovers the entire LM Studio model library. Selected via LLM_BACKEND (lmstudio is the new default; ollama reverts to the original path). TTS stays Kokoro -- LM Studio has no audio API.
Orchestrator:
Frontend:
Docs/config:
Tests: