Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/virtual-models-mcp-cli.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'thinker': patch
---

Virtual models, part 5 (ADR 0004): virtual models in the MCP server and the `thinker` CLI. New MCP tools `thinker_routing_virtual_list`, `_create`, `_update` and `_delete`, and CLI commands `thinker routing virtual list|create|update|delete|enable|disable` (`--model`, repeatable `--fallback`, `--continue-on-default`; `delete` needs `--yes`). `thinker_routing_status` and `thinker routing status` list the virtual models, and `thinker_routing_test` / `thinker routing test --model <virtual>` report the virtual model and routing reason the request went through. Model params can be addressed by virtual model with the `virtual` field of `thinker_routing_params_get`/`_set` and `--virtual` on `thinker routing params get|set`, both exclusive with the tier selector, through the new `GET|PATCH /api/v1/routing/:agent/virtual-model-params?virtual=<name>` endpoint (scope `virtual:<id>`). The CLI agent skill documents them. `thinker_routing_status` keeps its `enabled`/`reason` verdict for `model: "auto"` (now labeled `status_for: "auto"`) and adds `routes_by_virtual_model` and `active_virtual_models`; when only virtual models can route, `reason` is `auto_not_routable` and `auto_reason` keeps the auto-route cause. `thinker routing status` adds the same two fields. Contract change: in the params view returned by `thinker_routing_params_get`/`_set` and the tier/virtual params endpoints, `tier` is now optional (present when the target is a tier) and `virtual` is set when the target is a virtual model. The params `model` selector (tiers and virtual models, MCP, CLI and REST) now also accepts a route's public id, as `/v1/models` lists it (`<alias>/<model>` for a custom provider), and its bare native id when only one connection of the chain has it, matched in layers (stored id, then public id, then native names) so an id that names one route exactly is never made ambiguous; the view adds `public_model` and `public_models`. The virtual model listing adds `routable`, computed on the chain the proxy builds (its own routes, then only the `default` slot with continue on Default), which both status readbacks use.
2 changes: 2 additions & 0 deletions docs/dev/guia.md
Original file line number Diff line number Diff line change
Expand Up @@ -193,6 +193,8 @@ Um tier pontuado sem modelo cai no tier `default`; se o `default` também não t
- A listagem calcula `shadows` (IDs reais que o nome sombreia) e `context_length` com um único snapshot do catálogo por requisição, sem `isRouteAvailable` rota a rota (`virtual-model-chain.ts`).
- A lista do agente fica em cache no `RoutingCacheService` (`getVirtualModels`) e cai junto com `invalidateAgent`, o que cobre as escritas no tier `default`.
- Apagar um virtual apaga as linhas de `agent_model_params` com `scope_key = 'virtual:<id>'` (`modelParamsScopeForVirtualModel`).
- **CLI e MCP** usam o mesmo `VirtualModelService`: `thinker routing virtual list|create|update|delete|enable|disable` e as ferramentas `thinker_routing_virtual_*` (`mcp/tools/virtual-models.tools.ts`). As duas acham o virtual pelo nome sem diferenciar maiúsculas, ou pelo `id`. `routing status`/`thinker_routing_status` listam os virtuais e acrescentam `routes_by_virtual_model` e `active_virtual_models`, a partir do campo `routable` da listagem (`virtualModelRoutable`: a mesma cadeia que o proxy monta, só com o slot `default` em `continue_on_default`, filtrada pelo snapshot do catálogo) (o `RoutingStatus` compartilhado com o dashboard continua falando só de `auto`; no MCP, `status_for: "auto"` e, quando só os virtuais roteiam, `reason: "auto_not_routable"` com a causa original em `auto_reason`), e `routing test`/`thinker_routing_test` aceitam o nome em `model` e devolvem `routing_reason` e `virtual_model` lidos dos cabeçalhos `X-Thinker-*`.
- **Parâmetros por nome:** `RouteModelParamsService` aceita como alvo um tier (string) ou `{ virtual: <nome> }`. São seletores separados, porque um tier custom e um virtual podem ter o mesmo nome. O REST é `GET|PATCH /api/v1/routing/:agentName/virtual-model-params?virtual=<nome>&model=<id>`, ao lado de `tiers/:tier/model-params`. O MCP usa o campo `virtual`, exclusivo com `tier`, e a CLI usa `--virtual`, exclusivo com `--tier`. Só a cadeia do próprio virtual é endereçável: as rotas emprestadas do Default usam os parâmetros do tier `default`. O endpoint por escopo (`PUT|DELETE /model-params` com `scope: "virtual:<id>"`) grava as mesmas linhas. O `model` casa em camadas, e a primeira com resultado vence: (1) o ID guardado, (2) o ID público da rota (`openAiModelId` do catálogo do agente, por exemplo `<alias>/<modelo>` de um custom provider), (3) o ID nativo, com ou sem o prefixo do provedor. Só há ambiguidade dentro da camada vencedora, e a mensagem sugere apenas nomes que selecionam uma única conexão; a resposta traz `public_model` e `public_models`.

**Fallback**: cada tier tem um modelo principal e até 5 de reserva (`routing/dto/routing.dto.ts`). Qualquer status HTTP >= 400 (`fallback-status-codes.ts`), timeout de tentativa (`PROVIDER_TIMEOUT_MS`), falta de primeiro trecho (`STREAM_WARMUP_MS`) ou silêncio em streaming (`STREAM_IDLE_TIMEOUT_MS`) leva à próxima rota (`proxy-fallback.service.ts`). Se todas falham, o cliente recebe o erro do provedor da última tentativa (não um código `M###`).

Expand Down
3 changes: 3 additions & 0 deletions packages/backend/src/mcp/mcp.controller.ts
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,7 @@ import { resolveMcpOperator } from './mcp-auth';
import { buildMcpServer } from './mcp-server.factory';
import { McpToolDeps } from './tool-deps';
import { RouteModelParamsService } from '../routing/model-params/route-model-params.service';
import { VirtualModelService } from '../routing/virtual-models/virtual-model.service';

/**
* The remote MCP endpoint.
Expand Down Expand Up @@ -69,6 +70,7 @@ export class McpController {
private readonly specificity: SpecificityService,
private readonly resolveAgentService: ResolveAgentService,
private readonly headerTiers: HeaderTierService,
private readonly virtualModels: VirtualModelService,
private readonly customProviders: CustomProviderService,
private readonly routeModelParams: RouteModelParamsService,
private readonly modelDiscovery: ModelDiscoveryService,
Expand Down Expand Up @@ -98,6 +100,7 @@ export class McpController {
specificity: this.specificity,
resolveAgent: this.resolveAgentService,
headerTiers: this.headerTiers,
virtualModels: this.virtualModels,
customProviders: this.customProviders,
routeModelParams: this.routeModelParams,
modelDiscovery: this.modelDiscovery,
Expand Down
2 changes: 2 additions & 0 deletions packages/backend/src/mcp/mcp.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ import { AnalyticsModule } from '../analytics/analytics.module';
import { CommonModule } from '../common/common.module';
import { CustomProviderModule } from '../routing/custom-provider/custom-provider.module';
import { HeaderTiersModule } from '../routing/header-tiers/header-tiers.module';
import { VirtualModelsModule } from '../routing/virtual-models/virtual-models.module';
import { RoutingCoreModule } from '../routing/routing-core/routing-core.module';
import { ModelDiscoveryModule } from '../model-discovery/model-discovery.module';
import { ModelPricesModule } from '../model-prices/model-prices.module';
Expand All @@ -26,6 +27,7 @@ import { McpController } from './mcp.controller';
AnalyticsModule,
RoutingCoreModule,
HeaderTiersModule,
VirtualModelsModule,
CustomProviderModule,
ModelDiscoveryModule,
ModelPricesModule,
Expand Down
2 changes: 2 additions & 0 deletions packages/backend/src/mcp/tool-deps.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import { ResolveAgentService } from '../routing/routing-core/resolve-agent.servi
import { HeaderTierService } from '../routing/header-tiers/header-tier.service';
import { CustomProviderService } from '../routing/custom-provider/custom-provider.service';
import { RouteModelParamsService } from '../routing/model-params/route-model-params.service';
import { VirtualModelService } from '../routing/virtual-models/virtual-model.service';
import { ModelDiscoveryService } from '../model-discovery/model-discovery.service';
import { PricingSyncService } from '../database/pricing-sync.service';
import { ModelPricesService } from '../model-prices/model-prices.service';
Expand All @@ -41,6 +42,7 @@ export interface McpToolDeps {
specificity: SpecificityService;
resolveAgent: ResolveAgentService;
headerTiers: HeaderTierService;
virtualModels: VirtualModelService;
customProviders: CustomProviderService;
routeModelParams: RouteModelParamsService;
modelDiscovery: ModelDiscoveryService;
Expand Down
5 changes: 3 additions & 2 deletions packages/backend/src/mcp/tools/environment.tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,8 +129,9 @@ export function registerGuideTool(server: McpServer): void {
'Harnesses: thinker_agent_list, then thinker_agent_get. Create with thinker_agent_create; the returned key is shown once.',
'Providers: thinker_provider_catalog lists what is connectable; thinker_provider_connect stores a tenant-wide connection and discovers its models.',
'Routing: thinker_routing_status tells you whether a harness can route. Configure the default chain with the newer agent configure surface, or per-agent fallbacks with thinker_routing_fallbacks_set.',
'Model params (reasoning effort, temperature, …): thinker_routing_params_get lists what a tier route accepts and what is saved; thinker_routing_params_set saves values that override what the harness sends on routed requests (model "auto" or a matching custom-tier header). Address the route by tier ("default" or a custom tier name) and model.',
'Verify end-to-end with thinker_routing_test — it sends one real request through the selected platform surface.',
'Virtual models: public names a harness sends as model (e.g. "deepseek/flash"), each with its own chain and optional continue_on_default. thinker_routing_virtual_list, then _create/_update/_delete; thinker_routing_status lists them too.',
'Model params (reasoning effort, temperature, …): thinker_routing_params_get lists what a route accepts and what is saved; thinker_routing_params_set saves values that override what the harness sends on routed requests (model "auto", a virtual model name, or a matching custom-tier header). Address the route by tier ("default" or a custom tier name) or by virtual (a virtual model name), and model.',
'Verify end-to-end with thinker_routing_test — it sends one real request through the selected platform surface; pass a virtual model name as model to test that chain.',
'A connection that is active with zero cached models is unusable; run thinker_provider_refresh.',
],
}),
Expand Down
Loading
Loading