diff --git a/README.md b/README.md index 2ff5a69..b294c83 100644 --- a/README.md +++ b/README.md @@ -78,7 +78,7 @@ This is the stack that lets agents operate autonomously: **x402 + USDC + local r ## Quick Start -> **No wallet? 7 models work free out of the box.** Install, run, and pin `free/gpt-oss-120b` (or any of the 7) — no crypto, no signup, no balance required. Add USDC later when you want paid models. +> **No wallet? 7 models work free out of the box.** Install, run, and pin `free/nemotron-3.5-lightning` (or any of the 7) — no crypto, no signup, no balance required. Add USDC later when you want paid models. ### Option A — OpenClaw Agent @@ -127,7 +127,7 @@ npx @blockrun/xclawrouter ``` **2. Fund your wallet** — optional, skip for free tier -Your wallet address is printed on first run. For paid models, send a few USDC on Base or Solana — $5 covers thousands of requests. To stay at $0, pin any of the 7 free models (e.g. `free/gpt-oss-120b`) or use `/model free` inside OpenClaw. +Your wallet address is printed on first run. For paid models, send a few USDC on Base or Solana — $5 covers thousands of requests. To stay at $0, pin any of the 7 free models (e.g. `free/nemotron-3.5-lightning`) or use `/model free` inside OpenClaw. **3. Point your client at `http://localhost:8402`** @@ -216,12 +216,12 @@ Choose your routing strategy with `/model `: Request → Weighted Scorer (15 dimensions) → Tier → Best Model → Response ``` -| Tier | ECO Model | AUTO Model | PREMIUM Model | -| --------- | ----------------------------------- | ------------------------------------- | ---------------------------- | -| SIMPLE | free/gpt-oss-120b (**FREE**) | gemini-2.5-flash ($0.30/$2.50) | kimi-k2.6 | -| MEDIUM | gemini-3.1-flash-lite ($0.25/$1.50) | kimi-k2.5 ($0.60/$3.00) | gpt-5.3-codex ($1.75/$14.00) | -| COMPLEX | gemini-3.1-flash-lite ($0.25/$1.50) | gemini-3.1-pro ($2/$12) | claude-opus-4.6 ($5/$25) | -| REASONING | grok-4-1-fast ($0.20/$0.50) | grok-4-1-fast-reasoning ($0.20/$0.50) | claude-sonnet-4.6 ($3/$15) | +| Tier | ECO Model | AUTO Model | PREMIUM Model | +| --------- | -------------------------------------- | ------------------------------------- | ---------------------------- | +| SIMPLE | free/nemotron-3.5-lightning (**FREE**) | gemini-2.5-flash ($0.30/$2.50) | kimi-k2.6 | +| MEDIUM | gemini-3.1-flash-lite ($0.25/$1.50) | kimi-k2.5 ($0.60/$3.00) | gpt-5.3-codex ($1.75/$14.00) | +| COMPLEX | gemini-3.1-flash-lite ($0.25/$1.50) | gemini-3.1-pro ($2/$12) | claude-opus-4.6 ($5/$25) | +| REASONING | grok-4-1-fast ($0.20/$0.50) | grok-4-1-fast-reasoning ($0.20/$0.50) | claude-sonnet-4.6 ($3/$15) | **Blended average: $2.05/M** vs $25/M for Claude Opus = **84% savings** @@ -305,13 +305,13 @@ Edit existing images with `/img2img`: | Model | Input $/M | Output $/M | ~$/request | Context | Features | | ------------------------------------------- | --------: | ---------: | ---------: | ------- | -------------------------------------------- | -| free/gpt-oss-120b | **FREE** | **FREE** | **$0** | 128K | default free model | -| free/gpt-oss-20b | **FREE** | **FREE** | **$0** | 128K | smaller, faster | -| free/deepseek-v4-flash | **FREE** | **FREE** | **$0** | 1M | reasoning, ~5x faster than v4-pro | -| free/qwen3-coder-480b | **FREE** | **FREE** | **$0** | 131K | coding | -| free/llama-4-maverick | **FREE** | **FREE** | **$0** | 131K | reasoning | +| free/nemotron-3.5-lightning | **FREE** | **FREE** | **$0** | 1M | reasoning — default free model | +| free/nemotron-3-nano-30b | **FREE** | **FREE** | **$0** | 131K | reasoning, fastest free model (~121 tok/s) | +| free/laguna-xs-2.1 | **FREE** | **FREE** | **$0** | 131K | coding | +| free/north-mini-code | **FREE** | **FREE** | **$0** | 256K | coding | | free/nemotron-3-nano-omni-30b-a3b-reasoning | **FREE** | **FREE** | **$0** | 256K | reasoning, **vision** (text+img+video+audio) | -| free/glm-4.7 | **FREE** | **FREE** | **$0** | 131K | reasoning | +| free/nemotron-3-ultra-550b | **FREE** | **FREE** | **$0** | 1M | reasoning | +| free/llama-3.2-11b-vision | **FREE** | **FREE** | **$0** | 128K | **vision** | | openai/gpt-5-nano | $0.05 | $0.40 | $0.0002 | 128K | tools | | openai/gpt-4.1-nano | $0.10 | $0.40 | $0.0003 | 128K | tools | | google/gemini-2.5-flash-lite | $0.10 | $0.40 | $0.0003 | 1M | tools | @@ -382,7 +382,7 @@ Edit existing images with `/img2img`: | openai/gpt-5.2-pro | $21.00 | $168.00 | $0.0945 | 400K | reasoning, tools | | openai/gpt-5.4-pro | $30.00 | $180.00 | $0.1050 | 400K | reasoning, tools | -> **Free tier:** 7 models cost nothing — `/model free` points to gpt-oss-120b, or pick any free model directly (e.g., `/model nemotron-omni` for vision, `/model qwen-coder` for coding, `/model deepseek-v4-flash` for reasoning + 1M context, `/model maverick` for general chat). +> **Free tier:** 7 models cost nothing — `/model free` points to nemotron-3.5-lightning, or pick any free model directly (e.g., `/model nemotron-omni` for vision, `/model north-mini-code` for coding, `/model nemotron-3-ultra-550b` for reasoning + 1M context). > **Best value:** `gpt-5-nano` and `gemini-2.5-flash-lite` deliver strong results at ~$0.0003/request. --- @@ -458,7 +458,7 @@ For basic usage, no configuration needed. For advanced options: Block specific models from being routed to. Useful if a model doesn't follow your agent instructions or you want to control costs. ```bash -/exclude add free/gpt-oss-120b # Block the default free model +/exclude add free/nemotron-3.5-lightning # Block the default free model /exclude add grok-4 # Aliases work — blocks all grok-4 variants /exclude add gpt-5.4 # Skip expensive models /exclude # Show current exclusions @@ -622,7 +622,7 @@ ClawRouter works with any tool that makes OpenAI-compatible API calls — point ### Is ClawRouter free? -ClawRouter itself is free and MIT licensed. You pay only for the LLM API calls routed through it — and several NVIDIA-hosted models (`gpt-oss-120b`, `gpt-oss-20b`, `deepseek-v4-flash`, `qwen3-coder-480b`, `llama-4-maverick`, `nemotron-3-nano-omni-30b-a3b-reasoning`) are completely free. Use `/model free` to smart-route across them, or pick any by name. +ClawRouter itself is free and MIT licensed. You pay only for the LLM API calls routed through it — and several models (`nemotron-3.5-lightning`, `nemotron-3-nano-30b`, `laguna-xs-2.1`, `north-mini-code`, `nemotron-3-nano-omni-30b-a3b-reasoning`, `nemotron-3-ultra-550b`, `llama-3.2-11b-vision`) are completely free. Use `/model free` to smart-route across them, or pick any by name. --- diff --git a/skills/xclawrouter/SKILL.md b/skills/xclawrouter/SKILL.md index f9e1291..9f2745a 100644 --- a/skills/xclawrouter/SKILL.md +++ b/skills/xclawrouter/SKILL.md @@ -19,7 +19,7 @@ metadata: { "openclaw": { "emoji": "🦀", "requires": { "config": ["models.prov # XClawRouter -Smart LLM router that saves 84% on inference costs by routing each request to the cheapest model that can handle it. 76 models across 9 providers (11 free NVIDIA models), all through one wallet. +Smart LLM router that saves 84% on inference costs by routing each request to the cheapest model that can handle it. 76 models across 9 providers (7 free models), all through one wallet. ## Install @@ -55,7 +55,7 @@ Rules handle ~~80% of requests in <1ms. Only ambiguous queries hit the LLM class ## Available Models -76 models including: gpt-5.5, gpt-5.4, gpt-4o, o3, claude-opus-4.8, claude-opus-4.7, claude-sonnet-4.6, gemini-3.1-pro, gemini-3.5-flash, deepseek-v4-pro, deepseek-chat, grok-4.3, grok-build-0.1, kimi-k2.6, kimi-k2.5, and free NVIDIA-hosted models (gpt-oss-120b [default], gpt-oss-20b, deepseek-v4-flash, qwen3-coder-480b, llama-4-maverick, nemotron-3-nano-omni-30b-a3b-reasoning [vision]). +76 models including: gpt-5.5, gpt-5.4, gpt-4o, o3, claude-opus-4.8, claude-opus-4.7, claude-sonnet-4.6, gemini-3.1-pro, gemini-3.5-flash, deepseek-v4-pro, deepseek-chat, grok-4.3, grok-build-0.1, kimi-k2.6, kimi-k2.5, and the free models (nemotron-3.5-lightning [default], nemotron-3-nano-30b, laguna-xs-2.1, north-mini-code, nemotron-3-nano-omni-30b-a3b-reasoning [vision], nemotron-3-ultra-550b, llama-3.2-11b-vision [vision]). ## Example Output diff --git a/src/exclude-models.test.ts b/src/exclude-models.test.ts index f6974a5..7b8ed86 100644 --- a/src/exclude-models.test.ts +++ b/src/exclude-models.test.ts @@ -81,12 +81,12 @@ describe("ExcludeModels", () => { expect(set.size).toBe(1); }); - it("resolves aliases — 'nvidia' becomes 'free/gpt-oss-120b'", () => { + it("resolves aliases — 'nvidia' becomes 'free/nemotron-3.5-lightning'", () => { const resolved = addExclusion("nvidia", filePath); - expect(resolved).toBe("free/gpt-oss-120b"); + expect(resolved).toBe("free/nemotron-3.5-lightning"); const set = loadExcludeList(filePath); - expect(set.has("free/gpt-oss-120b")).toBe(true); + expect(set.has("free/nemotron-3.5-lightning")).toBe(true); expect(set.size).toBe(1); }); @@ -99,8 +99,8 @@ describe("ExcludeModels", () => { }); it("remove resolves aliases too", () => { - addExclusion("nvidia", filePath); // "nvidia" alias → "free/gpt-oss-120b" - const removed = removeExclusion("gpt-120b", filePath); // different alias for same model + addExclusion("nvidia", filePath); // "nvidia" alias → "free/nemotron-3.5-lightning" + const removed = removeExclusion("free", filePath); // different alias for same model expect(removed).toBe(true); const set = loadExcludeList(filePath); diff --git a/src/models.test.ts b/src/models.test.ts index b876fdc..0bfcfe7 100644 --- a/src/models.test.ts +++ b/src/models.test.ts @@ -35,7 +35,7 @@ describe("resolveModelAlias", () => { it("strips openai/ prefix from virtual routing profiles (issue #78)", () => { // OpenClaw sends virtual profiles as "openai/eco", "openai/auto", etc. expect(resolveModelAlias("openai/eco")).toBe("eco"); - expect(resolveModelAlias("openai/free")).toBe("free/gpt-oss-120b"); // "free" is now an alias, not a virtual profile + expect(resolveModelAlias("openai/free")).toBe("free/nemotron-3.5-lightning"); // "free" is now an alias, not a virtual profile expect(resolveModelAlias("openai/auto")).toBe("auto"); expect(resolveModelAlias("openai/premium")).toBe("premium"); }); diff --git a/src/models.ts b/src/models.ts index eba36d1..1c0bb91 100644 --- a/src/models.ts +++ b/src/models.ts @@ -112,11 +112,14 @@ export const MODEL_ALIASES: Record = { "xai/grok-3-fast": "xai/grok-4-fast-reasoning", // delisted (too expensive) // NVIDIA — backward compat aliases (nvidia/xxx → free/xxx) - // Default free model is gpt-oss-120b (heavy user demand). New free models - // added 2026-04-29 — deepseek-v4-pro/flash, nemotron-omni — are additions, - // not replacements. Retired-with-redirect entries below mirror BlockRun - // server-side decommissioning of slow models (nemotron family, etc.). - nvidia: "free/gpt-oss-120b", + // Default free model is nemotron-3.5-lightning as of the 2026-08-31 rebuild; + // gpt-oss-120b held this slot until it died upstream on 2026-08-16. The + // model-specific aliases below still resolve to their original targets even + // where those are retired — the entries stay in BLOCKRUN_MODELS so an + // explicit pin keeps working; they are simply off the picker and the + // auto-pick cascade. Retired-with-redirect entries mirror BlockRun's own + // server-side decommissioning. + nvidia: "free/nemotron-3.5-lightning", "gpt-120b": "free/gpt-oss-120b", "gpt-20b": "free/gpt-oss-20b", "nvidia/gpt-oss-120b": "free/gpt-oss-120b", @@ -172,7 +175,7 @@ export const MODEL_ALIASES: Record = { devstral: "free/qwen3-coder-480b", "devstral-2": "free/qwen3-coder-480b", maverick: "free/llama-4-maverick", - free: "free/gpt-oss-120b", + free: "free/nemotron-3.5-lightning", // MiniMax (minimax → current flagship: M3) minimax: "minimax/minimax-m3", @@ -1136,6 +1139,98 @@ export const BLOCKRUN_MODELS: BlockRunModel[] = [ reasoning: true, vision: true, }, + { + // Nemotron 3.5 Lightning 30B-A3B — the new free default. blockrun's own + // redirect target for the retired step-3.7-flash, so following it keeps the + // proxy and the gateway naming the same model. Served from OpenRouter's $0 + // pool (4.9s median vs 16.3s direct) which also publishes a 1M window + // against the 131K the NVIDIA NIM path gives; blockrun carries a hidden + // "-nim" twin as its fallback. Probed 1.3s with tools. + id: "free/nemotron-3.5-lightning", + name: "[Free] Nemotron 3.5 Lightning", + version: "3.5-lightning", + inputPrice: 0, + outputPrice: 0, + contextWindow: 1000000, + maxOutput: 16384, + reasoning: true, + }, + { + // Nemotron 3 Nano 30B-A3B — the fastest free model in the catalog + // (~121 tok/s on a realistic workload, not a 16-token ping). Returns + // reasoning_content. Also the tertiary rung of blockrun's own free cascade. + id: "free/nemotron-3-nano-30b", + name: "[Free] Nemotron 3 Nano 30B", + version: "3-nano-30b", + inputPrice: 0, + outputPrice: 0, + contextWindow: 131072, + maxOutput: 16384, + reasoning: true, + }, + { + // Poolside Laguna XS 2.1 on OUR NVIDIA key (~161 tok/s). Deliberately not + // the OpenRouter twin, which 429s on every attempt — so this rung and + // north-mini-code sit on DIFFERENT capacity pools, which is why they are + // adjacent in the cascade. + // NOTE: NOT an nvidia/* id upstream — see FREE_UPSTREAM_OVERRIDES in proxy.ts. + id: "free/laguna-xs-2.1", + name: "[Free] Poolside Laguna XS 2.1", + version: "xs-2.1", + inputPrice: 0, + outputPrice: 0, + contextWindow: 131072, + maxOutput: 16384, + }, + { + // Cohere North Mini Code, on OpenRouter's $0 pool — 607ms median, the + // fastest thing in the tier. Emits reasoning_content (content is clean once + // the budget is large enough to finish, verified live). + // NOTE: NOT an nvidia/* id upstream — see FREE_UPSTREAM_OVERRIDES in proxy.ts. + id: "free/north-mini-code", + name: "[Free] Cohere North Mini Code", + version: "north-mini-code", + inputPrice: 0, + outputPrice: 0, + contextWindow: 256000, + maxOutput: 16384, + reasoning: true, + }, + { + // Nemotron 3 Ultra 550B-A55B — the largest free model ever listed, 1M ctx, + // reachable ONLY through OpenRouter's $0 pool (build.nvidia.com answers 503). + // Deliberately LOW in the cascade: 16.8s on the tools probe, and blockrun + // measured 3 of 15 calls coming back as an HTTP 200 carrying an upstream + // 502/503 error object instead of choices. + id: "free/nemotron-3-ultra-550b", + name: "[Free] Nemotron 3 Ultra 550B", + version: "3-ultra-550b", + inputPrice: 0, + outputPrice: 0, + contextWindow: 1000000, + maxOutput: 16384, + reasoning: true, + }, + { + // Meta Llama 3.2 11B Vision — restores a free Llama after nemotron-super-49b + // (Llama-3.3-based) hit NVIDIA's 410 EOL. A 12-model sweep found exactly two + // survivors and only the 11B finishes a real completion. Older than the rest + // of the tier and deliberately so: it is a real Llama that actually answers. + // Slowest in the tier (~18 tok/s) — last rung of the cascade. + id: "free/llama-3.2-11b-vision", + name: "[Free] Llama 3.2 11B Vision", + version: "3.2-11b-vision", + inputPrice: 0, + outputPrice: 0, + contextWindow: 128000, + maxOutput: 16384, + // NO `vision: true` despite the name and blockrun's `categories: + // ["chat","vision"]` — 2026-08-31, three consecutive 64x64 PNG probes came + // back "I'm unable to see the image" / "you haven't provided an image", + // while a plain-text control on the same id answered fine. The model is + // alive; the image path is not. Mirroring the catalog's claim here would + // have routed image turns to it. See the nano-omni note above. + }, // Z.AI GLM-5 Models { diff --git a/src/proxy.ts b/src/proxy.ts index d336b9a..63d08cb 100644 --- a/src/proxy.ts +++ b/src/proxy.ts @@ -118,13 +118,23 @@ const ROUTING_PROFILES = new Set([ // server-redirected to qwen3-coder), and mistral-small-4-119b (upstream // timing out, 3/3 probes >60s). gpt-oss-120b is the historical // default — heavy users rely on it. New entries appended in chain order. +// 2026-08-31 rebuild. The previous cascade was frozen on 2026-06-08 and every +// rung but nemotron-omni had died: gpt-oss-120b/20b dead upstream since +// 2026-08-16 (the gateway 400s the free/ id) and withheld over NVIDIA's +// prompt-retention terms, deepseek-v4-flash EOL 2026-08-12 (410 Gone), +// qwen3-coder-480b EOL 2026-06-14, llama-4-maverick dropped 2026-07-17. +// None of them failed loudly — blockrun server-redirects retired free ids, so +// callers kept getting answers from a different model, which is exactly the +// shape that silently defeats /exclude. +// Insertion order IS the auto-pick order (pickFreeModel walks it). const FREE_MODELS = new Set([ - "free/gpt-oss-120b", - "free/gpt-oss-20b", - "free/deepseek-v4-flash", // 1M ctx, 1M ctx — NVIDIA v4-pro down 2026-05-14 - "free/qwen3-coder-480b", - "free/llama-4-maverick", - "free/nemotron-3-nano-omni-30b-a3b-reasoning", // first vision-capable free + "free/nemotron-3.5-lightning", // free-tier default — 1M ctx, thinking mode + "free/nemotron-3-nano-30b", // fastest free model (~121 tok/s) + "free/laguna-xs-2.1", // coding, ~161 tok/s — on our NVIDIA key + "free/north-mini-code", // coding, 607ms median — OpenRouter $0 pool + "free/nemotron-3-nano-omni-30b-a3b-reasoning", // vision (text/image/video/audio) + "free/nemotron-3-ultra-550b", // 1M ctx flagship + "free/llama-3.2-11b-vision", // vision ]); /** Pick the best available free model that isn't excluded. */ function pickFreeModel(excludeList?: Set): string | undefined { @@ -134,13 +144,26 @@ function pickFreeModel(excludeList?: Set): string | undefined { return undefined; // all free models excluded } // Keep backward-compat constant for places that don't have excludeList in scope -const FREE_MODEL = "free/gpt-oss-120b"; +const FREE_MODEL = "free/nemotron-3.5-lightning"; /** - * Map free/xxx model IDs to nvidia/xxx for upstream BlockRun API. - * The "free/" prefix is a XClawRouter convention for the /model picker; - * BlockRun server expects "nvidia/" prefix. + * Upstream ids for free models that are not NVIDIA-hosted. The free/ prefix is + * an XClawRouter picker convention; upstream most of these are nvidia/xxx, but + * north-mini-code is Cohere's and laguna-xs-2.1 is Poolside's, so the blanket + * rewrite below would send both to an id the gateway does not serve. + */ +const FREE_UPSTREAM_OVERRIDES: Record = { + "free/north-mini-code": "cohere/north-mini-code", + "free/laguna-xs-2.1": "poolside/laguna-xs-2.1", +}; + +/** + * Map free/xxx model IDs to their upstream BlockRun ID. + * The "free/" prefix is a XClawRouter convention for the /model picker; upstream + * they are `nvidia/xxx` by default, or whatever FREE_UPSTREAM_OVERRIDES says. */ function toUpstreamModelId(modelId: string): string { + const override = FREE_UPSTREAM_OVERRIDES[modelId]; + if (override) return override; if (modelId.startsWith("free/")) { return "nvidia/" + modelId.slice("free/".length); } diff --git a/src/router/config.ts b/src/router/config.ts index fb5096b..2681294 100644 --- a/src/router/config.ts +++ b/src/router/config.ts @@ -1051,7 +1051,7 @@ export const DEFAULT_ROUTING_CONFIG: RoutingConfig = { "google/gemini-2.5-flash-lite", // 1,353ms, $0.10/$0.40 "openai/gpt-5.4-nano", // $0.20/$1.25, 1M context "xai/grok-4-fast-non-reasoning", // 1,143ms, $0.20/$0.50 — fast fallback - "free/gpt-oss-120b", // 1,252ms, FREE fallback + "free/nemotron-3.5-lightning", // FREE fallback ], }, MEDIUM: { @@ -1094,10 +1094,13 @@ export const DEFAULT_ROUTING_CONFIG: RoutingConfig = { // Eco tier configs - absolute cheapest (blockrun/eco) ecoTiers: { SIMPLE: { - primary: "free/gpt-oss-120b", // FREE! $0.00/$0.00 — heavy user default + primary: "free/nemotron-3.5-lightning", // FREE $0.00/$0.00 — 1M ctx, thinking mode fallback: [ - "free/gpt-oss-20b", // FREE — smaller, faster - "free/deepseek-v4-flash", // FREE — 1M context, ~5x faster than v4-pro + // The free head keeps rotting with NVIDIA's hosting: deepseek-v4-flash + // 410 on 2026-08-12, gpt-oss-120b/20b dead since 2026-08-16, and on + // 2026-08-30 four of the five then-visible free models went at once. + // Each retirement retargets these free rungs; the paid rungs never move. + "free/nemotron-3-nano-30b", // FREE — fastest free model (~121 tok/s) "google/gemini-3.1-flash-lite", // $0.25/$1.50 — newest flash-lite "openai/gpt-5.4-nano", // $0.20/$1.25 — fast nano "google/gemini-2.5-flash-lite", // $0.10/$0.40 @@ -1173,7 +1176,7 @@ export const DEFAULT_ROUTING_CONFIG: RoutingConfig = { "openai/gpt-5.4", // Previous flagship (slow but stable, benchmarked at 6,213ms) "openai/gpt-5.3-codex", "deepseek/deepseek-chat", // Cheap, reliable - "free/qwen3-coder-480b", // NVIDIA free ultimate backstop + "free/nemotron-3.5-lightning", // free ultimate backstop ], }, REASONING: { @@ -1222,7 +1225,7 @@ export const DEFAULT_ROUTING_CONFIG: RoutingConfig = { "openai/gpt-5.5", // Newest flagship — native agent + computer use (exactly the agentic-tier use case) "openai/gpt-5.4", // Previous flagship — 6,213ms, reliable "deepseek/deepseek-chat", // 1,431ms — cheap, reliable - "free/qwen3-coder-480b", // NVIDIA free ultimate backstop + "free/nemotron-3.5-lightning", // free ultimate backstop ], }, REASONING: { diff --git a/src/top-models.json b/src/top-models.json index d1eedce..84f5abe 100644 --- a/src/top-models.json +++ b/src/top-models.json @@ -40,13 +40,14 @@ "xai/grok-4-1-fast-reasoning", "minimax/minimax-m3", "minimax/minimax-m2.7", - "free/gpt-oss-120b", - "free/gpt-oss-20b", - "free/deepseek-v4-flash", - "free/qwen3-coder-480b", - "free/llama-4-maverick", - "free/nemotron-3-nano-omni-30b-a3b-reasoning", "zai/glm-5.1", "zai/glm-5", - "zai/glm-5-turbo" + "zai/glm-5-turbo", + "free/nemotron-3.5-lightning", + "free/nemotron-3-nano-30b", + "free/laguna-xs-2.1", + "free/north-mini-code", + "free/nemotron-3-nano-omni-30b-a3b-reasoning", + "free/nemotron-3-ultra-550b", + "free/llama-3.2-11b-vision" ] diff --git a/src/top-models.test.ts b/src/top-models.test.ts index 1769de9..2a1dec1 100644 --- a/src/top-models.test.ts +++ b/src/top-models.test.ts @@ -1,8 +1,46 @@ import { describe, expect, it } from "vitest"; +import { BLOCKRUN_MODELS } from "./models.js"; import topModelsJson from "./top-models.json"; import { TOP_MODELS } from "./top-models.js"; +/** + * Free ids that must never return to the picker or the auto-pick cascade. + * + * Every one of these was still being advertised on 2026-08-31, months after it + * died, because nothing checked. They do not fail loudly: blockrun hides a + * retired free id and server-redirects the call, so the user gets an answer + * from a different model and `/exclude` on the id they can see does nothing. + * + * Append on every retirement. The entries may stay in BLOCKRUN_MODELS so an + * explicit pin keeps resolving — this guard is about user-facing surfaces. + */ +const RETIRED_FREE_IDS = [ + "free/gpt-oss-120b", // dead upstream 2026-08-16; withheld over NVIDIA prompt-retention terms + "free/gpt-oss-20b", // same + "free/deepseek-v4-flash", // 410 Gone at NVIDIA 2026-08-12 (blockrun #367) + "free/qwen3-coder-480b", // NVIDIA EOL 2026-06-14 + "free/glm-4.7", // NIM hung, server-redirected to qwen3-coder + "free/llama-4-maverick", // dropped from the live catalog 2026-07-17 + "free/mistral-large-3-675b", // 410 Gone in blockrun's 2026-07-28 re-probe + "free/seed-oss-36b", // NVIDIA sweep 2026-08-30 + "free/mistral-nemotron", // NVIDIA sweep 2026-08-30 (hung, not 410) + "free/step-3.7-flash", // NVIDIA sweep 2026-08-30 + "free/nemotron-nano-9b-v2", // NVIDIA sweep 2026-08-30 + "free/nemotron-nano-12b-v2-vl", // NVIDIA sweep 2026-08-30 +] as const; + +/** The live free tier as of the 2026-08-31 rebuild, in auto-pick order. */ +const LIVE_FREE_IDS = [ + "free/nemotron-3.5-lightning", + "free/nemotron-3-nano-30b", + "free/laguna-xs-2.1", + "free/north-mini-code", + "free/nemotron-3-nano-omni-30b-a3b-reasoning", + "free/nemotron-3-ultra-550b", + "free/llama-3.2-11b-vision", +] as const; + describe("TOP_MODELS", () => { it("loads the shared curated allowlist from top-models.json", () => { expect(TOP_MODELS).toEqual(topModelsJson); @@ -11,4 +49,21 @@ describe("TOP_MODELS", () => { expect(TOP_MODELS).toContain("xai/grok-4-0709"); expect(TOP_MODELS).toContain("deepseek/deepseek-reasoner"); }); + + it("advertises no retired free model", () => { + const resurrected = RETIRED_FREE_IDS.filter((id) => TOP_MODELS.includes(id)); + expect(resurrected).toEqual([]); + }); + + it("advertises the live free tier, contiguously at the end and in cascade order", () => { + const freeTail = TOP_MODELS.filter((id) => id.startsWith("free/")); + expect(freeTail).toEqual([...LIVE_FREE_IDS]); + expect(TOP_MODELS.slice(-freeTail.length)).toEqual(freeTail); + }); + + it("defines every advertised model, so no picker entry is a phantom id", () => { + const defined = new Set(BLOCKRUN_MODELS.map((m) => m.id)); + const phantom = TOP_MODELS.filter((id) => !defined.has(id)); + expect(phantom).toEqual([]); + }); }); diff --git a/test/integration/exclude-models.test.ts b/test/integration/exclude-models.test.ts index c1c8bfe..237cdcc 100644 --- a/test/integration/exclude-models.test.ts +++ b/test/integration/exclude-models.test.ts @@ -70,7 +70,7 @@ describe("exclude-models e2e", () => { it("exclude filter log appears for excluded models in eco SIMPLE tier", async () => { // Verify excluded models ARE in the unfiltered eco SIMPLE chain const ecoSimpleChain = getFallbackChain("SIMPLE", DEFAULT_ROUTING_CONFIG.ecoTiers!); - expect(ecoSimpleChain).toContain("free/gpt-oss-120b"); + expect(ecoSimpleChain).toContain("free/nemotron-3.5-lightning"); consoleLogs.length = 0;