Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
54 changes: 54 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,60 @@

All notable changes to blockrun-llm will be documented in this file.

## Unreleased

### Changed
- **Router Core re-synced to upstream `5ee7c23`** (was `d7bc10c`, two commits
behind — the same pin the TypeScript SDK bundles). Upstream V3.5 rebuilds
every tier chain around ids the public catalog actually lists: the withheld
`kimi-k2.5/k2.6/k2.7`, both grok-4-fast pairs, `grok-4-0709`,
`claude-opus-4.6` and `gemini-3-pro-preview` are gone from every rung,
including fallbacks, so a routed model is always one a user can find on
blockrun.ai/models. Before this sync every profile carried 3–4 off-catalog
rungs per decision; now it carries none.

Primaries moved only where `portfolio.py` already holds calibration evidence
for the successor — Gemini 3.5 Flash where Kimi K2.7 was, GPT-5 Mini for
agentic MEDIUM, Sonnet 5 over Sonnet 4.6, DeepSeek Reasoner for the cheap
reasoning head. The newer generation the catalog already sells enters as
fallback rungs: GPT-5.6 Luna/Terra, Gemini 3.6 Flash and 3.5 Flash-Lite,
GLM-5.3 and 5.3-Flash, Grok 4.3 and 4.5, Kimi K3, Qwen 3.7 Plus, MiniMax M3.
Promotion waits for a calibration run, because version recency is not a
quality signal. The expired GLM-5.1 promotion is dropped; the promotions
mechanism stays wired with an empty list.

- **Routing priors regenerated: 66 model profiles** (was 30) and a **71-model
capability snapshot** (`model_capabilities.py`), both from upstream's probe
and catalog-sync scripts. Two stale capability values had been reaching the
hard filter: Haiku 4.5 at 8K max output (actually 64K) and Sonnet 4.6 at 200K
context (actually 1M). Kimi K3 replaces K2.7 in the Mandarin extraction band,
widened to 0.12 so the auto affinity floor gap (0.10) cannot let price
re-select a non-native model, and K3 joins the extraction evidence pool since
it is no longer on the MEDIUM chain.

### Fixed
- **`routing_profile="free"` had collapsed to a single model with no
fallbacks.** Four of the five ids in the SDK-only `FREE_TIERS` table —
`nvidia/step-3.7-flash`, `nemotron-nano-9b-v2`, `mistral-nemotron`,
`nemotron-nano-12b-v2-vl` — were retired by NVIDIA, and they were every
primary plus all but one fallback. Nothing looked broken because the gateway
server-redirects a retired free id: a dead rung returns 200 and answers
normally, while quietly serving a different model. That is the shape that
defeats a host's `unavailable_models`, and it is why the table rotted
unnoticed.

The table is rebuilt from ids verified by a two-pass probe that reads back
the response's own `model` field, since a 200 proves nothing. Two models the
catalog still prices at $0 did not survive that check and are excluded:
`nemotron-3-ultra-550b` and `nemotron-3-nano-omni-30b-a3b-reasoning` both
answer as `nemotron-3-nano-30b`. Every tier is back to four candidates, and
the free tier is no longer NVIDIA-only — `cohere/north-mini-code` and
`poolside/laguna-xs-2.1` serve at $0 and carry the free coding load.

A new test asserts fallback *depth* per tier, not just membership: the table
stayed internally consistent all through the rot, so membership alone could
never have caught it.

## 1.13.0 — 2026-08-26

### Fixed
Expand Down
60 changes: 36 additions & 24 deletions blockrun_llm/router_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -61,47 +61,59 @@ class ResolvedRoutingDecision(RoutingDecision, total=False):
#: The BlockRun free tier is a gateway concept, not a Router Core profile: the
#: core's tiers rank paid models by task affinity, and its evidence candidates
#: are paid ids. ``routing_profile="free"`` therefore routes on the rules
#: strategy over this NVIDIA-only tier table, and the adapter additionally
#: drops any candidate the catalog does not price at $0.
#: strategy over this tier table, and the adapter additionally drops any
#: candidate the catalog does not price at $0.
#:
#: Refreshed 2026-08-15 against the live catalog. NVIDIA has EOL'd (HTTP 410)
#: the free DeepSeek family — ``deepseek-v4-flash`` was the last to go on
#: 2026-08-12 — plus ``llama-4-maverick`` and the qwen3 SKUs, which is what the
#: previous table pointed at. ``gpt-oss-120b/20b`` stay out of the primaries:
#: they are hidden from ``/v1/models`` (so they carry no catalog price) over
#: Refreshed 2026-08-31. Every id below was verified by asking the gateway for
#: it twice and reading back the ``model`` field of the reply, because a 200 is
#: not proof: blockrun server-redirects a retired free id to a live one, so a
#: dead rung answers normally while quietly serving something else. That is the
#: shape that defeats a host's ``/exclude``, and it is why the previous table
#: went stale unnoticed. Substituting on 2026-08-31, hence absent here:
#: ``step-3.7-flash``, ``nemotron-nano-9b-v2``, ``nemotron-nano-12b-v2-vl`` and
#: ``mistral-nemotron`` (retired upstream 2026-08-30), plus ``nemotron-3-ultra-550b``
#: and ``nemotron-3-nano-omni-30b-a3b-reasoning`` — the latter two still list at
#: $0 in ``/v1/models`` but both answer as ``nemotron-3-nano-30b``.
#:
#: The table is no longer NVIDIA-only: ``cohere/north-mini-code`` and
#: ``poolside/laguna-xs-2.1`` serve at $0 and carry the free coding load.
#: ``gpt-oss-120b/20b`` stay out — proxy-only ids with no catalog price, under
#: the NVIDIA free tier's prompt-retention policy.
FREE_TIERS: dict[str, TierConfig] = {
"SIMPLE": {
"primary": "nvidia/step-3.7-flash", # 131K ctx, fast general chat + reasoning
# Fastest free model (~121 tok/s), and latency is the only axis that
# separates free rungs — they all cost $0.
"primary": "nvidia/nemotron-3-nano-30b", # 131K ctx
"fallback": [
"nvidia/nemotron-nano-9b-v2",
"nvidia/mistral-nemotron",
"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
"nvidia/nemotron-3.5-lightning",
"nvidia/llama-3.2-11b-vision",
"poolside/laguna-xs-2.1",
],
},
"MEDIUM": {
"primary": "nvidia/step-3.7-flash",
"primary": "nvidia/nemotron-3.5-lightning", # 1M ctx — free tier flagship
"fallback": [
"nvidia/mistral-nemotron",
"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
"nvidia/nemotron-nano-9b-v2",
"nvidia/nemotron-3-nano-30b",
"poolside/laguna-xs-2.1",
"cohere/north-mini-code",
],
},
"COMPLEX": {
# Largest free context (256K) and the only free vision model, so it also
# absorbs long or multi-modal requests.
"primary": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
# Only free model above 256K, so it absorbs long inputs; the vision rung
# behind it absorbs multi-modal ones.
"primary": "nvidia/nemotron-3.5-lightning",
"fallback": [
"nvidia/step-3.7-flash",
"nvidia/mistral-nemotron",
"nvidia/nemotron-nano-12b-v2-vl",
"cohere/north-mini-code", # 256K ctx
"nvidia/nemotron-3-nano-30b",
"nvidia/llama-3.2-11b-vision", # free vision
],
},
"REASONING": {
"primary": "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
"primary": "nvidia/nemotron-3.5-lightning",
"fallback": [
"nvidia/step-3.7-flash",
"nvidia/nemotron-nano-9b-v2",
"nvidia/nemotron-3-nano-30b",
"cohere/north-mini-code",
"poolside/laguna-xs-2.1",
],
},
}
Expand Down
2 changes: 1 addition & 1 deletion blockrun_llm/router_core/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
Router Core — deterministic, constraint-first model routing.

Python port of `@blockrun/router-core <https://github.com/BlockRunAI/router-core>`_
(upstream commit ``d7bc10c``), the same routing engine the TypeScript SDK and
(upstream commit ``5ee7c23``), the same routing engine the TypeScript SDK and
the BlockRun gateway use. The package is deliberately product-neutral: task
classification, hard capability filtering, portfolio scoring, ordered
fallbacks, and routing configuration. It contains no wallet, gateway client,
Expand Down
Loading
Loading