fix(serve): honest overshoot notice when edges push a complete answer over budget (#2784) - #2785
Conversation
There was a problem hiding this comment.
Graphify reviewed this change.
Looks safe to merge — no coupling regressions and no blocking issues, checked against the code graph (not a self-assessment).
Formal verification. 1 change(s) tested, no difference found (not proven).
Graphify review — findings
This PR modifies the _subgraph_to_text function in graphify/serve.py to change how it handles the case where all nodes fit within the budget but the total output (including edges) exceeds the requested token budget. Previously it returned the full output silently; now it prepends an informational notice reporting the actual size and count of nodes/edges when the complete answer exceeds the budget, while still returning the full untruncated content. Two new tests are added to tests/test_serve.py: one asserting the new overshoot notice appears (with expected counts and specific wording constraints) when edges push output over budget, and one asserting no notice is added when the complete answer already fits within the budget.
No blocking issues surfaced. 4 lower-confidence candidates did not survive cross-model review.
Analysis details — impact, health, verification
Impact & health
Graphify review
Impact — 522 functions depend on the 315 functions this change touches.
Health — this change adds coupling hotspots:
- new:
dispatch_command()— 2 callers, 117 callees - new:
_query_graph_text()— 18 callers, 8 callees - new:
_score_query()— 15 callers, 5 callees - new:
_query_terms()— 17 callers, 3 callees - new:
run_benchmark()— 16 callers, 3 callees - new:
_build_server()— 2 callers, 16 callees - new:
_load_graph()— 9 callers, 3 callees - new:
_query_subgraph_tokens()— 7 callers, 3 callees - …and 9 more
Verification — 522 functions in the blast radius were not formally verified this run (proofs are advisory here).
Gate & verification
graphify gate
PASS — objectively clean (no health regressions, tests not run — proofs not run this pass (advisory)). Grounded, not self-assessed.
Advisory (not blocking):
- verification_scope: 369 function(s) in the blast radius were not formally verified this run
Formal verification
No difference found (not proven): No behavior difference found in \_subgraph\_to\_text (not a proof).
The verifier ran both versions of \_subgraph\_to\_text on many inputs and saw identical behavior every time. Strong evidence the change is safe, but evidence, not a proof.
Guarantee: Empirical: differential testing (both versions run on many generated inputs). A divergence on an untested input remains possible, so this is 'no counterexample found', not 'proven equivalent'.
Note: An input the sampler did not try could still differ.
· 17 more finding(s) on lines outside this diff (see the check run).
|
Landed in v0.9.46, just published to PyPI. Cherry-picked onto |
Fixes #2784.
Root cause:
_subgraph_to_text's budget cut computescut_count(nodesdropped) and, when it's 0, returned the full
outputstring unconditionally— bypassing the char_budget check for the EDGE section entirely. That's the
step-change from the issue: once node count stops growing with the budget,
edges get appended with no size check at all (4.5x-6.2x overshoot observed).
This return-everything-when-cut_count==0 path was introduced by #2601
(correctly stopping a misleading "0 nodes cut" truncation banner), so this
fix preserves that guarantee — edges are still never dropped once every node
fits — but replaces the silent bypass with an honest notice: real node/edge
counts and an estimated token count vs. the requested budget. It
deliberately does not repeat "raise --budget" advice in this branch, since
that's exactly what triggers the blow-up in the original report.
Tests: the existing #2601 regression test
(
test_subgraph_to_text_no_banner_when_only_edges_overflow) passesunmodified. Added two new tests — one covering the overshoot-notice path,
one confirming no notice appears when the complete answer already fits the
budget.
uv run pytest tests/ -q— 4518 passed, 7 skipped, 0 failures.