You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Recall.affordRerank gates the rerank call on max_tokens_per_investigation / max_cost_per_investigation via the loop's own budgetTrip. The recall it produces then runs confirmRecall and verifyFindings (loop.go around the tryRecall path, verify.gom.Complete), and neither consults the ceiling. So the gate on the rank call is what stops an over-budget investigation from paying for a verify.
Independently of the backend, a recall that fires just under the ceiling can still cross it in confirmRecall + verifyFindings with no budget_exceeded outcome, because those completions are not measured against the running total before being made.
Suggested shape: consult spend.refuses(estimate) before the confirm and verify completions on the recall path (the same predicate affordRerank uses), record a distinct rejection reason when they are refused, and then let affordRerank skip the gate for backends whose rank call is free.
Found while reviewing #587.
Recall.affordRerankgates the rerank call onmax_tokens_per_investigation/max_cost_per_investigationvia the loop's ownbudgetTrip. The recall it produces then runsconfirmRecallandverifyFindings(loop.goaround thetryRecallpath,verify.gom.Complete), and neither consults the ceiling. So the gate on the rank call is what stops an over-budget investigation from paying for a verify.Two consequences worth deciding on together:
rerank_backend: jevthe rank call itself spends no LLM tokens, yet it must stay gated, because letting it through would buy a verify the ceiling was about to refuse. fix(recall): overlap the shadow arm, split the confidence histogram by choice #587 keeps it gated and says why. If verify were gated, ajevrecall could be genuinely free on an over-budget incident, which is the outcome the feat(providers): a decision model behind the recall reranker and curation dedup #581 review wanted.confirmRecall+verifyFindingswith nobudget_exceededoutcome, because those completions are not measured against the running total before being made.Suggested shape: consult
spend.refuses(estimate)before the confirm and verify completions on the recall path (the same predicateaffordRerankuses), record a distinct rejection reason when they are refused, and then letaffordRerankskip the gate for backends whose rank call is free.