Skip to content

Instant recall's confirm and verify steps are not gated by the spend ceiling #589

Description

@Smana

Found while reviewing #587.

Recall.affordRerank gates the rerank call on max_tokens_per_investigation / max_cost_per_investigation via the loop's own budgetTrip. The recall it produces then runs confirmRecall and verifyFindings (loop.go around the tryRecall path, verify.go m.Complete), and neither consults the ceiling. So the gate on the rank call is what stops an over-budget investigation from paying for a verify.

Two consequences worth deciding on together:

  1. With rerank_backend: jev the rank call itself spends no LLM tokens, yet it must stay gated, because letting it through would buy a verify the ceiling was about to refuse. fix(recall): overlap the shadow arm, split the confidence histogram by choice #587 keeps it gated and says why. If verify were gated, a jev recall could be genuinely free on an over-budget incident, which is the outcome the feat(providers): a decision model behind the recall reranker and curation dedup #581 review wanted.
  2. Independently of the backend, a recall that fires just under the ceiling can still cross it in confirmRecall + verifyFindings with no budget_exceeded outcome, because those completions are not measured against the running total before being made.

Suggested shape: consult spend.refuses(estimate) before the confirm and verify completions on the recall path (the same predicate affordRerank uses), record a distinct rejection reason when they are refused, and then let affordRerank skip the gate for backends whose rank call is free.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions