Skip to content

JVNAUTOSCI-2614: Select minimum-adequate capability plans - #329

Merged
witbrock merged 1 commit into
mainfrom
codex/JVNAUTOSCI-2614-cost-sensitive-selection
Jul 29, 2026
Merged

witbrock merged 1 commit into
mainfrom
codex/JVNAUTOSCI-2614-cost-sensitive-selection

Conversation

@witbrock

Copy link
Copy Markdown
Member

What changed

  • Treat registered calls and represented workflows as capability plans on one bounded, cost-sensitive frontier.
  • Project declared workflow components, step/component counts, semantic effects, durable-runtime cost, and optional represented direct-equivalence metadata.
  • Promote actor-visible components of a semantically matched workflow as plausible simpler plans while leaving semantic adequacy with the adaptive model.
  • Apply mechanical dominance only to read-only plans with an explicit authoritative direct_equivalent_capability_names declaration; retain dominated workflows for exact-name inspection.
  • Distinguish a read-only workflow's semantic effect from its durable instance state effect.
  • Record the selected plan/effect/cost profile and actual transport cost in turn telemetry.
  • Replace substring matching and generic one-word prose matches with whole-token, corpus-relative routing evidence, and prioritise structurally declared components over weak lexical matches.
  • Document the routing-profile extension and its conservative use.

Why

Workflow relevance was scaled into a separate score range (up to 1000) while direct capabilities received small literal scores. That made representedness an accidental priority and prevented the model from comparing a matched workflow with its simpler direct components. Short substrings and generic routing words also admitted unrelated tools.

The new boundary keeps authority unchanged and does not add a workflow classifier or compulsory controller stage. The adaptive model judges material outcome/evidence adequacy; declared cost breaks ties only after adequacy, and hard pruning requires represented equivalence.

Impact

For the reported Can you list PhD students supervised by me? shape, the canonical entity-retrieval workflow's five read components and the workflow occupy the top six catalogue positions. find_relations_with_argument is fourth, while the workflow remains available when its full five-part evidence contract is genuinely needed.

Validation

  • 111 passed post-rebase across adaptive-turn, workflow-turn capability, routing-profile extension, and routing metadata projection tests.
  • Ruff and git diff --check pass.
  • Broader workflow/discovery validation reached 201 passed before stopping the unrelated Mongo-heavy remainder after 13m46s with no failures.
  • Faithful replay with the real default catalogue, canonical entity-retrieval seed, and live tool metadata placed all five declared read components plus the workflow in the top six and recorded the workflow as semantic read-only but operationally durable.
  • Live Vontology discovery was read-only exercised but the current capability index reported capability_index_wait_timed_out_build_in_progress; this is a separate availability boundary, not treated as selection success.

Jira: JVNAUTOSCI-2614

Copilot AI review requested due to automatic review settings July 29, 2026 19:48
@witbrock
witbrock marked this pull request as ready for review July 29, 2026 19:48

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates Von’s ordinary-turn capability discovery and ranking so that registered direct tools and represented workflows are treated as comparable “capability plans” on a single, bounded, cost-sensitive frontier. It adds explicit plan/effect/cost metadata to catalogue entries, promotes workflow-declared component tools as plausible simpler alternatives, and introduces conservative mechanical pruning only when a workflow declares an authoritative read-only direct equivalent.

Changes:

  • Add workflow plan projection (declared components, step counts, direct-equivalence metadata) and effect profiling (semantic vs operational durable-state effect) to represented workflow capabilities.
  • Replace previous substring/generic-token routing with whole-token, corpus-relative evidence and interleaved frontier ordering that keeps both direct and workflow options visible while leaving adequacy to the adaptive model.
  • Record selection policy and per-invocation plan/effect/cost context in telemetry traces, and expand/adjust test coverage accordingly.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/backend/test_workflow_turn_capability_service.py Adds/updates assertions for workflow plan metadata, semantic vs operational effects, and write/unknown effect marking behaviour.
tests/backend/test_adaptive_turn_service.py Adds extensive tests for the new frontier ranking, component promotion, rarity-based routing evidence, dominance pruning rules, and selection trace enrichment.
src/backend/services/workflow_turn_capability_service.py Projects workflow plan/effect/cost metadata into catalogue entries and allows discovery to use registered capability categories to classify semantic effect when evidence is sufficient.
src/backend/services/adaptive_turn_service.py Implements the cost-sensitive frontier ranking, corpus-relative routing evidence, conservative dominance pruning, compact projection of selection profiles, and selection telemetry enrichment.
docs/engineering/von_workflow_language_manual.md Documents direct_equivalent_capability_names and the new capability-plan selection/dominance rules.

@witbrock
witbrock merged commit 8e51186 into main Jul 29, 2026
5 checks passed
@witbrock
witbrock deleted the codex/JVNAUTOSCI-2614-cost-sensitive-selection branch August 1, 2026 14:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants