Skip to content

Improve Thinking workflow labels and model telemetry - #351

Merged
witbrock merged 1 commit into
mainfrom
fix/thinking-workflow-display-labels
Aug 6, 2026
Merged

witbrock merged 1 commit into
mainfrom
fix/thinking-workflow-display-labels

Conversation

@witbrock

@witbrock witbrock commented Aug 6, 2026

Copy link
Copy Markdown
Member

Summary

  • carry a trusted, human-readable display label for represented workflow capabilities while retaining the per-turn capability ID and canonical workflow identity
  • render the human workflow name in Default and Expert Thinking modes, with raw invocation IDs reserved for Debug
  • recover useful workflow names for legacy stored Thinking cards without workflow-specific mappings or a new database lookup
  • preserve requested, selected, and provider-observed model identities in durable workflow LLM-call telemetry

Why

Represented workflows were surfaced as Represented Workflow <hash>, so ordinary users could not tell what Von was doing. Separately, nested workflow recorders accepted only the selected model name and dropped the requested/effective identity fields already produced by the gateway.

User impact

Default Thinking now explains the named workflow in human terms. Expert retains semantic and canonical identity, while Debug adds the raw invocation identifier. Workflow telemetry can now distinguish the model the user requested, the model routing selected, and the model the provider reported.

Validation

  • 194 affected backend tests passed
  • 255 Thinking frontend tests passed
  • frontend static lint passed
  • git diff --check passed
  • authenticated live browser replay of stored request 727deaec-f483-47ce-a752-f2746b9b38b7: Default displayed “Arxiv Paper Representation Workflow” without the generated or raw hash; Expert retained the human workflow name; Debug exposed the raw invocation ID alongside it

The historic represented-workflow invocation used for display validation had failed; the browser replay validates display and legacy recovery, not that historic workflow effect.

Jira: JVNAUTOSCI-2619

Copilot AI lite review requested due to automatic review settings August 6, 2026 11:00

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR improves how “represented workflow” capabilities are labelled and displayed in Thinking cards by carrying a trusted human-readable display name alongside stable capability/workflow identities, and it expands workflow LLM telemetry to preserve requested/selected/provider-observed model identities.

Changes:

  • Add capability_display_name to represented-workflow invocation summaries/diagnostics and propagate it through adaptive turn execution and MCP catalogue summaries.
  • Update Thinking semantic projection + frontend rendering to prefer human workflow labels in Default/Expert modes (with canonical IDs still inspectable, and raw invocation IDs reserved for Debug), including legacy-card name recovery without DB lookups.
  • Extend durable LLM-call recording to capture requested_model, selected_model, effective_model, and model_identity_source, with updated backend test coverage.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file
File Description
tests/backend/test_turn_execution_record_service_execution_summary.py Asserts capability_display_name is preserved in execution summaries for represented workflows.
tests/backend/test_tool_result_hint_actions.py Verifies durable LLM-call outputs include requested/selected model identities across fallbacks.
tests/backend/test_thinking_semantic_projection_service.py Adds coverage for human workflow labels while retaining stable identities and legacy normalisation behaviour.
tests/backend/test_rag_turn_execution_records_mcp_read_tools.py Ensures diagnostics hydration and turn-execution read paths include capability_display_name.
tests/backend/test_llm_step_executor.py Expands assertions to confirm gateway LLM-call metadata is preserved end-to-end.
tests/backend/test_adaptive_turn_service.py Validates represented workflow invocations and progress events carry the display name and semantic operation label.
src/frontend/web/von_interface/static/js/test/chatTab.test.js Adds Thinking UI tests for default/expert/debug display rules and legacy workflow-name recovery.
src/frontend/web/von_interface/static/js/chatTab.js Implements capability descriptor extraction + opaque-label replacement + mode-specific workflow display behaviour.
src/backend/workflows/llm_step_executor.py Records requested/selected/effective model identities (and source) into workflow LLM-call telemetry.
src/backend/workflows/durable/tool_result_hint_actions.py Mirrors LLM-call metadata recording for durable nested workflow recorders.
src/backend/services/turn_execution_record_service.py Includes capability_display_name in summarised tool invocation projections.
src/backend/services/turn_execution_diagnostics_service.py Includes capability_display_name in summarised tool invocation history for diagnostics.
src/backend/services/thinking_semantic_projection_service.py Introduces bounded capability label selection logic and uses it in semantic operation normalisation/projection.
src/backend/services/adaptive_turn_service.py Threads workflow display names through contained capability execution and semantic projection emission.
src/backend/integrations/internal_mcp/catalogue.py Adds capability_display_name to MCP catalogue turn-execution tool invocation summaries.

@witbrock
witbrock merged commit 9815daa into main Aug 6, 2026
5 checks passed
@witbrock
witbrock deleted the fix/thinking-workflow-display-labels branch August 18, 2026 20:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants