feat(mcp): integrate diagnostics for end-to-end rehearsal

[reachjalil]

MCP diagnostics integration rehearsal

Important

This is an independent, agent-run integration rehearsal, not the controlled release branch.

• Agent verification: Passed
• Verified by Jalil: Not started
• Integrated into the real parent branch: No
• Parent release ledger: #2674

Managerial summary

We set out to make enterprise MCP failures understandable at three levels:

1. explain exactly which connection phase failed, who owns the next action,
   and what support-safe evidence exists;
2. provide a deterministic ServiceNow/Microsoft-style OAuth and MCP server so
   the workflow can be tested without customer data or live-provider risk; and
3. let an organization administrator watch Den diagnose a connection in real
   time, preserve the highest proven health, repair the fault, and retry.

Each level was originally built as a separate source PR. This branch plays out
Jalil's proposed release process independently: merge the levels in order,
review every overlap semantically, exercise them as one user journey, fix what
the integration exposes, and record the evidence before anything is accepted
into the real feature branch.

The result is positive: the three levels now work together in the rehearsal.
The agent found and fixed several release-blocking correctness and support
problems along the way. This does not replace Jalil's review, and it does
not prove conformance with a real ServiceNow or Microsoft 365 tenant.

Before and after: combined rehearsal

Before this PR	Added or proven in this PR	Practical outcome
Structured errors, the enterprise mock/test, and live tracing had each passed separately but had not been merged and exercised as one release candidate.	The three source heads were integrated in release order, their overlapping MCP client/transport changes were reconciled, and the result was exercised through one operator journey.	We now know the three levels can work together; this is stronger evidence than three isolated green PRs.
Several cross-branch correctness problems were invisible in isolation.	The rehearsal fixed the public callback origin, late persistence after timeout, stale post-callback state, 401-versus-403 classification, MCP isError handling, deletion races, realistic confidential-client proof, fixture leaks, and proof-viewer clarity/accessibility issues.	Jalil is less likely to encounter these failures during the controlled review.
There was no compact way for a maintainer to replay what the browser automation proved.	A small React proof viewer presents the canonical screenshots, machine assertions, evidence boundaries, status, and limitations as a guided story.	A reviewer can understand the result without first operating the entire stack.
“Agent tested” risked being read as “ready to merge.”	The PR visibly separates agent verification, Jalil verification, parent integration, and live-provider validation.	The rehearsal informs the decision but does not make it.

Not claimed: no real customer ServiceNow or Microsoft tenant was used; the parent release branch remains unchanged; and Connect still requires the separate Diagnose action for the full timeline. The recommended promotion path remains one accepted source level at a time through #2674.

Current checkpoint

Capability	Source PR	Source head or rehearsal branch	Agent-verified here	Verified by Jalil	In parent #2674
Structured connection errors	#2669	7b99407c	Yes	Not started	No
Enterprise diagnostic mock and read-only connection test	#2670	e1569ec3	Yes	Not started	No
Live Den diagnostic tracing	#2672	3e84bd30	Yes	Not started	No
Combined rehearsal	This PR	fc21b086 on feature/mcp-diagnostics-integration-rehearsal	Yes	Not started	Not applicable

The real release branch remains deliberately unchanged. The next action is for
Jalil to choose one capability, review its short demonstration, and decide
whether it is accepted, needs another fix, or should be deferred.

What the combined journey now proves

The browser automation performs one connected operator scenario:

1. An unreachable endpoint is diagnosed as Network TCP, owned by the
   Network Admin, with safe evidence instead of fetch failed.
2. A ServiceNow-style connection uses a true pre-registered confidential
   client, exact Den callback, client_secret_post, PKCE, and no dynamic client
   registration.
3. Test connection initializes MCP revision 2025-06-18, reads the full
   four-tool catalog across two pages, and closes the session without invoking
   a tool.
4. A dynamically injected version mismatch preserves Authorized as the
   highest proven health and names MCP Version / Provider Admin as the first
   failure and owner.
5. Repair and retry reach Catalog Ready, with an explicit warning that this
   does not prove provider operations or mutations.
6. Only then, one bounded synthetic read is denied by provider policy through
   Den's real capability execution path. Den preserves protocol readiness and
   reports provider_policy_denied, phase PROVIDER_AUTHORIZATION, and the
   Provider Admin remediation without returning credentials, session IDs,
   arguments, or provider content.

The superseding Fraimz run passed one flow, eight operational chapters (setup,
six diagnostic steps, and cleanup), 66 assertions, and nine evidence frames
with zero failures or skips. Local evidence is recorded under
evals/results/2026-07-11T21-25-52-239Z/; the repeatable flow itself is
evals/flows/mcp-diagnostics-integration-rehearsal.flow.mjs.

Guided proof viewer

This branch includes a small React viewer so a maintainer can replay the
canonical evidence without starting Den, the mock server, or using
credentials:

pnpm dev:mcp-diagnostics-proof

Open http://127.0.0.1:3334. The viewer walks through all eight chapters,
shows the nine captured frames, explains what each frame proves, and keeps the
approval boundary visible: Agent verified / Jalil not started / parent none.
See the proof viewer README
for the short start and replay guide.

The viewer itself is checked with:

pnpm --filter @openwork/mcp-diagnostics-proof test
pnpm --filter @openwork/mcp-diagnostics-proof typecheck
pnpm --filter @openwork/mcp-diagnostics-proof build

The viewer's own Fraimz run 2026-07-11T22-59-33-547Z passed 1/0/0 with four
user-facing proof frames and 24 assertions, including voice-over coverage.
Manual browser review covered all eight hash-addressable chapters and nine
loaded images, the full-size image dialog, local review persistence/reset,
keyboard navigation, and 720px/390px responsive layouts. A command-grid
overflow found at 390px was fixed and rerun with no horizontal overflow. The
final console had no warnings or errors.
The final flow explicitly asserts that the skip control moves focus without
changing the current hash and that the 390px document has no horizontal
overflow.

That run verifies the proof viewer, not the MCP implementation. The canonical
MCP behavior evidence remains 2026-07-11T21-25-52-239Z.

Recommended human review order: status and scope, Level 1 network ownership,
Level 2 confidential OAuth and catalog-only test, Level 3 fault/repair and
provider denial, then limitations. Record each selected level as verified,
needs changes, or deferred before integrating its source SHA into #2674.

What we found and why it mattered

Finding	User or support impact	Severity at discovery	Resolution in the rehearsal
OAuth callback used the web origin instead of the public Den API origin.	Consent could succeed and return to a 404.	Release blocker	Callback origin is explicit and regression-tested.
A timed-out token/client operation could write after the attempt ended.	The UI could say “failed” while credentials changed later.	Release blocker	Lifecycle generations fence every late persistence boundary.
Post-callback validation could use stale connection state.	Valid consent could immediately look unauthorized.	Release blocker	The callback reloads persisted credentials before MCP validation.
Provider permission 403s and credential 401s were conflated.	Support could tell users to reconnect when the provider administrator actually needed to grant access.	Release blocker	Reauthentication and provider-policy failures now have distinct phases, codes, owners, and actions.
Structured tools/call errors could look like successful execution.	Provider denial or throttling could be hidden behind a healthy transport.	Release blocker	Tool-result errors are normalized as provider authorization or throttling failures while connection health remains truthful.
Connection deletion could race a diagnostic start or leave attempts/events behind.	Deleted connections could keep background work or orphan diagnostic records.	Release blocker	Deletion is transactional, cancels runners, and has race/recovery audit tests.
The convenient OAuth mock path did not prove the realistic pre-registered confidential-client path.	A demo could pass while ServiceNow-style onboarding failed.	Release blocker	The proof uses exact pre-registration, callback replacement, client_secret_post, and asserts no DCR request.
The mock accumulated socket listeners and generated callbacks across repeated runs.	Long review sessions could warn, leak listeners, or exhaust the callback limit.	Review blocker	Listener buildup is removed; one runtime callback replaces the prior value.
Private mock controls were available unless each route remembered its guard.	A fixture could expose fault controls accidentally.	Test-safety blocker	All runtime controls fail closed behind a diagnostics key, including when no key is configured.
Small dialogs and unstable row selection made the proof flaky.	A maintainer could not reliably reach Add, Test, or Diagnose.	Review blocker	The dialog scrolls, connection rows expose stable identity selectors, and the flow cleans stale rows.
“Catalog Ready” could be read as operational readiness.	An admin might assume real provider actions or mutations were proven.	Product clarity blocker	A visible status banner states the exact catalog-only boundary.

Verification performed

• 272 focused tests / 1,443 expectations across structured diagnostics,
  URL and transport safety, request-log redaction, persisted tracing,
  connection testing, real capability execution, Den Web contracts, and the
  enterprise mock.
• Den API, Den Web, and @openwork/types production builds passed.
• Den API and Den Web TypeScript checks passed.
• Existing-database db:push, clean supported db:bootstrap, and schema
  generation with no drift passed.
• The final combined browser journey passed after restarting the hardened mock.
• Upstream GitHub Actions for fc21b086 passed on Linux and macOS, including
  schema, i18n, Helm, and Den builds. Three Vercel preview contexts report
  authorization required for the fork; those are preview-access blockers,
  not code or test failures.
• git diff --check passed.

The focused total is intentionally reported separately from GitHub CI. CI on
the source PRs continues to be the repository-level gate; the rehearsal adds
integration and user-journey evidence.

Remaining limitations and decisions

These are visible release decisions, not hidden claims of completeness:

• The mock is a deterministic synthetic contract. A nonproduction live
  ServiceNow tenant and a nonproduction Microsoft 365 tenant still need
  provider-conformance verification.
• The active authorization URL and runtime fixture controls are process-local.
  A Den restart or instance hop ends an in-flight authorization clearly and
  retryably; seamless cross-instance continuation is not claimed.
• An already-running SDK network promise is bounded by the diagnostic
  lifecycle, but cannot always be forcibly aborted inside the third-party SDK.
  Late persistence is fenced.
• Test connection intentionally proves authorization, initialization, the
  complete bounded catalog, and shutdown. It does not invoke provider tools or
  prove mutation permissions.
• The quick connection test and the production catalog loader still have
  different safety bounds and should be aligned before being described as
  behaviorally identical.
• A raw empty historical migration replay fails before the new MCP migrations;
  the repository's supported fresh db:bootstrap path passes. This is a setup
  constraint, not an MCP schema failure.

Safe promotion plan

This rehearsal should not be merged wholesale into the parent release branch.
Instead:

1. Jalil selects one level, starting with structured errors.
2. Review the visible checkpoint and its focused source PR.
3. Record Jalil-verified, needs changes, or deferred in feat(mcp): stage the diagnostics verification release #2674.
4. Integrate only the accepted source commit into
   feature/mcp-diagnostics-release.
5. Rerun the cumulative focused gate and combined browser journey.
6. Update the parent ledger with the accepted commit, proof, and limitations.
7. Repeat for the next level, then run approved live-provider checks.

Branch topology

All feature branches live only on Jalil's forks. Draft PRs are opened only in
different-ai/openwork and target upstream dev.

GitHub cannot use a branch that exists only on a fork as the literal base of an
upstream-repository PR. Therefore #2669, #2670, and #2672 remain dev-based
source/evidence PRs and point to the parent ledger. Accepted source SHAs will
be integrated into the fork-only parent branch after Jalil's approval.

This PR is a sibling rehearsal based on upstream dev; it is not a substitute
base and does not change the status of the real parent.

Exact administrator-facing Connect behavior

The rehearsal verified two related but distinct UI layers:

1. Connect failure: Den shows a red page alert and an inline row message containing the connection name, a safe explanation of the actual failing source layer, and a diagnostic reference. Example: “Could not connect ‘ServiceNow’: Den resolved the MCP host but could not establish a network connection. Reference: <reference>.” Callback failures after provider consent use a Connection failed browser page with the same safe explanation and reference.
2. Diagnostic investigation: The adjacent Diagnose action opens the live dialog with Highest health, First failure, the remediation message, owner, recommended action, phase timings, safe evidence, and the redacted timeline.

This means the old generic “Failed to connect” behavior is corrected, but the Connect alert itself does not display every structured field and does not automatically open Diagnose. The richer detail is one explicit click away. The rehearsal proves the current behavior; whether to add an inline phase badge or View diagnostics button is a product-review decision for the controlled parent #2674.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
openwork-landing	Ready	Preview, Comment, Open in v0	Jul 11, 2026 11:48pm

[vercel]

@reachjalil is attempting to deploy a commit to the Different AI Team on Vercel.

A member of the Team first needs to authorize it.

[reachjalil]

CI checkpoint — 2026-07-11

Repository checks for b44089c are complete and green:

• Linux and macOS OpenWork tests
• Den API and Den Web builds
• inference build
• schema and migration sync
• i18n audit
• Helm validation
• artifact-version resolution

The three Vercel application preview contexts report Authorization required to deploy. The same fork-preview permission condition appears on the source PRs; it is an external preview authorization gate, not a failed application build. The landing preview passed.

Release status is unchanged: agent-verified yes, verified by Jalil not started, integrated into #2674 no.

[reachjalil]

Published exact rehearsal head fc21b086 with the manager proof viewer and nine canonical evidence frames. Product replay 2026-07-11T21-25-52-239Z passed 8 operational chapters / 66 assertions / 9 screenshots; viewer replay 2026-07-11T22-59-33-547Z passed 24 assertions / 4 screenshots. All GitHub Actions passed. The three remaining red Vercel contexts say authorization required for fork previews and are not code/test failures. Status remains Agent verified; Jalil verification not started; parent #2674 unchanged.