Skip to content

feat: render the unset-baseline callout on the PR comment#150

Merged
veksen merged 2 commits into
mainfrom
feat-render-unset-baseline
Jun 16, 2026
Merged

feat: render the unset-baseline callout on the PR comment#150
veksen merged 2 commits into
mainfrom
feat-render-unset-baseline

Conversation

@veksen

@veksen veksen commented Jun 16, 2026

Copy link
Copy Markdown
Member

Renderer half of Site #3297. Closes Site #3312. Part of the unset-baseline surfacing thread (Site #3292 follow-ups, epic Site #3106).

Problem

When a project has no comparison branch configured, the baseline silently falls back — to head on a re-push, or (since #147) the PR base. So hasComparison is true and the PR comment renders counts with no warning, e.g. 0 regressed · 0 improved · 0 new on a PR full of new queries (Site #3292, d2armory-next PR #775). The existing "no baseline found / add a push trigger" copy only fires when there's no baseline run at all — a different condition — so this misconfiguration shipped silently.

Change

Site now emits metadata.baseline on POST /ci/runs (Site #3297, merged). This renders it:

  • site-api.ts — type the baseline field on CiRunMetadata (optional → tolerates an older Site API and a degraded null read; both render nothing).
  • success.md.j2 — a ⚠️ callout when hasComparison and runMetadata.baseline.unset. It names the fallback branch, distinguishes the acute head-vs-head case ("0 new on a PR full of new queries") from a base-branch fallback ("diverges from the dashboard / breaks on non-PR runs"), and surfaces the get_repo_config MCP call to inspect/fix it.

Driven off the Site signal (defined once, Site #3302) rather than re-deriving the rule here.

Scope note

Gated to hasComparison — it targets exactly the gap (a comparison was produced but the baseline is misconfigured). The no-comparison path keeps its existing "add a push trigger" guidance, so the two messages never stack.

Tests

github.test.ts (+4): unset+head-vs-head renders the callout with branch + 0 new + MCP call; base-branch fallback frames the divergence/non-PR risk; configured → no callout; absent/null baseline → no callout. 55/55 reporter tests pass, existing snapshots unchanged, tsc --noEmit clean.

Cross-repo

Producer: Site #3297 (merged) · shared detection Site #3302 · MCP sibling Site #3298 · dashboard sibling Site #3299.

🤖 Generated with Claude Code

When the Site API reports `metadata.baseline.unset` (no comparison branch
configured, or it collapsed to a head-vs-head comparison), the PR comment now
warns — even when a comparison was produced. That's the silent case the
existing "no baseline found" copy misses: an unconfigured baseline falls back to
head (or the PR base), so `hasComparison` is true and the comment renders counts
with no warning, e.g. "0 new" on a PR full of new queries (Site #3292).

The callout names the branch the comparison fell back to, distinguishes the
acute head-vs-head case from a base-branch fallback, and surfaces the
`get_repo_config` MCP call to inspect/fix it. Driven off the Site signal
(`metadata.baseline`) rather than re-deriving the rule; the field is optional so
an older Site API (deploy skew) and a degraded `null` read both render nothing.

Renderer half of Site #3297; closes Site #3312.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Query Doctor Analysis

3 queries analyzed
0 regressed · 0 improved · 0 new · 0 removed

2 pre-existing issues

Using assumed statistics (10000000 rows/table). For better results, sync production stats.

More detail → get_ci_run({ runId: "019ed207-da2d-7965-9d92-72c58c082324" }) · view run · docs

@veksen veksen force-pushed the feat-render-unset-baseline branch 2 times, most recently from 7445384 to 2dfa6e8 Compare June 16, 2026 20:00
Render the callout with GitHub's `> [!WARNING]` alert instead of a manual ⚠️
blockquote, so it shows as the native styled warning box (and drops the
hand-placed emoji). Content unchanged.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@veksen veksen force-pushed the feat-render-unset-baseline branch from 2dfa6e8 to 8a254cd Compare June 16, 2026 20:02
@veksen veksen merged commit a2ecd2a into main Jun 16, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant