Skip to content

test(e2e): steady the debounced-search and script-await tests - #49

Merged
jarrednorrisdev merged 1 commit into
mainfrom
test/e2e-flakes
Oct 10, 2026
Merged

jarrednorrisdev merged 1 commit into
mainfrom
test/e2e-flakes

Conversation

@jarrednorrisdev

@jarrednorrisdev jarrednorrisdev commented Oct 10, 2026 •

Copy link
Copy Markdown
Owner

Steadies the two e2e tests that have failed CI on and off since 8 October. Between them they caused 6 of the recent red runs on main and on PRs. Only test files change.

The other recent red runs were real bugs, since fixed: the "registry is disposed" unit-test failures were fixed by #47, and the phone search-button test failed in all three browsers at once. Failures from before 6 October are in tests that have since been given a paused clock.

1. Cookbook › debounced search: quick key presses run one search

Failed twice in WebKit (run 38045417859, run 37867595813). The Cookbook example debounces a search box. Atom.debounce (from Effect's effect/reactivity) passes a value on only after it has stopped changing for 400 ms:

const queryAtom = Atom.make("");
const debouncedAtom = Atom.debounce(queryAtom, "400 millis");

The test types sch with 50 ms between keys and expects one search:

await page.getByTestId("search").pressSequentially("sch", { delay: 50 });
await expect(page.getByTestId("search-runs")).toHaveText("1");

Bug: the test saw 2 searches (and once only 2 key presses counted). The trace shows typing the three keys took 1.34 s on the loaded CI machine instead of about 150 ms, so more than 400 ms passed between two keys and the debounce fired mid-word. The library behaved correctly; the test depended on wall-clock time.

Fix: pause the page's clock while typing, then let it run. Playwright's page.clock fakes the page's timers, so a slow machine can't push the debounce past its delay. Several tests in demo.test.ts already do this.

+    await page.clock.install();
     await page.goto("/cookbook");
     await page.waitForLoadState("networkidle");
+    await page.clock.pauseAt(Date.now() + 1000);
     await page.getByTestId("search").pressSequentially("sch", { delay: 50 });
     await expect(page.getByTestId("search-keys")).toHaveText("3");
+    await page.clock.resume();

To check the fix, I ran a copy that waits 600 ms between keys, longer than the debounce. The old test fails it ("Expected 1, Received 3"). The new one passes in Chromium and WebKit.

2. Suspense › useAtomResult waits in the script once, then current follows the atom

Failed four times, in WebKit and Firefox (38059263780, 38000309250, 38011590299, 37859559427). The Suspense page's "Awaiting in the script" example has a toggle button that mounts a <Notes> component. That component awaits useAtomResult in its <script> inside a <svelte:boundary>:

<button aria-pressed={mounted} onclick={() => (mounted = !mounted)}>Mount the component</button>
{#if mounted}
  <svelte:boundary>
    <Notes />  <!-- const notes = await useAtomResult(notesAtom); -->
    {#snippet pending()}Pending: the script is waiting{/snippet}
  </svelte:boundary>
{/if}

Bug: after mount.click(), nothing happened for the next 5 s. The button stayed aria-pressed="false", the box still said "Not mounted.", and the pending snippet never appeared. The traces show:

  • the page was hydrated: the layout had set <html data-hydrated> 1.6 s before the click;
  • every other example on the page had settled, with each $effect.pending() at 0;
  • the pointer landed on the button: the screencast shows its hover state;
  • there were no page errors or console errors.

I couldn't reproduce it locally. I tried 60 plain runs, 48 parallel runs, CPU throttled 4–12×, and clicks timed across a 300 ms window around the page's slowest await resolving. Every click mounted the component.

Fix: use the existing setPressed helper (e2e/toggle.ts) for the three mount and unmount clicks. It presses, waits up to 1 s for aria-pressed to change, and presses again if it didn't. The Async atoms test already uses it for the same symptom.

-    await mount.click();
+    await setPressed(mount, true);

Decisions

  • The Suspense fix retries the click rather than finding the cause. A hydrated button that really swallowed a click would be a bug in Svelte or this library, and the retry hides it from CI. The alternative was to leave the test as it was and add a page-level click counter, so the next CI failure would show whether the handler ran. Merged with the retry: the test checks how the script waits, not how clicks are delivered, and the suite already uses the same retry elsewhere.

Tests

  • Both changed tests: 40/40 passed locally (10 repeats each, 8 workers, Chromium and WebKit). Firefox runs only on CI, because headless Firefox doesn't start locally.
  • The stall check described in section 1, in Chromium and WebKit.
Where to look in the diff
  • apps/demo/e2e/cookbook.test.ts: debounced search: quick key presses run one search
  • apps/demo/e2e/features-async.test.ts: useAtomResult waits in the script once, then current follows the atom

🤖 Generated with Claude Code

The debounced search test typed three keys 50 ms apart and expected one
search, but on a loaded CI machine the typing took over a second, so the
400 ms debounce fired between keys. The page's clock is now paused while
typing.

The Suspense script-await test's Mount click has left the toggle off on
CI, with the page hydrated and settled. It now uses setPressed, which
presses again until the toggle sticks.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jarrednorrisdev
jarrednorrisdev merged commit 39cbbc1 into main Oct 10, 2026
16 checks passed
@jarrednorrisdev
jarrednorrisdev deleted the test/e2e-flakes branch October 10, 2026 16:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant