Skip to content

Stop tests leaking UserDefaults suite domains#878

Open
robinebers wants to merge 1 commit into
mainfrom
fix/test-userdefaults-suite-leak
Open

Stop tests leaking UserDefaults suite domains#878
robinebers wants to merge 1 commit into
mainfrom
fix/test-userdefaults-suite-leak

Conversation

@robinebers

@robinebers robinebers commented Jul 8, 2026

Copy link
Copy Markdown
Owner

TL;DR

The test suite created unique UserDefaults(suiteName:) domains but usually cleared them only before a test, never afterward, so repeated swift test runs accumulated thousands of test preference plists in ~/Library/Preferences. This routes every leaking scratch suite through one shared helper that removes its data in teardown. Test infrastructure only — no app behavior change.

What was happening

  • Test helpers built suites with fresh UUIDs, called removePersistentDomain once at the top for a clean slate, and returned them.
  • Nothing removed those domains when the test finished, so each run left newly written preferences behind.
  • A few files already used a correct defer { removePersistentDomain } pattern; those remain unchanged.

What this changes

  • Adds XCTestCase.makeScratchDefaults(suiteName:) in TestSupport.swift. It pre-clears the suite and registers an addTeardownBlock that removes its persistent domain after the test.
  • Routes all 28 leaking call sites across 27 test files through that helper, including the newer LayoutBootstrapTests and LayoutPersistenceTests added after this branch was first created.
  • Keeps the existing XCTest class hierarchy unchanged and continues returning plain UserDefaults instances to test bodies.

Heads-up

  • macOS 27 caveat: on macOS ≤ 26 (including the macos-26 CI runner), removePersistentDomain removes the suite plist. On macOS 27 beta, cfprefsd recreates an empty 42-byte dictionary for touched suites when the test process exits. The teardown still removes all test data, but a post-run filesystem cleanup is required to remove those empty shells.
  • A complete one-time cleanup must include the non-OpenUsageTests prefixes used by Copilot, notification, and snapshot-cache tests:
find "$HOME/Library/Preferences" -maxdepth 1 -type f \( \
  -name 'OpenUsageTests*.plist' -o \
  -name 'CopilotProviderTests-*.plist' -o \
  -name 'WidgetDataStoreNotificationTests*.plist' -o \
  -name 'providerSnapshotCache.test.*.plist' \
\) -delete

Tests

  • swift test --filter 'Layout(Bootstrap|Persistence)Tests' — 5 passed.
  • swift test — 953 passed, 3 skipped, 0 failures.
  • Verified from a purged clean slate on macOS 27: the full suite created 216 plist shells, all 42-byte empty dictionaries; zero matching files retained test data.
  • Removed the pre-existing 5,623-file local backlog and the post-verification empty shells; both matching filesystem files and defaults domains were zero afterward.

@github-actions github-actions Bot added the tests label Jul 8, 2026
@robinebers robinebers added the gate-passed Passed the automated PR gatekeeper checks (gatekeeper skips it on later runs) label Jul 8, 2026 — with Cursor
@robinebers

Copy link
Copy Markdown
Owner Author

I found one issue that should be fixed before merging.

This covered every leaking test when the branch was created, but two layout test groups were added to main afterward. Their temporary preferences still get cleared only before the test, not after it. That means a full test run would continue leaving those two sets of saved data behind, while the PR says it stops the leak everywhere.

Please update the branch from main and route both newer layout helpers through the shared cleanup helper as well. The shared helper and the rest of this cleanup look sound.

Every `make…Defaults` helper across the test suite created a fresh
`UserDefaults(suiteName:)` with a UUID-embedded name, cleared it once at
the top for a clean slate, and returned it — with nothing removing the
suite afterward. Each run left a permanent domain behind, accumulating
tens of thousands of `OpenUsageTests.*` domains in ~/Library/Preferences.

Route every scratch suite through a single `XCTestCase.makeScratchDefaults`
helper that keeps the clean-slate pre-clear and also registers an
`addTeardownBlock` to remove the persistent domain once the test finishes.
This mirrors the defer-based pattern the correct helpers already used, in
one DRY place, and drops ~60 lines of duplicated boilerplate.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@robinebers robinebers force-pushed the fix/test-userdefaults-suite-leak branch from 826a686 to c08d357 Compare July 8, 2026 10:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gate-passed Passed the automated PR gatekeeper checks (gatekeeper skips it on later runs) tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant