Skip to content

feat(search): issue #7 M2 shared preprocessing + CPU reference - #16

Merged
codewarnab merged 5 commits into
mainfrom
feat/issue-7-m2-preprocess
Sep 17, 2026
Merged

codewarnab merged 5 commits into
mainfrom
feat/issue-7-m2-preprocess

Conversation

@codewarnab

Copy link
Copy Markdown
Owner

M2 for Issue #7 (per ISSUE-7-PLAN.md:165-171, contract sections 1-2). No WGSL/engine swap (M3).

What is inside

  • scripts/generate-fold-table.ts + src/fold-table.ts: pinned CaseFolding-16.0.0.txt (URL/date/license header), C+F only (1,557: 1,453 C in 687 ranges + 104 F = 88x1-2 + 16x1-3), S+T excluded, sparse range/delta eager encoding.
  • src/unicode-preprocess.ts: normalizeText() trim toWellFormed (+regex fallback) NFC C+F fold (folded only) NFC u32[]. One path for records + queries, post-fold sizing.
  • src/cpu-reference.ts: substring + fuzzy mirroring WGSL integer formulas (|0/Math.imul), post-fold lengths, two-key sort. uFuzzy quarantined to cpuAlgorithm:'ufuzzy' with negative test.
  • src/hybrid-index.ts: post-fold query gate + tokenCount, QueryTooLongError/onQueryTooLong preserved, unified query:'' on zero-token queries, GPU-failure fallback to parity.
  • Tests: scripts/test-m2-preprocess.ts (93 passed), check-parity-lint.ts, check-m2-bundle.ts.

Gates

  • check:shaders + typecheck + build + test:mock green; unit 93/0; bundle 65,840 B raw / 16,849 B gzip, delta +6,643 B gzip (cap +8,192), total under 20 KB; tr-TR run green. test:browser orchestration defined, not gating (known CI infra debt, M4).

Known notes

  • U+03F4 has C->03B8 in 16.0.0 so C+F folds it (contract T-only label stale; pinned in code/tests).
  • Mark/VS/ZWJ/tatweel-only survive as 1 token per survival rule, so only truly zero-token queries get query:'' (pinned in 8b).
  • TODO.md + package-lock.json left untracked.

Copilot AI lite review requested due to automatic review settings September 17, 2026 08:45
@vercel

vercel Bot commented Sep 17, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
webgpu-fuzzy-search Ignored Ignored Sep 17, 2026 9:32am UTC

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

…afety, CI gates

- shared runtime-guards (clampLimit/nowMs/abortError/throwIfAborted,
  printable-ASCII GPU gate); no unguarded DOM refs
- hybrid-index: items.slice, isDestroyed+fail-closed destroy,
  preferGpu:false forces CPU, measured memory budget, abort-first,
  exact post-fold query gate (1M+ estimate only), GPU legacy-only
  routing (printable-ASCII + <=59 tokens), honest M2 comments
- cpu-reference: wrap-free comparator, shared upper clamp, WORD_BOUNDARY_PREV
- unicode-preprocess: lookbehind-free pair-preserving fallback, captured
  native toWellFormed, ASCII fast path, single-pass tokens, chunked spread,
  typed input errors, uFFFD escape
- fold-table: frozen tables + shared-frozen single/expansion returns
  (zero per-fold alloc, no process-wide mutation)
- webgpu-engine: two-key parity sort, shared clamp, guarded clocks/abort
- cpu-engine: totalMatches counts caseSensitive skips, quarantine docs
- scripts: Node-portable parity lint (comment/string stripping, extended
  bans, legacy allowlist), deterministic fail-closed bundle gate (js+cjs,
  newest-src staleness), generator timeout/input validation/atomic write/
  CRLF-tolerant --check + sha256 logging
- tests: same-mode ufuzzy!=parity proof, direct clamp checks (9000/
  Infinity/2.7), 8193-corpus huge-limit, Infinity->50
- CI: fixed bun --cwd absolute path, wired lint/fold-check/typecheck/
  build/bundle/preprocess/tr-TR/mock gates
- contract: M2 CPU-parity status, GPU legacy note, survival rule,
  U+03F4 C-fold + I/ß matrix corrections, comparator/regex truth,
  deterministic-gzip baseline note (delta +8097/8192, tight headroom)
- packaging: sideEffects true (eager maps), files list excludes maps,
  LONE_SURROGATE_SOURCE + guards + WORD_BOUNDARY_PREV exports,
  countUnicodeCodePoints deprecated, ProfileMismatchError property
ubuntu-latest runners ship no Chrome binary, so test-and-build failed at
the CHROME_BIN guard before running any browser test. Setup stable Chrome
via browser-actions and accept its PATH name + CHROME_PATH fallback.
The installed binary lives outside PATH (.../setup-chrome/chromium/stable),
so PATH lookup missed it. Prefer steps.setup-chrome.outputs.chrome-path
with PATH lookup as fallback (self-hosted / act runs).
bun 1.4.2 rejects 'bun --cwd <dir> run dev -- ...' (prints help, exit 1),
which killed the dev server before readiness. Use (cd ... && bun run dev)
with absolute GITHUB_WORKSPACE paths instead.
@codewarnab
codewarnab merged commit 1310d1a into main Sep 17, 2026
4 checks passed
@codewarnab
codewarnab deleted the feat/issue-7-m2-preprocess branch October 21, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants