Skip to content

Make global search tuning reproducible - #261

Merged
thomasahle merged 15 commits into
masterfrom
codex/upstream-global-tuning-tools-v2-20260824
Aug 24, 2026
Merged

Make global search tuning reproducible#261
thomasahle merged 15 commits into
masterfrom
codex/upstream-global-tuning-tools-v2-20260824

Conversation

@thomasahle

Copy link
Copy Markdown
Owner

Purpose

Put the complete search-only global-tuning workflow in tools/tune/, so the running campaign can be audited and reproduced from the repository instead of private orchestration files.

This does not change sunfish.py or claim an Elo gain.

What is included

  • a frozen 2:1:1 opponent panel with calibration and source/binary provenance
  • weighted, reproducibly randomized panel support in logistic GP and SPSA
  • the full search-knob inventory, including exact off-boundaries and conditional IID axes
  • a deterministic policy gate for fuel, mate-band, and eventual-widening invariants
  • immutable per-run training protocols binding engines, books, panel, space, settings, and source files
  • resumable campaign orchestration and crash-safe pending-pair journals
  • an honest split between auditing a frozen benchmark manifest and verifying its historical method checkout

The C twin only gains the UCI knobs needed by the shared parameter space; its defaults remain node-identical to Python.

Verification

  • python3 -m unittest tools.tune.logistic_gp.test_logistic_gp tests.test_tuning_tools — 160 passed
  • python3 -m unittest discover -s tools/tune -p 'test_*.py' — 10 passed
  • make -C tools/ctwin test — 247 search probes and 423 move-generation lists, 0 mismatches
  • python3 tools/tune/verify_recovery.py --method-root <frozen-checkout> — verified
  • git diff --check — clean

@thomasahle

Copy link
Copy Markdown
Owner Author

Full C-twin audit found and fixed one stale test assumption.

NULL_CAP_MARGIN is intentionally independent in the lab twin, while Python currently reuses EVAL_ROUGHNESS. The old gate changed only EVAL_ROUGHNESS, so C and Python were no longer testing the same recurrence. The gate now explicitly couples the two values whenever it is checking Python parity.

Results after that correction:

  • defaults: 818 probes / 901 move lists, 0 mismatches
  • depth 7 subset: 235 probes, 0 mismatches
  • altered QS/QS_A/roughness: 779 probes, 0 mismatches
  • QS=0 focused subset: 111 probes, 0 mismatches
  • altered LMR: 818 probes, 0 mismatches
  • table sizes 500 and 50: 913 and 915 probes, 0 mismatches

The complete 27-position QS=0 sweep remains pending on faster hardware: it is retained in make gate, but the Python reference took over six minutes and 1 GB locally. This PR should not merge before that final sweep passes.

@thomasahle
thomasahle merged commit 23e4c05 into master Aug 24, 2026
3 checks passed
@thomasahle

Copy link
Copy Markdown
Owner Author

Added an independent prevalidation audit for the terminal consensus. It now recomputes the support/L1/SHA winner from all 20 canonical lane optima and rejects a tampered selection or evidence record. The combined tuner suite is now 161 tests, all passing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant