fix(bench): rebuild each round's level set from its runs, not from a snapshot

Browse Source

The round comparison was about to be wrong. levels/campaign-50-r2.json was
copied from levels/campaign-50.json to preserve "the set round 2 played", but
select-50 writes that path in place and several selection experiments ran
between the round and the copy. The file kept 28 of the 50 boards round 2 had
actually played and 46 of round 3's — so the comparison was scoring one round
against another round's levels, and it read as round 3 regressing.

Runs store the whole board and its campaign index, so a round is its own record
and cannot drift. round-set.mjs rebuilds it, the report uses that, and the
snapshot file is gone rather than left lying around looking authoritative.

With the rounds correctly matched, round 3 is not a regression and not an
improvement: back band 2.17 against round 2's 2.39, on a rating that retests at
0.55, which is not enough to separate seventeen levels. The report says that
rather than picking whichever reading flatters the change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

...

This commit is contained in:

cabinw

2026-09-14 17:22:31 +08:00

co-authored by Claude Opus 5

parent dcdd4a0ada

commit 4aa0c28cef

9 changed files with 223558 additions and 85313 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

docs/review-50.html

+387 -387

File diff suppressed because one or more lines are too long