fix(bench): rebuild each round's level set from its runs, not from a snapshot

The round comparison was about to be wrong. levels/campaign-50-r2.json was
copied from levels/campaign-50.json to preserve "the set round 2 played", but
select-50 writes that path in place and several selection experiments ran
between the round and the copy. The file kept 28 of the 50 boards round 2 had
actually played and 46 of round 3's — so the comparison was scoring one round
against another round's levels, and it read as round 3 regressing.

Runs store the whole board and its campaign index, so a round is its own record
and cannot drift. round-set.mjs rebuilds it, the report uses that, and the
snapshot file is gone rather than left lying around looking authoritative.

With the rounds correctly matched, round 3 is not a regression and not an
improvement: back band 2.17 against round 2's 2.39, on a rating that retests at
0.55, which is not enough to separate seventeen levels. The report says that
rather than picking whichever reading flatters the change.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
cabinw
2026-09-14 17:22:31 +08:00
co-authored by Claude Opus 5
parent dcdd4a0ada
commit 4aa0c28cef
9 changed files with 223558 additions and 85313 deletions
+1 -1
View File
@@ -14,7 +14,7 @@ import { layout } from '../src/geometry.js';
import { metrics } from '../src/bench.js';
const args = Object.fromEntries(process.argv.slice(2).map(a => { const [k, ...v] = a.replace(/^--/, '').split('='); return [k, v.length ? v.join('=') : true]; }));
const AGENT = args.agent ?? 'R1 · grok xhigh';
const AGENT = args.agent ?? 'R3 · grok xhigh';
const OUT = resolve(args.out ?? 'docs/review-50.html');
const raw = JSON.parse(readFileSync(resolve('levels/campaign-50.json'), 'utf8'));