Files
HexTrails/docs
cabinwandClaude Opus 5 d123b17a54 docs: regenerate bench reliability report with four-way cross-check
Adds the kimi-k3 and codex-spark batches, so the rotation counters are now
compared across four independent runs of three models from three vendors on
the fixed board: 17 of 18 levels agree exactly, triple for triple.

The clean test-retest (same model, same harness, same board, run twice) also
corrects two earlier readings taken from data that had a board-format change
mixed in. Difficulty ratings reproduce at r = 0.93 rather than 0.63 — the
earlier low figure was the format change, not the metric — so an agent can be
trusted to rank 1000 levels by difficulty even though its absolute scale may
be offset. And the mechanical counters are highly reproducible rather than
deterministic: 0.82-0.92, not the 1.00 the confounded pair suggested.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-14 15:10:06 +08:00
..