fix(bench): rebuild each round's level set from its runs, not from a snapshot
The round comparison was about to be wrong. levels/campaign-50-r2.json was copied from levels/campaign-50.json to preserve "the set round 2 played", but select-50 writes that path in place and several selection experiments ran between the round and the copy. The file kept 28 of the 50 boards round 2 had actually played and 46 of round 3's — so the comparison was scoring one round against another round's levels, and it read as round 3 regressing. Runs store the whole board and its campaign index, so a round is its own record and cannot drift. round-set.mjs rebuilds it, the report uses that, and the snapshot file is gone rather than left lying around looking authoritative. With the rounds correctly matched, round 3 is not a regression and not an improvement: back band 2.17 against round 2's 2.39, on a rating that retests at 0.55, which is not enough to separate seventeen levels. The report says that rather than picking whichever reading flatters the change. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -110,7 +110,7 @@ footer{margin-top:56px;padding-top:20px;border-top:1px solid var(--rule);color:v
|
||||
<div class="tile ok"><div class="k">候选关卡</div><div class="v">50 <span style="font-size:14px;color:var(--ink3)">/ 50</span></div><div class="n">当前关卡集</div></div>
|
||||
<div class="tile ok"><div class="k">盲转 / 关</div><div class="v">0.00</div><div class="n">目标 0</div></div>
|
||||
<div class="tile ok"><div class="k">转错方向 / 关</div><div class="v">0.00</div><div class="n">目标 0</div></div>
|
||||
<div class="tile ok"><div class="k">猴子 动作÷棋子</div><div class="v">6.43</div><div class="n">目标 ≥3.0(不无脑)</div></div>
|
||||
<div class="tile ok"><div class="k">猴子 动作÷棋子</div><div class="v">6.11</div><div class="n">目标 ≥3.0(不无脑)</div></div>
|
||||
</div>
|
||||
|
||||
<div class="scroll"><table>
|
||||
@@ -118,12 +118,12 @@ footer{margin-top:56px;padding-top:20px;border-top:1px solid var(--rule);color:v
|
||||
<tbody>
|
||||
<tr><td>1 转</td><td class="n">5</td><td class="sub">浅</td></tr>
|
||||
<tr><td>2 转</td><td class="n">5</td><td class="sub">浅</td></tr>
|
||||
<tr><td>3 转</td><td class="n">24</td><td class="sub">浅</td></tr>
|
||||
<tr><td>4 转</td><td class="n">12</td><td class="sub">中</td></tr>
|
||||
<tr><td>3 转</td><td class="n">26</td><td class="sub">浅</td></tr>
|
||||
<tr><td>4 转</td><td class="n">10</td><td class="sub">中</td></tr>
|
||||
<tr><td>5 转</td><td class="n">4</td><td class="sub">中</td></tr>
|
||||
|
||||
</tbody></table></div>
|
||||
<p class="sub">路线家族分布:v05 ×10 · spiral ×2 · switchback ×12 · weave ×18 · orbits ×8 终局可走朝向分布:{"3":44,"4":5,"5":1}</p>
|
||||
<p class="sub">路线家族分布:v05 ×10 · spiral ×5 · switchback ×13 · orbits ×9 · weave ×13 终局可走朝向分布:{"3":44,"4":6}</p>
|
||||
|
||||
<h2><span class="num">B</span>什么叫「难度合适且好玩」</h2>
|
||||
<div class="callout"><p class="sub">旋转是免费的、是这个游戏的特色,玩家应该频繁做——所以不该限制旋转次数。该限制的是每一次旋转值不值:转了有没有棋子出来、能不能看出来该转哪、以及转完之后有没有「原来这么简单」的反差。评分里只有 difficulty(复测 r=0.93)和 stuck(r=0.85)可信,rotation / clarity / fun 是噪声(0.32 / 0.19 / 0.56),所以「好玩」全部换成机械代理。 另外,演出本身就是奖励——棋子沿弯路走出去的那几秒是减压点,所以开局必须有得看,而不是只有两枚贴边棋子和 0.3 秒。</p></div>
|
||||
@@ -162,8 +162,8 @@ footer{margin-top:56px;padding-top:20px;border-top:1px solid var(--rule);color:v
|
||||
<li class="done"><div class="t"><span class="tid">T1.2</span><span class="tt">解除 rotations===3 的硬限制,允许 4/5/6+</span><span class="tag good">完成</span></div></li>
|
||||
<li class="done"><div class="t"><span class="tid">T1.3</span><span class="tt">反差感指标进 deepLevelStats 并作门槛</span><span class="tag good">完成</span></div><div class="note">contrastStats 已进 deepLevelStats 并作门槛;实测 weave 22–33、switchback 29、spiral 18–24、orbits 12–17,petals 仅 2.0–5.7、crossing 3.8–5.5</div></li>
|
||||
<li class="done"><div class="t"><span class="tid">T1.4</span><span class="tt">每转释放 ≥2 枚的门槛</span><span class="tag good">完成</span></div><div class="note">改为按朝向配额插棋(每个朝向 ≥2 枚在等),取代原来的随机比例——随机比例会留下空转的朝向</div></li>
|
||||
<li class="done"><div class="t"><span class="tid">T1.5</span><span class="tt">扩产到 ≥120 候选并去重、家族均衡</span><span class="tag good">完成</span></div><div class="note">扫了 480 + 240 个种子,得 684 个候选;过两道新闸门后 228 关。扩产的目的是付得起闸门的代价,不是数量本身</div></li>
|
||||
<li class="blocked"><div class="t"><span class="tid">T1.6</span><span class="tt">终局三枚互锁(hero+gate+tail)</span><span class="tag bad">受阻</span></div><div class="note">仍是深关卡的瓶颈:684 个候选里深度≥4 的只有 67 个,过闸门后剩 16 个——后档只能做到平均 4.1 转,是供给限制不是设计选择</div></li>
|
||||
<li class="done"><div class="t"><span class="tid">T1.5</span><span class="tt">扩产到 ≥120 候选并去重、家族均衡</span><span class="tag good">完成</span></div><div class="note">两批共 720 个种子 → 1113 个候选 → 过闸门 369 关(原 228)。家族 weave 262 / switchback 58 / orbits 39 / spiral 10</div></li>
|
||||
<li class="blocked"><div class="t"><span class="tid">T1.6</span><span class="tt">终局三枚互锁(hero+gate+tail)</span><span class="tag bad">受阻</span></div><div class="note">量化了:扫 1113 个候选,过闸门后深度≥4 的只有 23 个(2%)。第二次扩池把种子数翻倍,深关卡只从 16 涨到 23 —— 难度上限是构造的供给瓶颈,不是筛选问题。后档平均 4.0 转就是这个供给能支撑的上限</div></li>
|
||||
<li class="done"><div class="t"><span class="tid">T1.7</span><span class="tt">开局展示位(快速爽到)</span><span class="tag good">完成</span></div><div class="note">实测整个构造的开局中位只有 2 枚可点、最佳演出 0.57s——锁链开局全锁死。在构造里预留 2 个开局位(立即可点且路线长到值得看),实测开局可点 2→3 枚、最佳演出 0.72s→1.12s</div></li>
|
||||
<li class="done"><div class="t"><span class="tid">T1.8</span><span class="tt">离场演出封顶</span><span class="tag good">完成</span></div><div class="note">计时原本线性无上限,最长一次 17.4s,正式关 18.8% 超过 2s。拐点 2.4s 以下完全不变,以上加速逼近 3.2s 上限。labs/exit-timing 可同步对比</div></li>
|
||||
</ul>
|
||||
@@ -204,7 +204,8 @@ footer{margin-top:56px;padding-top:20px;border-top:1px solid var(--rule);color:v
|
||||
<div class="scroll"><table>
|
||||
<thead><tr><th>批次</th><th class="n">关数</th><th class="n">difficulty</th><th class="n">stuck</th><th class="n">盲转/关</th><th class="n">多转/关</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>R2 · grok xhigh</td><td class="n">99</td><td class="n">2.16</td><td class="n">1.37</td><td class="n">0.12</td><td class="n">0.24</td></tr>
|
||||
<tr><td>R2 · grok xhigh</td><td class="n">100</td><td class="n">2.16</td><td class="n">1.37</td><td class="n">0.12</td><td class="n">0.24</td></tr>
|
||||
<tr><td>R3 · grok xhigh</td><td class="n">50</td><td class="n">2.08</td><td class="n">1.32</td><td class="n">0.14</td><td class="n">0.26</td></tr>
|
||||
<tr><td>R1 · grok xhigh</td><td class="n">50</td><td class="n">2.10</td><td class="n">1.44</td><td class="n">0.24</td><td class="n">0.38</td></tr>
|
||||
<tr><td>Codex CLI · gpt-6-astra low</td><td class="n">18</td><td class="n">2.44</td><td class="n">1.44</td><td class="n">1.17</td><td class="n">1.22</td></tr>
|
||||
<tr><td>Codex CLI · spark xhigh</td><td class="n">18</td><td class="n">3.11</td><td class="n">2.28</td><td class="n">1.17</td><td class="n">1.17</td></tr>
|
||||
@@ -220,6 +221,7 @@ footer{margin-top:56px;padding-top:20px;border-top:1px solid var(--rule);color:v
|
||||
|
||||
<h2><span class="num">F</span>日志</h2>
|
||||
<ul class="log">
|
||||
<li><b>供给瓶颈</b>扩池两轮共 720 种子、1113 候选,深度≥4 的过闸门后只有 23 个(2%)。种子翻倍只让深关卡从 16 涨到 23,说明 growDeepLevel 难以造出需要四转以上的关卡——这是后档难度的硬上限</li>
|
||||
<li><b>R2 结果</b>难度评分按档 2.00/2.05/2.40(R1 是三档全平),无解感 1.21/1.15/1.74。意外把每关跑了两次,得到 45 对复测:84% 重放了完全相同的动作序列,机械指标 r=1.00,难度评分 r=0.55——且评分分歧的那几对全是同轨迹。结论:agent 近乎确定性,同一关不必跑两遍</li>
|
||||
<li><b>R2 依据</b>后段 14 条抱怨里两类:路线太长(感知,已对照 v0.5 基线否掉)与「转了没反应」(机械,可设闸门)。只改后者:endgameMinOpen 2→3、doubleDead 无闸门→≤2。两条合计只留 16% 候选,所以先扩池</li>
|
||||
<li><b>未擅自改动</b>agent 33 条抱怨里两大主题是路线太长(68 格 27 弯)与紧贴易看漏。但已上线的 v0.5 在这两项上更严重(最长 88 格 vs 84 格),属于游戏本来的性格;且文字盘面会同时夸大和缩小感知问题。因此只在审核页标出,不擅自设闸门</li>
|
||||
|
||||
Reference in New Issue
Block a user