cabinwandClaude Opus 5
6af7277179
The round accidentally ran every level twice, which turned into the most useful thing it produced: 45 retest pairs on this exact set. 84% of them replayed the identical action sequence, so the agent is near-deterministic and a second run of a level buys almost nothing — the mechanical metrics retest at r = 1.00 for that reason, not because they survived a hard test. The ratings do not. Difficulty retests at r = 0.55 here, against the 0.93 this project had on record from a smaller sample, and every pair that disagreed had played the same trajectory: same moves, same outcome, different number. The report now prints the reliability it measured rather than quoting the older figure, and says which one it replaces. Averaged by band over ~20 runs, though, the ratings finally track the curve — 2.00 / 2.05 / 2.40 with stuck at 1.21 / 1.15 / 1.74, where round 1 was a flat line. So the rule is not "ratings are useless", it is "ratings are a band statistic, never a per-level rank", and that is what the rules section says. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
264 lines
11 KiB
JSON
264 lines
11 KiB
JSON
{
|
||
"goal": "调出 50 关难度合适、好玩的关卡，交付人工审核",
|
||
"updated": "09-14 16:45",
|
||
"round": 2,
|
||
"definition": {
|
||
"title": "「难度合适且好玩」的操作定义",
|
||
"why": "旋转是免费的、是这个游戏的特色，玩家应该频繁做——所以不该限制旋转次数。该限制的是每一次旋转值不值：转了有没有棋子出来、能不能看出来该转哪、以及转完之后有没有「原来这么简单」的反差。评分里只有 difficulty（复测 r=0.93）和 stuck（r=0.85）可信，rotation / clarity / fun 是噪声（0.32 / 0.19 / 0.56），所以「好玩」全部换成机械代理。 另外，演出本身就是奖励——棋子沿弯路走出去的那几秒是减压点，所以开局必须有得看，而不是只有两枚贴边棋子和 0.3 秒。",
|
||
"rules": [
|
||
{
|
||
"k": "旋转要深",
|
||
"v": "难度来自「需要转很多次」，不是「转错受罚」。旋转数随关号上升，不设上限",
|
||
"how": "求解器最少旋转数"
|
||
},
|
||
{
|
||
"k": "每转都有回报",
|
||
"v": "每一次旋转至少放出 2 枚棋子；无效转动 = 0",
|
||
"how": "deepLevelStats.releasePerTurn / 规则机器人"
|
||
},
|
||
{
|
||
"k": "演出是回报",
|
||
"v": "开局就要有得看：可点 ≥3 枚、其中最好的一条离场演出 ≥0.9s；全关长巡游（≥0.9s）≥3 次",
|
||
"how": "showStats，用真实动画计时"
|
||
},
|
||
{
|
||
"k": "反差感",
|
||
"v": "平均反差 ≥12：棋子在某个朝向看着像噩梦（长路线多弯），转过去就是几步走出（代价 = 格数 + 2×弯数）",
|
||
"how": "六朝向逐棋子代价差"
|
||
},
|
||
{
|
||
"k": "不无脑",
|
||
"v": "猴子 动作÷棋子 ≥3.0（v0.5 全库 1.67 = 乱点就能过）",
|
||
"how": "生成时算"
|
||
},
|
||
{
|
||
"k": "不挫败",
|
||
"v": "stuck ≤2.0；终局任一朝向卡住时，左右转都能开",
|
||
"how": "grok 抽样 + 求解器"
|
||
},
|
||
{
|
||
"k": "不重复",
|
||
"v": "盘面签名互不相同，路线家族分布不偏科",
|
||
"how": "deepLevelKey"
|
||
}
|
||
]
|
||
},
|
||
"targets": {
|
||
"levels": 50,
|
||
"rotationDepth": [
|
||
{
|
||
"band": "L1–L12",
|
||
"rot": "2–3 转",
|
||
"difficulty": "1.5 – 2.0"
|
||
},
|
||
{
|
||
"band": "L13–L32",
|
||
"rot": "3–5 转",
|
||
"difficulty": "2.0 – 2.6"
|
||
},
|
||
{
|
||
"band": "L33–L50",
|
||
"rot": "5–8 转",
|
||
"difficulty": "2.6 – 3.0"
|
||
}
|
||
],
|
||
"gates": {
|
||
"releasePerTurn": "≥2",
|
||
"blindRotations": 0,
|
||
"contrast": "≥12",
|
||
"monkeyAPT": "≥3.0"
|
||
}
|
||
},
|
||
"phases": [
|
||
{
|
||
"id": "P0",
|
||
"title": "判据：先说清楚什么叫「好」",
|
||
"tasks": [
|
||
{
|
||
"id": "T0.1",
|
||
"title": "难度曲线与旋转配额定义",
|
||
"status": "done",
|
||
"note": "50 关分三档，0 转关卡死 10% 上限（用户硬要求）"
|
||
},
|
||
{
|
||
"id": "T0.2",
|
||
"title": "机械验收门槛",
|
||
"status": "done",
|
||
"note": "盲转=0、转错=0、终局可走朝向≥2 —— 前两项已由改锁结构达成"
|
||
},
|
||
{
|
||
"id": "T0.3",
|
||
"title": "「好玩」的机械代理指标",
|
||
"status": "done",
|
||
"note": "猴子 APT ≥3.0。fun 评分复测 r=0.56 不可用，改用「乱点过不了」当代理"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"id": "P1",
|
||
"title": "产能：解除旋转上限，把反差感做进筛选",
|
||
"tasks": [
|
||
{
|
||
"id": "T1.1",
|
||
"title": "growDeepLevel 扩展到 6 个朝向",
|
||
"status": "done",
|
||
"note": "现在只在朝向 0/1/2 插棋子，所以最少旋转恒等于 3 —— 这是「太简单」的根因"
|
||
},
|
||
{
|
||
"id": "T1.2",
|
||
"title": "解除 rotations===3 的硬限制，允许 4/5/6+",
|
||
"status": "done"
|
||
},
|
||
{
|
||
"id": "T1.3",
|
||
"title": "反差感指标进 deepLevelStats 并作门槛",
|
||
"status": "done",
|
||
"note": "contrastStats 已进 deepLevelStats 并作门槛；实测 weave 22–33、switchback 29、spiral 18–24、orbits 12–17，petals 仅 2.0–5.7、crossing 3.8–5.5"
|
||
},
|
||
{
|
||
"id": "T1.4",
|
||
"title": "每转释放 ≥2 枚的门槛",
|
||
"status": "done",
|
||
"note": "改为按朝向配额插棋（每个朝向 ≥2 枚在等），取代原来的随机比例——随机比例会留下空转的朝向"
|
||
},
|
||
{
|
||
"id": "T1.5",
|
||
"title": "扩产到 ≥120 候选并去重、家族均衡",
|
||
"status": "done",
|
||
"note": "扫了 480 + 240 个种子，得 684 个候选；过两道新闸门后 228 关。扩产的目的是付得起闸门的代价，不是数量本身"
|
||
},
|
||
{
|
||
"id": "T1.6",
|
||
"title": "终局三枚互锁（hero+gate+tail）",
|
||
"status": "blocked",
|
||
"note": "仍是深关卡的瓶颈：684 个候选里深度≥4 的只有 67 个，过闸门后剩 16 个——后档只能做到平均 4.1 转，是供给限制不是设计选择"
|
||
},
|
||
{
|
||
"id": "T1.7",
|
||
"title": "开局展示位（快速爽到）",
|
||
"status": "done",
|
||
"note": "实测整个构造的开局中位只有 2 枚可点、最佳演出 0.57s——锁链开局全锁死。在构造里预留 2 个开局位（立即可点且路线长到值得看），实测开局可点 2→3 枚、最佳演出 0.72s→1.12s"
|
||
},
|
||
{
|
||
"id": "T1.8",
|
||
"title": "离场演出封顶",
|
||
"status": "done",
|
||
"note": "计时原本线性无上限，最长一次 17.4s，正式关 18.8% 超过 2s。拐点 2.4s 以下完全不变，以上加速逼近 3.2s 上限。labs/exit-timing 可同步对比"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"id": "P2",
|
||
"title": "标定：把机械指标映射到真实难度",
|
||
"tasks": [
|
||
{
|
||
"id": "T2.1",
|
||
"title": "grok 抽样跑候选池，收集 difficulty / stuck",
|
||
"status": "done",
|
||
"note": "R1：grok-4.6 xhigh 全量跑 50 关，收齐 difficulty / stuck / 机械指标与自由文本抱怨"
|
||
},
|
||
{
|
||
"id": "T2.2",
|
||
"title": "拟合 difficulty ~ 机械特征",
|
||
"status": "blocked",
|
||
"note": "做不成，且不是样本量问题：27 关里 26 关 difficulty 都打 2 分，响应几乎无方差，回归的是常数。根因在 bench 本身——文字盘面把每颗棋子的路线、挡路者、紧贴者直接列成清单，agent 不必用眼睛顺线走，而那正是这游戏的难点。agent 原话：「挡路名单已经写全，不用顺着格子数」。为解释它另建的 legibility 指标与评分相关 -0.02，同样无效，已记录为负结果不采用"
|
||
},
|
||
{
|
||
"id": "T2.3",
|
||
"title": "用模型给全部候选打分并按曲线选 50 关",
|
||
"status": "done",
|
||
"note": "select-50 按机械特征打分并均匀铺开，家族上限 36%。权重是设计假设而非拟合结果（见 T2.2），由人工审核裁定"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"id": "P3",
|
||
"title": "迭代：跑几轮把曲线调平",
|
||
"tasks": [
|
||
{
|
||
"id": "T3.1",
|
||
"title": "第 1 轮：grok 全量跑 50 关",
|
||
"status": "done",
|
||
"note": "50 关全部跑完；机械面干净：空点 0、盲转 ~0.3/关、全部通关。总体读数：整批偏易一档（2 分 vs 刚好的 3 分）"
|
||
},
|
||
{
|
||
"id": "T3.2",
|
||
"title": "按实测曲线调整选关，重选",
|
||
"status": "done",
|
||
"note": "连续空转设闸门（≤1）+ 难度曲线改为按旋转深度配额。同一规则机器人对照：盲转 0.24→0.12，最长连续盲转 4→1，决策数不变"
|
||
},
|
||
{
|
||
"id": "T3.3",
|
||
"title": "第 2 轮验证",
|
||
"status": "done",
|
||
"note": "R2：99 局覆盖 50 关，全部通关，空点 0。难度按档 2.00 / 2.05 / 2.40，无解感 1.21 / 1.15 / 1.74 —— 评分第一次出现梯度"
|
||
},
|
||
{
|
||
"id": "T3.4",
|
||
"title": "收敛判定（难度曲线单调、stuck 达标）",
|
||
"status": "doing",
|
||
"note": "机械面已收敛：空点 0、不可解态 0、连续空转≤1、终局朝向≥3。难度未达标：后档 2.40 vs 目标 3。瓶颈是深关卡供给（见 T1.6），正在扫第二批种子。注意逐关评分复测 r=0.55 且同轨迹会打出不同分，只有按档≈20 局才可读"
|
||
}
|
||
]
|
||
},
|
||
{
|
||
"id": "P4",
|
||
"title": "交付",
|
||
"tasks": [
|
||
{
|
||
"id": "T4.1",
|
||
"title": "50 关关卡集",
|
||
"status": "done",
|
||
"note": "levels/campaign-50.json + src/levels-c50.js，已注册为可玩版本 c50，198 项测试全绿"
|
||
},
|
||
{
|
||
"id": "T4.2",
|
||
"title": "人工审核页（可直接试玩每一关）",
|
||
"status": "done",
|
||
"note": "docs/review-50.html：50 张开局盘面缩略图 + 机械指标 + agent 实测 + 试玩直达；并标注「哪些数字能信」，把感知类问题（路线长度、紧贴）显式交给人眼"
|
||
},
|
||
{
|
||
"id": "T4.3",
|
||
"title": "后续关卡制作流程的优化总结",
|
||
"status": "done",
|
||
"note": "docs/level-process.html：两轮对照、复测可信度表（本批现算）、五条规则各带证据、被否掉的做法清单"
|
||
}
|
||
]
|
||
}
|
||
],
|
||
"rounds": [],
|
||
"log": [
|
||
{
|
||
"at": "起点",
|
||
"text": "已完成前置工作：擦碰漏报修复（113/581 例）、终局单出口门槛、锁结构改为双向可开 —— 规则机器人盲转/转错/多转已全部归零"
|
||
},
|
||
{
|
||
"at": "方向修正",
|
||
"text": "旋转配额的思路被推翻：旋转免费且是核心特色，不该限次数，该限的是每次旋转的回报与可读性。实测发现 17 关最少旋转全部恒为 3（生成器写死），且 petals/crossing 两个家族几乎没有反差感（平均 2.0–5.7 对比 weave 的 22–33）"
|
||
},
|
||
{
|
||
"at": "P1 进展",
|
||
"text": "旋转深度打开：生成分布从「全部恒为3」变成 {3,4,5,6,7,8,9}。两次结构性猜测失败并已撤回（插入端多朝向出口过滤实测零效果）。深关卡的瓶颈定位为终局三枚互锁，改用扩大扫描量应对"
|
||
},
|
||
{
|
||
"at": "演出",
|
||
"text": "离场动画封顶定稿（拐点 2.4s / 上限 3.2s），最长 17.4s→3.5s；拐点以下手感不变。同时发现开局演出是结构性缺陷并在构造里修掉"
|
||
},
|
||
{
|
||
"at": "R1 结论",
|
||
"text": "难度评分在这批关卡上无区分度（26/27 都是 2 分），原因是文字 bench 把判断白送给了 agent。结论：agent 轮次用于抓缺陷（死局、盲转、空点、演出），难度排序只能靠机械假设 + 人工审核。这条改变了后续流程的设计"
|
||
},
|
||
{
|
||
"at": "未擅自改动",
|
||
"text": "agent 33 条抱怨里两大主题是路线太长（68 格 27 弯）与紧贴易看漏。但已上线的 v0.5 在这两项上更严重（最长 88 格 vs 84 格），属于游戏本来的性格；且文字盘面会同时夸大和缩小感知问题。因此只在审核页标出，不擅自设闸门"
|
||
},
|
||
{
|
||
"at": "R2 依据",
|
||
"text": "后段 14 条抱怨里两类：路线太长（感知，已对照 v0.5 基线否掉）与「转了没反应」（机械，可设闸门）。只改后者：endgameMinOpen 2→3、doubleDead 无闸门→≤2。两条合计只留 16% 候选，所以先扩池"
|
||
},
|
||
{
|
||
"at": "R2 结果",
|
||
"text": "难度评分按档 2.00/2.05/2.40（R1 是三档全平），无解感 1.21/1.15/1.74。意外把每关跑了两次，得到 45 对复测：84% 重放了完全相同的动作序列，机械指标 r=1.00，难度评分 r=0.55——且评分分歧的那几对全是同轨迹。结论：agent 近乎确定性，同一关不必跑两遍"
|
||
}
|
||
]
|
||
} |