toshiocn/HexTrails
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(bench): fit difficulty from a played round, and report when it cannot

Browse Source 
refit-weights regresses the weights select-50 scores with against what agents
actually experienced, so a round of play is a flag on the next selection rather
than an edit. Ridge, because seven collinear features on fifty levels otherwise
hand back large coefficients that cancel.

Most of the work here is in refusing to produce a number. The response averages
the difficulty rating with two continuous measures of the same thing, and a
component with no spread is dropped by name instead of being silently averaged
in as zeros — which is what the first version did, reporting R² = 0.75 for a
fit that never saw a rating. When the rating is flat the script now says so and
says what the remaining fit is really measuring, which is board size.

legibility() asks what it costs to know what you may tap: how far the eye must
follow the cheapest free piece, and how many pieces are held by somebody their
line never touches. It was built to explain why ratings came back flat. It does
not — it correlates -0.02 with them — so it stays a measurement, printed on the
review sheet beside the longest route on the board, and gates nothing.

prototype-d5 takes --seedFrom so a wide sweep shards across cores.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
cabinw
2026-09-14 17:03:43 +08:00
co-authored by Claude Opus 5
parent b35ea201db
commit d374c208f0
10 changed files with 3886 additions and 287 deletions
Download Patch File Download Diff File Expand all files Collapse all files
bench/board.json
+249 -237
View File
@@ -1,240 +1,252 @@
{
"goal": "调出 50 关难度合适、好玩的关卡，交付人工审核",
"updated": "09-14 16:45",
"round": 0,
"definition": {
"title": "「难度合适且好玩」的操作定义",
"why": "旋转是免费的、是这个游戏的特色，玩家应该频繁做——所以不该限制旋转次数。该限制的是每一次旋转值不值：转了有没有棋子出来、能不能看出来该转哪、以及转完之后有没有「原来这么简单」的反差。评分里只有 difficulty（复测 r=0.93）和 stuck（r=0.85）可信，rotation / clarity / fun 是噪声（0.32 / 0.19 / 0.56），所以「好玩」全部换成机械代理。 另外，演出本身就是奖励——棋子沿弯路走出去的那几秒是减压点，所以开局必须有得看，而不是只有两枚贴边棋子和 0.3 秒。",
"rules": [
{
"k": "旋转要深",
"v": "难度来自「需要转很多次」，不是「转错受罚」。旋转数随关号上升，不设上限",
"how": "求解器最少旋转数"
},
{
"k": "每转都有回报",
"v": "每一次旋转至少放出 2 枚棋子；无效转动 = 0",
"how": "deepLevelStats.releasePerTurn / 规则机器人"
},
{
"k": "演出是回报",
"v": "开局就要有得看：可点 ≥3 枚、其中最好的一条离场演出 ≥0.9s；全关长巡游（≥0.9s）≥3 次",
"how": "showStats，用真实动画计时"
},
{
"k": "反差感",
"v": "平均反差 ≥12：棋子在某个朝向看着像噩梦（长路线多弯），转过去就是几步走出（代价 = 格数 + 2×弯数）",
"how": "六朝向逐棋子代价差"
},
{
"k": "不无脑",
"v": "猴子 动作÷棋子 ≥3.0（v0.5 全库 1.67 = 乱点就能过）",
"how": "生成时算"
},
{
"k": "不挫败",
"v": "stuck ≤2.0；终局任一朝向卡住时，左右转都能开",
"how": "grok 抽样 + 求解器"
},
{
"k": "不重复",
"v": "盘面签名互不相同，路线家族分布不偏科",
"how": "deepLevelKey"
}
]
},
"targets": {
"levels": 50,
"rotationDepth": [
{
"band": "L1–L12",
"rot": "2–3 转",
"difficulty": "1.5 – 2.0"
},
{
"band": "L13–L32",
"rot": "3–5 转",
"difficulty": "2.0 – 2.6"
},
{
"band": "L33–L50",
"rot": "5–8 转",
"difficulty": "2.6 – 3.0"
}
],
"gates": {
"releasePerTurn": "≥2",
"blindRotations": 0,
"contrast": "≥12",
"monkeyAPT": "≥3.0"
}
},
"phases": [
{
"id": "P0",
"title": "判据：先说清楚什么叫「好」",
"tasks": [
{
"id": "T0.1",
"title": "难度曲线与旋转配额定义",
"status": "done",
"note": "50 关分三档，0 转关卡死 10% 上限（用户硬要求）"
},
{
"id": "T0.2",
"title": "机械验收门槛",
"status": "done",
"note": "盲转=0、转错=0、终局可走朝向≥2 —— 前两项已由改锁结构达成"
},
{
"id": "T0.3",
"title": "「好玩」的机械代理指标",
"status": "done",
"note": "猴子 APT ≥3.0。fun 评分复测 r=0.56 不可用，改用「乱点过不了」当代理"
}
]
},
{
"id": "P1",
"title": "产能：解除旋转上限，把反差感做进筛选",
"tasks": [
{
"id": "T1.1",
"title": "growDeepLevel 扩展到 6 个朝向",
"status": "done",
"note": "现在只在朝向 0/1/2 插棋子，所以最少旋转恒等于 3 —— 这是「太简单」的根因"
},
{
"id": "T1.2",
"title": "解除 rotations===3 的硬限制，允许 4/5/6+",
"status": "done"
},
{
"id": "T1.3",
"title": "反差感指标进 deepLevelStats 并作门槛",
"status": "done",
"note": "contrastStats 已进 deepLevelStats 并作门槛；实测 weave 22–33、switchback 29、spiral 18–24、orbits 12–17，petals 仅 2.0–5.7、crossing 3.8–5.5"
},
{
"id": "T1.4",
"title": "每转释放 ≥2 枚的门槛",
"status": "done",
"note": "改为按朝向配额插棋（每个朝向 ≥2 枚在等），取代原来的随机比例——随机比例会留下空转的朝向"
},
{
"id": "T1.5",
"title": "扩产到 ≥120 候选并去重、家族均衡",
"status": "doing",
"note": "候选池已达 235 关（深度 {3:165,4:46,5:22,7:2}），正用含开局展示位的构造重扫"
},
{
"id": "T1.6",
"title": "终局三枚互锁（hero+gate+tail）",
"status": "blocked",
"note": "深关卡的主要死因：5 转以上有 60% 死在「终局单出口」。实测终局是锁链自己的三枚棋子在 6 个朝向里堵死 5 个。双向释放约束只覆盖了 hero/tail 两枚，三枚组合覆盖不到——这是可达状态的性质，构造时难保证。当前对策是提高扫描量靠筛选拿到，不是修构造"
},
{
"id": "T1.7",
"title": "开局展示位（快速爽到）",
"status": "done",
"note": "实测整个构造的开局中位只有 2 枚可点、最佳演出 0.57s——锁链开局全锁死。在构造里预留 2 个开局位（立即可点且路线长到值得看），实测开局可点 2→3 枚、最佳演出 0.72s→1.12s"
},
{
"id": "T1.8",
"title": "离场演出封顶",
"status": "done",
"note": "计时原本线性无上限，最长一次 17.4s，正式关 18.8% 超过 2s。拐点 2.4s 以下完全不变，以上加速逼近 3.2s 上限。labs/exit-timing 可同步对比"
}
]
},
{
"id": "P2",
"title": "标定：把机械指标映射到真实难度",
"tasks": [
{
"id": "T2.1",
"title": "grok 抽样跑候选池，收集 difficulty / stuck",
"status": "todo",
"note": "分层抽样，覆盖各旋转档与各家族"
},
{
"id": "T2.2",
"title": "拟合 difficulty ~ 机械特征",
"status": "todo",
"note": "候选特征：旋转数、锁住率、猴子 APT、棋子数、路线长度、终局路口数"
},
{
"id": "T2.3",
"title": "用模型给全部候选打分并按曲线选 50 关",
"status": "todo"
}
]
},
{
"id": "P3",
"title": "迭代：跑几轮把曲线调平",
"tasks": [
{
"id": "T3.1",
"title": "第 1 轮：grok 全量跑 50 关",
"status": "todo"
},
{
"id": "T3.2",
"title": "按实测曲线调整选关，重选",
"status": "todo"
},
{
"id": "T3.3",
"title": "第 2 轮验证",
"status": "todo"
},
{
"id": "T3.4",
"title": "收敛判定（难度曲线单调、stuck 达标）",
"status": "todo"
}
]
},
{
"id": "P4",
"title": "交付",
"tasks": [
{
"id": "T4.1",
"title": "50 关关卡集",
"status": "todo"
},
{
"id": "T4.2",
"title": "人工审核页（可直接试玩每一关）",
"status": "todo"
},
{
"id": "T4.3",
"title": "后续关卡制作流程的优化总结",
"status": "todo"
}
]
}
],
"rounds": [],
"log": [
{
"at": "起点",
"text": "已完成前置工作：擦碰漏报修复（113/581 例）、终局单出口门槛、锁结构改为双向可开 —— 规则机器人盲转/转错/多转已全部归零"
},
{
"at": "方向修正",
"text": "旋转配额的思路被推翻：旋转免费且是核心特色，不该限次数，该限的是每次旋转的回报与可读性。实测发现 17 关最少旋转全部恒为 3（生成器写死），且 petals/crossing 两个家族几乎没有反差感（平均 2.0–5.7 对比 weave 的 22–33）"
},
{
"at": "P1 进展",
"text": "旋转深度打开：生成分布从「全部恒为3」变成 {3,4,5,6,7,8,9}。两次结构性猜测失败并已撤回（插入端多朝向出口过滤实测零效果）。深关卡的瓶颈定位为终局三枚互锁，改用扩大扫描量应对"
},
{
"at": "演出",
"text": "离场动画封顶定稿（拐点 2.4s / 上限 3.2s），最长 17.4s→3.5s；拐点以下手感不变。同时发现开局演出是结构性缺陷并在构造里修掉"
}
"goal": "调出 50 关难度合适、好玩的关卡，交付人工审核",
"updated": "09-14 16:45",
"round": 1,
"definition": {
"title": "「难度合适且好玩」的操作定义",
"why": "旋转是免费的、是这个游戏的特色，玩家应该频繁做——所以不该限制旋转次数。该限制的是每一次旋转值不值：转了有没有棋子出来、能不能看出来该转哪、以及转完之后有没有「原来这么简单」的反差。评分里只有 difficulty（复测 r=0.93）和 stuck（r=0.85）可信，rotation / clarity / fun 是噪声（0.32 / 0.19 / 0.56），所以「好玩」全部换成机械代理。 另外，演出本身就是奖励——棋子沿弯路走出去的那几秒是减压点，所以开局必须有得看，而不是只有两枚贴边棋子和 0.3 秒。",
"rules": [
{
"k": "旋转要深",
"v": "难度来自「需要转很多次」，不是「转错受罚」。旋转数随关号上升，不设上限",
"how": "求解器最少旋转数"
},
{
"k": "每转都有回报",
"v": "每一次旋转至少放出 2 枚棋子；无效转动 = 0",
"how": "deepLevelStats.releasePerTurn / 规则机器人"
},
{
"k": "演出是回报",
"v": "开局就要有得看：可点 ≥3 枚、其中最好的一条离场演出 ≥0.9s；全关长巡游（≥0.9s）≥3 次",
"how": "showStats，用真实动画计时"
},
{
"k": "反差感",
"v": "平均反差 ≥12：棋子在某个朝向看着像噩梦（长路线多弯），转过去就是几步走出（代价 = 格数 + 2×弯数）",
"how": "六朝向逐棋子代价差"
},
{
"k": "不无脑",
"v": "猴子 动作÷棋子 ≥3.0（v0.5 全库 1.67 = 乱点就能过）",
"how": "生成时算"
},
{
"k": "不挫败",
"v": "stuck ≤2.0；终局任一朝向卡住时，左右转都能开",
"how": "grok 抽样 + 求解器"
},
{
"k": "不重复",
"v": "盘面签名互不相同，路线家族分布不偏科",
"how": "deepLevelKey"
}
]
},
"targets": {
"levels": 50,
"rotationDepth": [
{
"band": "L1–L12",
"rot": "2–3 转",
"difficulty": "1.5 – 2.0"
},
{
"band": "L13–L32",
"rot": "3–5 转",
"difficulty": "2.0 – 2.6"
},
{
"band": "L33–L50",
"rot": "5–8 转",
"difficulty": "2.6 – 3.0"
}
],
"gates": {
"releasePerTurn": "≥2",
"blindRotations": 0,
"contrast": "≥12",
"monkeyAPT": "≥3.0"
}
},
"phases": [
{
"id": "P0",
"title": "判据：先说清楚什么叫「好」",
"tasks": [
{
"id": "T0.1",
"title": "难度曲线与旋转配额定义",
"status": "done",
"note": "50 关分三档，0 转关卡死 10% 上限（用户硬要求）"
},
{
"id": "T0.2",
"title": "机械验收门槛",
"status": "done",
"note": "盲转=0、转错=0、终局可走朝向≥2 —— 前两项已由改锁结构达成"
},
{
"id": "T0.3",
"title": "「好玩」的机械代理指标",
"status": "done",
"note": "猴子 APT ≥3.0。fun 评分复测 r=0.56 不可用，改用「乱点过不了」当代理"
}
]
},
{
"id": "P1",
"title": "产能：解除旋转上限，把反差感做进筛选",
"tasks": [
{
"id": "T1.1",
"title": "growDeepLevel 扩展到 6 个朝向",
"status": "done",
"note": "现在只在朝向 0/1/2 插棋子，所以最少旋转恒等于 3 —— 这是「太简单」的根因"
},
{
"id": "T1.2",
"title": "解除 rotations===3 的硬限制，允许 4/5/6+",
"status": "done"
},
{
"id": "T1.3",
"title": "反差感指标进 deepLevelStats 并作门槛",
"status": "done",
"note": "contrastStats 已进 deepLevelStats 并作门槛；实测 weave 22–33、switchback 29、spiral 18–24、orbits 12–17，petals 仅 2.0–5.7、crossing 3.8–5.5"
},
{
"id": "T1.4",
"title": "每转释放 ≥2 枚的门槛",
"status": "done",
"note": "改为按朝向配额插棋（每个朝向 ≥2 枚在等），取代原来的随机比例——随机比例会留下空转的朝向"
},
{
"id": "T1.5",
"title": "扩产到 ≥120 候选并去重、家族均衡",
"status": "doing",
"note": "候选池已达 235 关（深度 {3:165,4:46,5:22,7:2}），正用含开局展示位的构造重扫"
},
{
"id": "T1.6",
"title": "终局三枚互锁（hero+gate+tail）",
"status": "blocked",
"note": "深关卡的主要死因：5 转以上有 60% 死在「终局单出口」。实测终局是锁链自己的三枚棋子在 6 个朝向里堵死 5 个。双向释放约束只覆盖了 hero/tail 两枚，三枚组合覆盖不到——这是可达状态的性质，构造时难保证。当前对策是提高扫描量靠筛选拿到，不是修构造"
},
{
"id": "T1.7",
"title": "开局展示位（快速爽到）",
"status": "done",
"note": "实测整个构造的开局中位只有 2 枚可点、最佳演出 0.57s——锁链开局全锁死。在构造里预留 2 个开局位（立即可点且路线长到值得看），实测开局可点 2→3 枚、最佳演出 0.72s→1.12s"
},
{
"id": "T1.8",
"title": "离场演出封顶",
"status": "done",
"note": "计时原本线性无上限，最长一次 17.4s，正式关 18.8% 超过 2s。拐点 2.4s 以下完全不变，以上加速逼近 3.2s 上限。labs/exit-timing 可同步对比"
}
]
},
{
"id": "P2",
"title": "标定：把机械指标映射到真实难度",
"tasks": [
{
"id": "T2.1",
"title": "grok 抽样跑候选池，收集 difficulty / stuck",
"status": "done",
"note": "R1：grok-4.6 xhigh 全量跑 50 关，收齐 difficulty / stuck / 机械指标与自由文本抱怨"
},
{
"id": "T2.2",
"title": "拟合 difficulty ~ 机械特征",
"status": "blocked",
"note": "做不成，且不是样本量问题：27 关里 26 关 difficulty 都打 2 分，响应几乎无方差，回归的是常数。根因在 bench 本身——文字盘面把每颗棋子的路线、挡路者、紧贴者直接列成清单，agent 不必用眼睛顺线走，而那正是这游戏的难点。agent 原话：「挡路名单已经写全，不用顺着格子数」。为解释它另建的 legibility 指标与评分相关 -0.02，同样无效，已记录为负结果不采用"
},
{
"id": "T2.3",
"title": "用模型给全部候选打分并按曲线选 50 关",
"status": "done",
"note": "select-50 按机械特征打分并均匀铺开，家族上限 36%。权重是设计假设而非拟合结果（见 T2.2），由人工审核裁定"
}
]
},
{
"id": "P3",
"title": "迭代：跑几轮把曲线调平",
"tasks": [
{
"id": "T3.1",
"title": "第 1 轮：grok 全量跑 50 关",
"status": "done",
"note": "50 关全部跑完；机械面干净：空点 0、盲转 ~0.3/关、全部通关。总体读数：整批偏易一档（2 分 vs 刚好的 3 分）"
},
{
"id": "T3.2",
"title": "按实测曲线调整选关，重选",
"status": "doing",
"note": "按 R1 唯一可用的总体读数「偏易」调整：扩池到 400 种子后把旋转深度整体上推，而不是按关排名重选"
},
{
"id": "T3.3",
"title": "第 2 轮验证",
"status": "todo"
},
{
"id": "T3.4",
"title": "收敛判定（难度曲线单调、stuck 达标）",
"status": "todo"
}
]
},
{
"id": "P4",
"title": "交付",
"tasks": [
{
"id": "T4.1",
"title": "50 关关卡集",
"status": "todo"
},
{
"id": "T4.2",
"title": "人工审核页（可直接试玩每一关）",
"status": "done",
"note": "docs/review-50.html：50 张开局盘面缩略图 + 机械指标 + agent 实测 + 试玩直达；并标注「哪些数字能信」，把感知类问题（路线长度、紧贴）显式交给人眼"
},
{
"id": "T4.3",
"title": "后续关卡制作流程的优化总结",
"status": "todo"
}
]
}
],
"rounds": [],
"log": [
{
"at": "起点",
"text": "已完成前置工作：擦碰漏报修复（113/581 例）、终局单出口门槛、锁结构改为双向可开 —— 规则机器人盲转/转错/多转已全部归零"
},
{
"at": "方向修正",
"text": "旋转配额的思路被推翻：旋转免费且是核心特色，不该限次数，该限的是每次旋转的回报与可读性。实测发现 17 关最少旋转全部恒为 3（生成器写死），且 petals/crossing 两个家族几乎没有反差感（平均 2.0–5.7 对比 weave 的 22–33）"
},
{
"at": "P1 进展",
"text": "旋转深度打开：生成分布从「全部恒为3」变成 {3,4,5,6,7,8,9}。两次结构性猜测失败并已撤回（插入端多朝向出口过滤实测零效果）。深关卡的瓶颈定位为终局三枚互锁，改用扩大扫描量应对"
},
{
"at": "演出",
"text": "离场动画封顶定稿（拐点 2.4s / 上限 3.2s），最长 17.4s→3.5s；拐点以下手感不变。同时发现开局演出是结构性缺陷并在构造里修掉"
},
{
"at": "R1 结论",
"text": "难度评分在这批关卡上无区分度（26/27 都是 2 分），原因是文字 bench 把判断白送给了 agent。结论：agent 轮次用于抓缺陷（死局、盲转、空点、演出），难度排序只能靠机械假设 + 人工审核。这条改变了后续流程的设计"
},
{
"at": "未擅自改动",
"text": "agent 33 条抱怨里两大主题是路线太长（68 格 27 弯）与紧贴易看漏。但已上线的 v0.5 在这两项上更严重（最长 88 格 vs 84 格），属于游戏本来的性格；且文字盘面会同时夸大和缩小感知问题。因此只在审核页标出，不擅自设闸门"
}
]
}