diff --git a/labs/bench/AGENT.md b/labs/bench/AGENT.md new file mode 100644 index 0000000..d36e1a2 --- /dev/null +++ b/labs/bench/AGENT.md @@ -0,0 +1,165 @@ +# 给 Agent:来玩一关 HEX TRAILS + +**你现在是一名玩家,不是程序员。** 读完这个文件就直接开始,不需要再读任何别的文档。 + +你要像人一样玩一关六角轨道解谜:看盘、决定点哪枚或转哪边、看反馈、再决定。有人会在网页上实时看着你操作,所以每一步都要写清楚你为什么这么做。 + +**禁止**:读 `src/` 下的任何文件、读 `levels.json`、读 `runs/` 里的存档、调用求解器、写脚本或循环批量执行、修改任何文件。 +你和棋盘之间只有 `node scripts/bench.mjs` 这一个命令——你知道的必须全部来自它打印给你的东西。绕过它就等于作弊,这一局的数据也就废了。 + +--- + +## 第一步:开一局 + +```bash +node scripts/bench.mjs start --model=<你的模型> --agent="<你的运行环境 · 模式>" --level=1 --persona=tapper --no-rules +``` + +**请如实报告你自己是谁**——观察页会把这两项显示在棋盘正上方,人类靠它知道现在是谁在玩: + +- `--model`:你的模型标识符,例如 `claude-opus-5`、`grok-4.6`、`deepseek-flash`、`gpt-5`。不确定就写最接近的。 +- `--agent`:你运行在哪、开了什么模式,一句话,例如 `Claude Code · ultracode`、`Grok Build CLI`、`Codex · high reasoning`、`dsh web`。包含任何你认为影响自己表现的配置(推理强度、是否联网、是否有思考预算)。不知道就写 harness 名字。 +- `--level`:关号,不指定就是第 1 关。`node scripts/bench.mjs list` 可以看有哪些关。 +- `--persona`:`tapper`(试错型,默认)/ `reader`(读盘型)/ `turner`(转盘型)/ `novice`(新手)。按这个风格玩。 + +命令会打印两样东西,都要留意: + +1. **局名**(形如 `grok-4.6-L1-tapper`)——后面每一条命令都要带 `--run=<那个局名>`,原样抄,别自己改。 +2. **观察页地址**——把那一行原样转告人类,他们能实时看你玩。 + +--- + +## 规则(30 秒) + +- 棋盘上画着轨道。**点一枚棋子,它沿自己箭头所指的那条轨道一路走,直到走出棋盘边缘**——这就是离场。 +- 路上撞到别的棋子就退回原位,什么都不扣。棋子有宽度,**擦到紧挨着的棋子也算撞**。 +- **旋转底盘**:整块棋盘连轨道转 60°,棋子的位置和箭头不动——所以每枚棋子接上的是另一条轨道,所有路线全变。左右转免费、可撤销,转 6 次回到原样。 +- 目标:让所有棋子离场。**没有失败**——移走一枚棋子只会让别的路更通,不会把谁困住,所以不存在"点错一步就死"。有步数预算(默认 80 步)。 + +## 你会看到什么 + +最重要的是这一段——每枚棋子当前的路线,就是人眼在屏幕上看到的那条线: + +``` +每枚棋子当前的路线(箭头出发 → 依次经过的格 → 出界/闭环;[tX] = 那格上压着棋子 tX): + t5 @ -2,4 ↙:就在边上,直接出界(0 格)· 路上没有棋子 + t0 @ 4,-4 ↖:3,-4 → 2,-4[t9] → 1,-3 → 0,-3 → 出界(4 格 · 1 弯)· 路上有 t9 + t2 @ 0,-2 ↙:-1,-1 → -2,0 → -3,1 → -4,2 → 出界(4 格 · 0 弯)· 路上没有棋子,但紧贴 t7(会擦到) + t1 @ 3,-1 ↓:3,0[t6] → 2,1 → 1,2 → 出界(3 格 · 1 弯)· 路上有 t6,另外紧贴 t4(会擦到) + t3 @ 2,2 ↗:3,1 → 3,0 → 2,1 → 2,2 → 闭环(4 格 · 2 弯)· 路上没有棋子 +``` + +| 你看到 | 意味着 | +|---|---| +| `→ 出界 · 路上没有棋子` | **点它就能走** | +| `2,-4[t9] … 路上有 t9` | 被 t9 压着,**先让 t9 走** | +| `但紧贴 t7(会擦到)` | 棋子有宽度,线从 t7 身边擦过去,**点了会被挡**;同样先让 t7 走 | +| `路上有 t6,另外紧贴 t4` | 两个都挡着,**t6 和 t4 都清掉**这枚才能走 | +| `→ 闭环` | 这条轨道绕回来了,当前朝向出不去,**只有旋转能改变它** | + +坐标 `q,r` 只是格子的名字,不需要做坐标运算。 + +**这一行把挡住它的棋子全部列全了**——路上压着的和身边擦到的都在里面。要让这枚走,得把列出来的全部清掉;反过来,只要这一行说「路上没有棋子」且没有「紧贴」,点它就一定能走。 + +## 怎么玩得好 + +1. 先点**路上没有棋子、不紧贴谁、能出界**的。 +2. 有 `[tX]` 或 `紧贴 tX` 的,先解决 tX;tX 也被挡就顺着往下找,先解最外层。 +3. **一枚都走不了**(每条路都被压着/擦着、或都是闭环)→ 旋转。转完所有路线都变,**必须重新看盘**。 +4. **转一次没打开出口,不代表这一关无解。** 有些关要转 2–3 次、甚至换个方向转,出口才出现。转到没见过的朝向去,别在同一个朝向反复点。 +5. 每一步都算预算。有把握再点,不确定就先看清楚。 + +--- + +## 第二步:反复出招 + +**像人一样连点**:看清一次盘面,就把有把握的几步用逗号一起提交,不要每点一枚重新读一遍。 + +```bash +node scripts/bench.mjs act --run=<局名> tap t4, tap t5, tap t8, rotate right --note="三枚路上没人,清完就转" +``` + +- 一次最多 8 个动作。链会在**第一个没成功的动作**和**任何一次旋转**之后自动停下,并打印新盘面——所以不会闭眼连点到错的地方。 +- 动作:`tap tX` · `rotate right` / `rotate left` · `undo` · `giveup 原因` +- `--note` 必填,一句话说明你为什么这么做。这是人类唯一能看到的你的想法。 +- 只想再看一次盘面:`node scripts/bench.mjs observe --run=<局名>` + +**如果你觉得这一关无解,不要硬撑**,用 `act --run=<局名> giveup 我认为无解,因为……` 说明你卡在哪、试过什么。**诚实地放弃比勉强通关更有价值**——我们正是要找出让人觉得无解的关卡。预算用完局也会自动结束,同样要评分。 + +### 命令报错了怎么办 + +| 报错 | 原因 | 怎么办 | +|---|---|---| +| `没有这局:X(先 start)` | `--run` 写错了 | 用 start 打印的那个局名,原样抄 | +| `无效:没有棋子 tX` | 棋子 id 记错,或它已经离场 | 先 `observe` 重新看一遍当前有哪些棋子 | +| `act 需要:tap ID \| rotate …` | 动作写法不对 | 只有 `tap tX` / `rotate left\|right` / `undo` / `giveup 理由` 四种 | +| `X 已存在` | 局名重复 | 不要自己指定 `--run`,让 start 自动生成 | + +报错不扣步数,改对了重发就行。 + +--- + +## 第三步:评分(必做) + +清盘、放弃或预算用完之后: + +```bash +node scripts/bench.mjs rate --run=<局名> --json='{"clarity":4,"rotation":3,"stuck":2,"fun":3,"difficulty":3,"worst":"第 6 步点 t2 被 t7 擦碰挡住,路线上看不出来"}' +``` + +| 键 | 问题 | 1 | 3 | 5 | +|---|---|---|---|---| +| `clarity` | 路线看得清吗 | 完全看不清 | 大致能读 | 一眼看清 | +| `rotation` | 旋转有意义吗 | 随便转都行/没用 | 有点用 | 每次转都是关键决定 | +| `stuck` | 有没有觉得无解的时刻 | 从未 | 卡过一会儿 | 长时间以为无解 | +| `fun` | 好玩吗 | 无聊 | 还行 | 很好玩 | +| `difficulty` | 难度是否合适 | **太简单** | **刚刚好** | **太难** | +| `worst` | 最困惑的一步是第几步、为什么 | 一句话 | | | + +注意 `difficulty` 是两头都不好:3 分是最佳,1 和 5 都是问题。 + +**请诚实评分**,尤其 `rotation` 和 `stuck`——我们正是要用这两项判断关卡好不好,一律给高分帮不上忙。 + +`rate` 会打印本局统计和回放链接。最后用两三句话向人类汇报:局名、状态(cleared / gave-up / budget)、步数、决策次数、最困惑的一步。 + +--- + +## 完整示例 + +``` +$ node scripts/bench.mjs start --model=grok-4.6 --agent="Grok Build CLI" --level=1 --persona=tapper --no-rules +开始 grok-4.6-L1-tapper · 关卡 proto-d5-switchback-r4-s6 · 人设 tapper · 模型 grok-4.6 · Grok Build CLI · 预算 80 步 +观察页(人类实时观看):http://localhost:4173/labs/bench/index.html?run=grok-4.6-L1-tapper&live=1 +后续命令都用 --run=grok-4.6-L1-tapper +…盘面… + +$ node scripts/bench.mjs act --run=grok-4.6-L1-tapper tap t4, tap t5, tap t8 --note="三枚路上没人也不紧贴谁" +点击 t4 → 离场(走了 1 格、0 个弯) +点击 t5 → 离场(走了 1 格、0 个弯) +点击 t8 → 离场(走了 3 格、1 个弯) +…新盘面… + +$ node scripts/bench.mjs act --run=grok-4.6-L1-tapper rotate right --note="剩下的互相压着或闭环,一枚都走不了" +右转 → 底盘转到朝向 1,4 枚棋子的出口打开了 +(旋转后盘面已变,后面的动作没有执行) +…新盘面… + +$ node scripts/bench.mjs rate --run=grok-4.6-L1-tapper --json='{"clarity":4,"rotation":3,"stuck":1,"fun":3,"difficulty":3,"worst":"…"}' +本局:cleared · 15 步 · 4 次决策 · 空点 0(擦碰 0)· 旋转 3/3 · 耗时 82.3 s +回放:http://localhost:4173/labs/bench/index.html?run=grok-4.6-L1-tapper +``` + +--- + +## 其它命令(一般用不到) + +| 命令 | 作用 | +|---|---| +| `node scripts/bench.mjs list` | 列出本组关卡 | +| `node scripts/bench.mjs status --run=X` | 回看自己走过的每一步 | +| `node scripts/bench.mjs metrics --run=X` | 本局指标 | +| `node scripts/bench.mjs open --run=X` | 起服务并打开观察页 | +| `--format=full` | 观察时额外打印每格的端口配对表(读长路线时才需要) | +| `--set=<关卡文件>` | 换一组关卡;`start` 和 `list` 都支持 | + +**要玩多关就重复第一步到第三步**,每关一个新局,`--level` 换成下一关。不要写循环一次跑完——一次一关,一次一个决定。 diff --git a/scripts/bench.mjs b/scripts/bench.mjs new file mode 100644 index 0000000..340a015 --- /dev/null +++ b/scripts/bench.mjs @@ -0,0 +1,173 @@ +#!/usr/bin/env node +// Agent bench CLI. One command per step so any agent with a shell can play. +// Agents are given labs/bench/AGENT.md and need nothing else. +// node scripts/bench.mjs list [--set=levels/prototypes/d5.json] +// node scripts/bench.mjs start --model=NAME [--agent='运行环境 · 模式'] [--level=N] [--run=NAME] [--persona=…] [--set=…] [--peek] [--budget=80] [--no-rules] +// node scripts/bench.mjs observe --run=NAME [--format=json|full] +// node scripts/bench.mjs act --run=NAME tap t3, tap t5, rotate right [--note=…] [--quiet] +// node scripts/bench.mjs rate --run=NAME --json='{"clarity":3,…}' +// node scripts/bench.mjs status|metrics --run=NAME [--json] +// node scripts/bench.mjs open [--run=NAME] # 起服务、打印/打开观察页 +// node scripts/bench.mjs rules [--peek] +// Runs live in bench/runs/NAME.json; the observer page polls that file. +import { spawn, spawnSync } from 'node:child_process'; +import { existsSync, mkdirSync, readFileSync, writeFileSync, readdirSync } from 'node:fs'; +import { createConnection } from 'node:net'; +import { resolve } from 'node:path'; +import { createRun, observe, actMany, rate, metrics, rulesText, RATING_QUESTIONS, describeAction, describeResult } from '../src/bench.js'; + +const RUNS = resolve('bench/runs'); +const argv = process.argv.slice(2); +const command = argv[0]; +const flags = {}, positional = []; +for (const a of argv.slice(1)) { + if (a.startsWith('--')) { const [k, ...v] = a.slice(2).split('='); flags[k] = v.length ? v.join('=') : true; } + else positional.push(a); +} +const fail = message => { console.error(message); process.exit(1); }; +// text = routes only (default, what a person sees); full = also the per-cell rail table; json = machine. +const fmt = () => flags.format === 'json' ? 'json' : flags.format === 'full' || flags.full ? 'full' : 'text'; +const loadSet = path => { + const raw = JSON.parse(readFileSync(resolve(path), 'utf8')); + return Array.isArray(raw) ? raw : Array.isArray(raw.levels) ? raw.levels : Object.values(raw).filter(l => l && l.cells); +}; +const runPath = name => resolve(RUNS, `${name}.json`); +const PORT = Number(process.env.PORT || 4173); +const slug = s => String(s).replace(/[^\w.-]+/g, '-').replace(/^-+|-+$/g, '') || 'agent'; +// Name the run so the agent never has to invent one: model-L3-tapper, then -2, -3… +function autoRunName(model, level, persona) { + const base = `${slug(model)}-L${level}-${slug(persona)}`; + if (!existsSync(runPath(base))) return base; + for (let n = 2; n < 99; n++) if (!existsSync(runPath(`${base}-${n}`))) return `${base}-${n}`; + return `${base}-${Date.now()}`; +} +const portOpen = () => new Promise(done => { + const socket = createConnection({ port: PORT, host: '127.0.0.1' }); + const finish = ok => { socket.destroy(); done(ok); }; + socket.setTimeout(400); + socket.on('connect', () => finish(true)); + socket.on('error', () => finish(false)); + socket.on('timeout', () => finish(false)); +}); +// Bring the observer page within reach: start the static server if it is not +// already up, then print (and on macOS open) the URL for this run. Returns +// null in a portable pack, which ships no page and no server — a URL there +// would point at whatever else happens to hold the port. +const OBSERVER = resolve('labs/bench/index.html'), SERVER = resolve('scripts/serve.mjs'); +async function observerUrl(run, { open = false } = {}) { + if (!existsSync(OBSERVER) || !existsSync(SERVER)) return null; + const url = `http://localhost:${PORT}/labs/bench/index.html${run ? `?run=${encodeURIComponent(run)}&live=1` : ''}`; + if (!(await portOpen())) { + const server = spawn(process.execPath, [SERVER], { detached: true, stdio: 'ignore' }); + server.unref(); + for (let i = 0; i < 20 && !(await portOpen()); i++) await new Promise(r => setTimeout(r, 150)); + } + if (open && process.platform === 'darwin') spawnSync('open', [url], { stdio: 'ignore' }); + return url; +} +const loadRun = () => { + if (!flags.run) fail('需要 --run=NAME'); + if (!existsSync(runPath(flags.run))) fail(`没有这局:${flags.run}(先 start)`); + return JSON.parse(readFileSync(runPath(flags.run), 'utf8')); +}; +const saveRun = runObj => { + mkdirSync(RUNS, { recursive: true }); + runObj.updatedAt = new Date().toISOString(); + writeFileSync(runPath(runObj.run), JSON.stringify(runObj)); + const indexPath = resolve(RUNS, 'index.json'); + const index = existsSync(indexPath) ? JSON.parse(readFileSync(indexPath, 'utf8')) : []; + const entry = { run: runObj.run, level: runObj.level.id, persona: runObj.persona, model: runObj.model, peek: runObj.peek, status: runObj.status, steps: runObj.steps.length, updatedAt: runObj.updatedAt }; + const at = index.findIndex(e => e.run === runObj.run); + if (at >= 0) index[at] = entry; else index.push(entry); + writeFileSync(indexPath, JSON.stringify(index, null, 1)); +}; + +if (command === 'list') { + const set = flags.set || 'levels/prototypes/d5.json'; + const levels = loadSet(set); + console.log(`${set} · ${levels.length} 关`); + levels.forEach((l, i) => console.log(` ${String(i + 1).padStart(3)} ${l.id} ${l.cells.length} 格 · ${l.tiles.length} 枚${l.prototype ? ` · 锁住 ${Math.round(l.prototype.stats.lockedShare * 100)}% · 最少 ${l.prototype.stats.rotations} 转` : ''}`)); +} else if (command === 'rules') { + console.log(rulesText(!!flags.peek)); +} else if (command === 'start') { + const levels = loadSet(flags.set || 'levels/prototypes/d5.json'); + const wanted = flags.level ?? 1; + const level = levels.find(l => l.id === wanted) || levels[Number(wanted) - 1]; + if (!level) fail(`找不到关卡 ${wanted}(本组共 ${levels.length} 关,用 list 查看)`); + const persona = flags.persona || 'tapper'; + const model = flags.model || process.env.BENCH_MODEL || 'unknown'; + if (model === 'unknown') console.warn('提示:没有 --model,本局记为 unknown;填上你的模型标识才能按模型比较。'); + const name = flags.run ? String(flags.run) : autoRunName(model, levels.indexOf(level) + 1, persona); + if (!/^[\w.-]+$/.test(name)) fail('run 名只能用字母、数字、_ - .'); + if (existsSync(runPath(name)) && !flags.force) fail(`${name} 已存在;换个名字或加 --force`); + const agent = flags.agent ? String(flags.agent) : ''; + const runObj = createRun({ level, run: name, persona, model, agent, peek: !!flags.peek, budget: Number(flags.budget || 80), startedAt: new Date().toISOString() }); + saveRun(runObj); + const url = await observerUrl(name, { open: !!flags.open }); + console.log(`开始 ${runObj.run} · 关卡 ${level.id} · 人设 ${runObj.persona} · 模型 ${runObj.model}${runObj.agent ? ` · ${runObj.agent}` : ''} · 预算 ${runObj.budget} 步 · peek ${runObj.peek ? '开' : '关'}`); + if (!runObj.agent) console.warn('提示:没有 --agent,观察页横幅只会显示模型名。填上运行环境和模式,例如 --agent="Claude Code · ultracode"。'); + if (url) console.log(`观察页(人类实时观看):${url}`); + console.log(`后续命令都用 --run=${runObj.run}`); + console.log(''); + // --no-rules: an agent that already read AGENT.md skips the rules block. + if (!flags['no-rules']) { console.log(rulesText(runObj.peek)); console.log(''); } + console.log(observe(runObj, { format: fmt() })); +} else if (command === 'open') { + const url = await observerUrl(flags.run ? String(flags.run) : null, { open: flags.open !== 'false' }); + console.log(url || '这个副本没有观察页(便携包);把 runs/ 交回原仓库用 bench-import.mjs 导入后再看。'); +} else if (command === 'observe') { + const runObj = loadRun(); + const view = observe(runObj, { format: fmt() }); + console.log(typeof view === 'string' ? view : JSON.stringify(view, null, 1)); +} else if (command === 'act') { + const runObj = loadRun(); + // One action, or several separated by commas: `tap t4, tap t5, rotate right`. + // A chain stops at the first tap that fails and after any turn. + const parse = words => { + const [kind, ...rest] = words; + if (kind === 'tap' || kind === 'peek') return { type: kind, id: rest[0] }; + if (kind === 'rotate') return { type: 'rotate', direction: ['right', 'r', '1', '+1', '右'].includes(rest[0]) ? 1 : ['left', 'l', '-1', '左'].includes(rest[0]) ? -1 : 0 }; + if (kind === 'undo') return { type: 'undo' }; + if (kind === 'giveup') return { type: 'giveup', reason: rest.join(' ') }; + fail('act 需要:tap ID | rotate left|right | undo | peek ID | giveup 原因;多个动作用逗号隔开'); + }; + const groups = positional.join(' ').split(',').map(s => s.trim()).filter(Boolean).map(s => parse(s.split(/\s+/))); + if (!groups.length) fail('act 需要一个动作'); + if (groups.length > 8) fail('一次最多连做 8 个动作'); + const { outcomes, stopped } = actMany(runObj, groups, { note: flags.note ? String(flags.note) : '' }); + saveRun(runObj); + for (const out of outcomes) console.log(`${describeAction(out.step?.action || out.rejected?.action)} → ${out.feedback}`); + if (stopped) console.log(`(${stopped})`); + if (!flags.quiet && runObj.status === 'playing') { console.log(''); console.log(observe(runObj, { format: fmt() === 'json' ? 'text' : fmt() })); } + if (runObj.status !== 'playing') console.log(`\n本局结束:${runObj.status}。请用 rate 提交评分。`); + if (flags.json) console.log(JSON.stringify({ outcomes, stopped })); +} else if (command === 'rate') { + const runObj = loadRun(); + let answers; + try { answers = JSON.parse(flags.json || '{}'); } catch { fail('--json 需要合法 JSON'); } + const rating = rate(runObj, answers); + saveRun(runObj); + console.log('已记录评分:' + JSON.stringify(rating)); + const missing = RATING_QUESTIONS.filter(q => rating[q.key] === null).map(q => q.key); + if (missing.length) console.log('未填:' + missing.join(', ')); + // Close the loop for the agent: the numbers it should report, and the link. + const m = metrics(runObj); + const secs = ms => ms == null ? '—' : ms >= 1000 ? `${(ms / 1000).toFixed(1)} s` : `${ms} ms`; + console.log(`本局:${m.status} · ${m.steps} 步 · ${m.decisions} 次决策 · 空点 ${m.deadTaps}(擦碰 ${m.grazeDeadTaps})· 旋转 ${m.rotations}/${m.minimumRotations ?? '?'} · 耗时 ${secs(m.timing.totalMs)}`); + const replayUrl = await observerUrl(runObj.run); + if (replayUrl) console.log(`回放:${replayUrl}`); +} else if (command === 'status') { + const runObj = loadRun(); + console.log(`${runObj.run} · ${runObj.level.id} · ${runObj.status} · ${runObj.steps.length}/${runObj.budget} 步 · 人设 ${runObj.persona} · 模型 ${runObj.model}`); + for (const s of runObj.steps) console.log(` ${String(s.n).padStart(3)} ${describeAction(s.action).padEnd(10)} → ${describeResult(s.result)}${s.note ? ` 「${s.note}」` : ''}`); +} else if (command === 'metrics') { + const runObj = loadRun(); + const m = metrics(runObj); + if (flags.json) console.log(JSON.stringify(m, null, 1)); + else console.log(Object.entries(m).filter(([k]) => k !== 'rating').map(([k, v]) => `${k}: ${typeof v === 'number' ? Number(v.toFixed(2)) : v}`).join('\n') + (m.rating ? '\nrating: ' + JSON.stringify(m.rating) : '')); +} else if (command === 'runs') { + const dir = existsSync(RUNS) ? readdirSync(RUNS).filter(f => f.endsWith('.json') && f !== 'index.json') : []; + for (const f of dir) { const r = JSON.parse(readFileSync(resolve(RUNS, f), 'utf8')); console.log(`${r.run.padEnd(28)} ${r.level.id.padEnd(34)} ${r.status.padEnd(9)} ${String(r.steps.length).padStart(3)} 步 ${r.persona}/${r.model}`); } +} else { + fail('用法:node scripts/bench.mjs list|start|act|observe|rate|status|metrics|runs|open|rules …\n完整说明(给 Agent 读的那一份):labs/bench/AGENT.md'); +} diff --git a/src/bench.js b/src/bench.js new file mode 100644 index 0000000..7304ed4 --- /dev/null +++ b/src/bench.js @@ -0,0 +1,292 @@ +import { DIRS, Game, available, clone, opposite, rotateBoard, solveRotations, trace } from './core.js'; + +// Agent bench: a level session that any agent can drive through observe/act. +// The observation carries only what a person can read off the screen — cells, +// rails, pieces, arrows, orientation. Never which pieces can currently leave, +// and never the solver. Metrics use the solver afterwards, on the recorded run. + +export const PORT_NAMES = ['↑', '↗', '↘', '↓', '↙', '↖']; +export const ACTIONS = ['tap', 'rotate', 'undo', 'peek', 'giveup']; +const countBends = route => route.steps.filter(s => s.enter !== null && opposite(s.enter) !== s.exit).length; + +export function rulesText(peek) { + return [ + '规则:棋盘由六角格组成,每格画着轨道。点击一枚棋子,它沿着自己箭头所指的那条轨道一路走,直到走出棋盘边缘(离场)。途中如果撞上另一枚棋子,就退回原位(棋子有宽度,擦到紧邻的棋子也算撞)。被挡不扣任何东西。', + '旋转底盘:整块棋盘连轨道转 60°,棋子的位置和箭头不动——于是每枚棋子接上的是另一条轨道,路线全部改变。左转右转免费、可反复、可撤销;转 6 次回到原样。', + '目标:让所有棋子离场。没有失败;有步数预算。', + '', + '怎么看盘面:观察里给出每枚棋子当前的路线——它会依次经过哪些格、在哪里出界(或绕成闭环)。路线上压着别的棋子,标成 [tX];路线紧贴着别的棋子(棋子有宽度,会擦到),标成"紧贴 tX"。这就是人眼在屏幕上看到的那条线。', + '怎么玩:', + '1) 路上没有棋子、不紧贴谁、而且能出界的,点它就能离场。先点这些。', + '2) 路上有 [tX] 或紧贴 tX 的,得先让 tX 走:tX 自己能走就先点 tX;tX 也被挡就顺着往下找,先解开最外层的。', + '3) 每条路线都有人挡、或都是闭环——一枚都走不了,那就旋转。旋转后所有路线都变,重新看。转完没有出口,可以再转或转回去。', + '4) 每一步都算预算:先做有把握的,别乱试。' + (peek ? ' peek 与观察里的路线是同一信息,一般用不着。' : ''), + ].join('\n'); +} + +// Routes are shown in full: a folded "…(共 54 格)" hides exactly the part a +// careful player would re-check. Callers that must save tokens pass routeLimit. +const ROUTE_SHOW = Infinity; +// The drawn line, as a person sees it: the cells it passes, who is sitting on +// it, and — because pieces are wide — who it brushes past. The brush is taken +// from the real swept collider, since that is what the eye judges from the +// curve hugging a neighbour; a cell-adjacency guess would over-warn. +function pieceRoutes(game) { + const occupant = new Map(game.tiles.map(t => [`${t.q},${t.r}`, t.id])); + return game.tiles.map(t => { + const geo = trace(game.level, t, [], true); + const cells = geo.steps.map(s => { const who = occupant.get(`${s.q},${s.r}`); return { q: s.q, r: s.r, piece: who && who !== t.id ? who : null }; }); + const onRoute = [...new Set(cells.filter(c => c.piece).map(c => c.piece))]; + // Every piece that would stop this one, not just the first. Reporting only + // the leading blocker hid every graze behind it: in 28% of "blocked by a + // piece on the line" cases, clearing that piece sprang a collision the + // agent had never been shown. Test each candidate on its own so nothing + // masks anything else. + const grazes = []; + for (const other of game.tiles) { + if (other.id === t.id || onRoute.includes(other.id)) continue; + const hit = trace(game.level, t, [other]); + if (hit.status === 'blocked' && hit.blocker === other.id) grazes.push({ id: other.id, at: hit.steps.length }); + } + grazes.sort((a, b) => a.at - b.at); + return { id: t.id, q: t.q, r: t.r, dir: t.dir, arrow: PORT_NAMES[t.dir], color: t.color, + route: { end: geo.status === 'exit' ? 'exit' : 'loop', cells: cells.slice(1), bends: countBends(geo), onRoute, grazes: grazes.map(g => g.id) } }; + }); +} +function routeLine(p) { + const shown = p.route.cells.slice(0, ROUTE_SHOW).map(c => `${c.q},${c.r}${c.piece ? `[${c.piece}]` : ''}`); + const tail = p.route.cells.length > ROUTE_SHOW ? ` … (共 ${p.route.cells.length} 格)` : ''; + const end = p.route.end === 'exit' ? '出界' : '闭环'; + const who = p.route.onRoute.length ? `路上有 ${p.route.onRoute.join(',')}` : '路上没有棋子'; + const brush = p.route.grazes.length ? `${p.route.onRoute.length ? ',另外' : ',但'}紧贴 ${p.route.grazes.join(',')}(会擦到)` : ''; + if (!p.route.cells.length) return `${p.id} @ ${p.q},${p.r} ${p.arrow}:就在边上,直接出界(0 格)· 路上没有棋子${brush}`; + return `${p.id} @ ${p.q},${p.r} ${p.arrow}:${shown.join(' → ')}${tail} → ${end}(${p.route.cells.length} 格 · ${p.route.bends} 弯)· ${who}${brush}`; +} + +// `model` is the machine-readable id used for grouping; `agent` is whatever the +// agent says about itself — harness, mode, reasoning effort — shown as the +// banner over the board so a watcher knows who is playing. +export function createRun({ level, run, persona = 'default', model = 'unknown', agent = '', peek = false, budget = 80, startedAt = null }) { + if (!level?.cells || !level?.tiles) throw new Error('createRun needs a level with cells and tiles'); + return { version: 1, run, persona, model, agent: String(agent || '').slice(0, 120), peek, budget, startedAt, updatedAt: startedAt, status: 'playing', level: clone(level), steps: [], rating: null, reason: null }; +} + +// Rebuild the live game by replaying the recorded actions. +export function replay(runObj) { + const game = new Game(runObj.level); + for (const step of runObj.steps) { + if (step.action.type === 'tap') game.tap(step.action.id); + else if (step.action.type === 'rotate') game.rotate(step.action.direction); + else if (step.action.type === 'undo') game.undo(); + } + return game; +} + +function pairs(tracks) { + const out = []; + for (let a = 0; a < 6; a++) if (a < tracks[a]) out.push(`${a}-${tracks[a]}`); + return out.join(' '); +} + +export function observe(runObj, { format = 'text' } = {}) { + const game = replay(runObj); + const last = runObj.steps[runObj.steps.length - 1] || null; + const state = { + run: runObj.run, level: runObj.level.id, status: runObj.status, step: runObj.steps.length, budget: runObj.budget, + orientation: game.orientation, rotationsUsed: game.rotations, remaining: game.tiles.length, total: runObj.level.tiles.length, + pieces: pieceRoutes(game), + cells: [...game.level.cells].sort((a, b) => a.r - b.r || a.q - b.q).map(c => ({ q: c.q, r: c.r, pairs: pairs(c.tracks) })), + last: last ? { action: last.action, result: last.result } : null, + }; + if (format === 'json') return state; + const lines = []; + lines.push(`LEVEL ${state.level} · ${state.cells.length} 格 · 剩余 ${state.remaining}/${state.total} 枚 · 朝向 ${state.orientation} · 已用 ${state.step}/${state.budget} 步 · 状态 ${state.status}`); + if (state.last) lines.push(`上一步:${describeAction(state.last.action)} → ${describeResult(state.last.result)}`); + lines.push('每枚棋子当前的路线(箭头出发 → 依次经过的格 → 出界/闭环;[tX] = 那格上压着棋子 tX):'); + for (const p of state.pieces) lines.push(' ' + routeLine(p)); + if (format === 'full') { + lines.push('格子轨道(q,r: 端口配对 a-b;端口 0=↑ 1=↗ 2=↘ 3=↓ 4=↙ 5=↖):'); + let row = []; + for (const c of state.cells) { + row.push(`${c.q},${c.r}:${c.pairs.replace(/ /g, ',')}`); + if (row.length === 4) { lines.push(' ' + row.join(' ')); row = []; } + } + if (row.length) lines.push(' ' + row.join(' ')); + } + return lines.join('\n'); +} + +export function describeAction(action) { + if (action.type === 'tap') return `点击 ${action.id}`; + if (action.type === 'rotate') return action.direction === 1 ? '右转' : '左转'; + if (action.type === 'peek') return `查看 ${action.id}`; + if (action.type === 'undo') return '撤销'; + if (action.type === 'giveup') return '放弃'; + return action.type; +} +export function describeResult(result) { + if (!result) return ''; + if (result.status === 'exit') return `离场(走了 ${result.cells} 格、${result.bends} 个弯)${result.opened ? `,另有 ${result.opened} 枚棋子的路通了` : ''}`; + if (result.status === 'blocked') return `被 ${result.blocker} 挡住,退回原位`; + if (result.status === 'loop') return '路线绕回原处,没有出口'; + if (result.status === 'rotated') return `底盘转到朝向 ${result.orientation}${result.opened ? `,${result.opened} 枚棋子的出口打开了` : ',这个朝向没有新的出口'}`; + if (result.status === 'undone') return '已撤销上一步'; + if (result.status === 'peek') return result.route === 'exit' ? `它现在能离场:走 ${result.cells} 格、${result.bends} 个弯,经过 ${result.path}` : result.route === 'blocked' ? `它现在会在 ${result.path} 被 ${result.blocker} 挡住` : '它的路线绕回原处'; + if (result.status === 'cleared') return '全部离场,通关'; + if (result.status === 'rejected') return `无效:${result.reason}`; + return result.status; +} + +export function act(runObj, action) { + if (runObj.status !== 'playing') return reject(runObj, action, `本局已结束(${runObj.status})`); + if (runObj.steps.length >= runObj.budget) { runObj.status = 'budget'; return reject(runObj, action, '步数预算用完'); } + if (!ACTIONS.includes(action?.type)) return reject(runObj, action, `未知动作 ${action?.type},可用:${ACTIONS.join(' / ')}`); + if (action.type === 'peek' && !runObj.peek) return reject(runObj, action, '本局未开启 peek'); + const game = replay(runObj); + let result; + if (action.type === 'giveup') { + runObj.status = 'gave-up'; runObj.reason = action.reason || ''; + result = { status: 'gave-up', reason: runObj.reason }; + } else if (action.type === 'undo') { + result = game.undo() ? { status: 'undone' } : { status: 'rejected', reason: '没有可撤销的步骤' }; + if (result.status === 'rejected') return reject(runObj, action, result.reason); + } else if (action.type === 'rotate') { + if (![1, -1].includes(action.direction)) return reject(runObj, action, 'rotate 需要 direction 1(右)或 -1(左)'); + const before = new Set(available(game.level, game.tiles).map(t => t.id)); + const turned = game.rotate(action.direction); + if (turned.status !== 'rotated') return reject(runObj, action, turned.status); + const opened = available(game.level, game.tiles).filter(t => !before.has(t.id)).length; + result = { status: 'rotated', orientation: game.orientation, opened }; + } else { + const tile = game.tiles.find(t => t.id === action.id); + if (!tile) return reject(runObj, action, `没有棋子 ${action.id}`); + if (action.type === 'peek') { + const route = trace(game.level, tile, game.tiles); + const path = route.steps.map(s => `${s.q},${s.r}`).join('→'); + result = { status: 'peek', route: route.status, cells: route.steps.length, bends: countBends(route), blocker: route.blocker || null, path }; + } else { + const before = new Set(available(game.level, game.tiles).map(t => t.id)); + const route = game.tap(tile.id); + if (route.status === 'exit') { + const opened = available(game.level, game.tiles).filter(t => !before.has(t.id)).length; + result = { status: 'exit', cells: route.steps.length, bends: countBends(route), opened }; + } else result = { status: route.status, blocker: route.blocker || null }; + } + } + // Wall-clock stamp per step. Steps in one batch land microseconds apart, so + // the gap between batches is the agent's real decision latency. + const step = { n: runObj.steps.length + 1, action, note: action.note || '', result, remaining: game.tiles.length, orientation: game.orientation, batch: null, at: Date.now() }; + runObj.steps.push(step); + if (game.won) { runObj.status = 'cleared'; step.result = { ...result, cleared: true }; } + else if (runObj.steps.length >= runObj.budget && runObj.status === 'playing') runObj.status = 'budget'; + return { step, feedback: describeResult(step.result) + (game.won ? '。全部离场,通关。' : '') }; +} +function reject(runObj, action, reason) { + return { step: null, feedback: `无效:${reason}`, rejected: { action, reason } }; +} + +// Several actions in one decision, the way a person chains the obvious taps. +// The chain stops at the first surprise (a tap that does not leave) and after +// any turn — the board has to be re-read after either. +export function actMany(runObj, actions, { note = '' } = {}) { + const last = runObj.steps[runObj.steps.length - 1]; + const batch = (last?.batch || 0) + 1; + const outcomes = []; + let stopped = null; + for (let i = 0; i < actions.length; i++) { + const action = { ...actions[i], note: actions[i].note || note }; + const out = act(runObj, action); + if (out.step) out.step.batch = batch; + outcomes.push(out); + if (!out.step || runObj.status !== 'playing') break; + if (action.type === 'tap' && out.step.result.status !== 'exit' && i < actions.length - 1) { stopped = '这一步没成功,后面的动作没有执行'; break; } + if (action.type === 'rotate' && i < actions.length - 1) { stopped = '旋转后盘面已变,后面的动作没有执行'; break; } + } + return { outcomes, stopped, batch }; +} + +// One decision = one look at the board: a lone step, or a whole chained batch. +// Its latency is the wall-clock gap since the previous decision was committed, +// which for a shell agent includes its own thinking and process startup. +export function decisionTimeline(runObj) { + const groups = []; + for (const step of runObj.steps) { + const previous = groups[groups.length - 1]; + if (previous && step.batch != null && previous.batch === step.batch) { previous.steps.push(step); previous.at = step.at ?? previous.at; } + else groups.push({ batch: step.batch, steps: [step], at: step.at ?? null, from: step.n }); + } + let cursor = runObj.startedAt ? Date.parse(runObj.startedAt) : null; + return groups.map(group => { + const ms = group.at != null && cursor != null ? group.at - cursor : null; + if (group.at != null) cursor = group.at; + return { from: group.from, actions: group.steps.length, at: group.at, ms }; + }); +} + +export const RATING_QUESTIONS = [ + { key: 'clarity', text: '路线看得清吗(1 完全看不清 – 5 一眼看清)' }, + { key: 'rotation', text: '旋转有意义吗(1 随便转都行/没用 – 5 每次转都是关键决定)' }, + { key: 'stuck', text: '有没有觉得无解的时刻(1 从未 – 5 长时间以为无解)' }, + { key: 'fun', text: '好玩吗(1 – 5)' }, + { key: 'difficulty', text: '难度(1 太简单 – 5 太难)' }, + { key: 'worst', text: '最困惑的一步是第几步,为什么(文字)' }, +]; +export function rate(runObj, answers) { + const rating = {}; + for (const q of RATING_QUESTIONS) rating[q.key] = answers?.[q.key] ?? null; + runObj.rating = rating; + return rating; +} + +// Post-hoc metrics. The solver is allowed here — the run is over. +export function metrics(runObj) { + const game = new Game(runObj.level); + const solution = solveRotations(runObj.level); + const m = { run: runObj.run, level: runObj.level.id, persona: runObj.persona, model: runObj.model, agent: runObj.agent || '', peek: runObj.peek, status: runObj.status, + steps: runObj.steps.length, taps: 0, exits: 0, deadTaps: 0, grazeDeadTaps: 0, longestDeadStreak: 0, rotations: 0, effectiveRotations: 0, blindRotations: 0, + wrongWayRotations: 0, longestBlindStreak: 0, undos: 0, peeks: 0, stuckEntries: 0, stepsToFirstExit: null, + minimumRotations: solution.solvable ? solution.rotations : null, cleared: runObj.status === 'cleared', gaveUp: runObj.status === 'gave-up', rating: runObj.rating }; + let dead = 0, blind = 0; + for (const step of runObj.steps) { + const a = step.action; + if (a.type === 'tap') { + m.taps++; + const tile = game.tiles.find(t => t.id === a.id); + const geoCells = tile ? new Set(trace(game.level, tile, [], true).steps.map(s => `${s.q},${s.r}`)) : new Set(); + const route = game.tap(a.id); + if (route.status === 'exit') { m.exits++; dead = 0; blind = 0; if (m.stepsToFirstExit === null) m.stepsToFirstExit = step.n; } + else { + m.deadTaps++; dead++; m.longestDeadStreak = Math.max(m.longestDeadStreak, dead); + const blocker = route.blocker && game.tiles.find(t => t.id === route.blocker); + if (blocker && !geoCells.has(`${blocker.q},${blocker.r}`)) m.grazeDeadTaps++; + } + } else if (a.type === 'rotate') { + m.rotations++; + const other = available(rotateBoard(game.level, -a.direction), game.tiles).length; + const before = new Set(available(game.level, game.tiles).map(t => t.id)); + game.rotate(a.direction); + const opened = available(game.level, game.tiles).filter(t => !before.has(t.id)).length; + if (opened) { m.effectiveRotations++; blind = 0; } + else { m.blindRotations++; blind++; m.longestBlindStreak = Math.max(m.longestBlindStreak, blind); if (other) m.wrongWayRotations++; } + } else if (a.type === 'undo') { m.undos++; game.undo(); } + else if (a.type === 'peek') m.peeks++; + if (!game.won && a.type !== 'peek' && !available(game.level, game.tiles).length) m.stuckEntries++; + } + m.rotationEfficiency = m.rotations ? m.effectiveRotations / m.rotations : null; + m.actionsPerPiece = runObj.level.tiles.length ? m.steps / runObj.level.tiles.length : null; + const timeline = decisionTimeline(runObj); + m.decisions = timeline.length; + m.actionsPerDecision = timeline.length ? m.steps / timeline.length : null; + const latencies = timeline.map(d => d.ms).filter(ms => typeof ms === 'number' && ms >= 0); + const sorted = [...latencies].sort((a, b) => a - b); + const at = p => sorted.length ? sorted[Math.min(sorted.length - 1, Math.floor(sorted.length * p))] : null; + const lastAt = runObj.steps.length ? runObj.steps[runObj.steps.length - 1].at : null; + m.timing = { + totalMs: lastAt != null && runObj.startedAt ? lastAt - Date.parse(runObj.startedAt) : null, + firstDecisionMs: latencies.length ? latencies[0] : null, + medianDecisionMs: at(.5), p90DecisionMs: at(.9), maxDecisionMs: sorted.length ? sorted[sorted.length - 1] : null, + // Pure model latency, when the direct-API loop recorded it. + thinkMs: runObj.timing?.decisions?.length ? runObj.timing.decisions.reduce((s, d) => s + d.thinkMs, 0) : null, + }; + return m; +} diff --git a/tests/bench.test.js b/tests/bench.test.js new file mode 100644 index 0000000..214b1cf --- /dev/null +++ b/tests/bench.test.js @@ -0,0 +1,153 @@ +import test from 'node:test'; +import assert from 'node:assert/strict'; +import { hexagon } from '../src/core.js'; +import { createRun, observe, act, actMany, rate, metrics, replay, rulesText, decisionTimeline } from '../src/bench.js'; + +// Straight rails on a 19-cell board: t1 sits on t0's route, so t0 is blocked +// until t1 leaves. Turning the board swaps which rails the pieces sit on. +function fixture() { + return { + version: 1, id: 'bench-fixture', name: 'fixture', cells: hexagon(2), rotationEnabled: true, moveLimit: 2, + tiles: [{ id: 't0', q: 0, r: 0, dir: 0, color: 0 }, { id: 't1', q: 0, r: -1, dir: 0, color: 1 }], + }; +} + +test('observation exposes board and pieces but never which pieces can leave', () => { + const run = createRun({ level: fixture(), run: 'obs', budget: 10 }); + const json = observe(run, { format: 'json' }); + assert.equal(json.remaining, 2); + assert.equal(json.cells.length, 19); + assert.deepEqual(Object.keys(json.pieces[0]).sort(), ['arrow', 'color', 'dir', 'id', 'q', 'r', 'route']); + // Routes are the drawn line a person sees: geometric, with pieces sitting on it marked. + const t0 = json.pieces.find(p => p.id === 't0'), t1 = json.pieces.find(p => p.id === 't1'); + assert.equal(t0.route.end, 'exit'); + assert.deepEqual(t0.route.onRoute, ['t1'], 't1 sits on t0\'s line'); + assert.deepEqual(t1.route.onRoute, []); + assert.deepEqual(t0.route.grazes, [], 'a blocker on the line is not a graze'); + assert.deepEqual(t1.route.grazes, [], 'nothing brushes a clear straight line'); + assert.ok(!JSON.stringify(json).includes('available')); + assert.ok(!JSON.stringify(json).includes('solvable')); + const text = observe(run); + assert.match(text, /剩余 2\/2 枚/); + assert.match(text, /t0 @ 0,0 ↑:0,-1\[t1\] → 0,-2 → 出界(2 格 · 0 弯)· 路上有 t1/); + assert.match(text, /t1 @ 0,-1 ↑:0,-2 → 出界(1 格 · 0 弯)· 路上没有棋子/); + assert.doesNotMatch(text, /端口配对/, 'the rail table is opt-in'); + assert.match(observe(run, { format: 'full' }), /端口配对/); + assert.match(rulesText(false), /怎么玩/); + assert.doesNotMatch(rulesText(false), /peek/); + assert.match(rulesText(true), /peek/); +}); + +test('actions go through the real game: blocked, exit with unlock, cleared', () => { + const run = createRun({ level: fixture(), run: 'play', budget: 10 }); + let out = act(run, { type: 'tap', id: 't0', note: '先试最里面的' }); + assert.equal(out.step.result.status, 'blocked'); + assert.equal(out.step.result.blocker, 't1'); + assert.match(out.feedback, /被 t1 挡住/); + out = act(run, { type: 'tap', id: 't1' }); + assert.equal(out.step.result.status, 'exit'); + assert.equal(out.step.result.opened, 1, 'removing the gate opens t0'); + out = act(run, { type: 'tap', id: 't0' }); + assert.equal(out.step.result.status, 'exit'); + assert.equal(run.status, 'cleared'); + assert.equal(replay(run).won, true); + assert.match(act(run, { type: 'tap', id: 't0' }).feedback, /已结束/); + const m = metrics(run); + assert.equal(m.taps, 3); assert.equal(m.exits, 2); assert.equal(m.deadTaps, 1); + assert.equal(m.longestDeadStreak, 1); assert.equal(m.stepsToFirstExit, 2); assert.equal(m.cleared, true); + assert.equal(run.steps[0].note, '先试最里面的'); +}); + +test('rotate reports how many pieces opened, undo restores, peek is gated', () => { + const run = createRun({ level: fixture(), run: 'turn', budget: 10 }); + const turned = act(run, { type: 'rotate', direction: 1 }); + assert.equal(turned.step.result.status, 'rotated'); + assert.equal(turned.step.result.orientation, 1); + assert.equal(typeof turned.step.result.opened, 'number'); + assert.equal(observe(run, { format: 'json' }).orientation, 1); + assert.equal(act(run, { type: 'undo' }).step.result.status, 'undone'); + assert.equal(observe(run, { format: 'json' }).orientation, 0); + assert.match(act(run, { type: 'peek', id: 't0' }).feedback, /未开启 peek/); + assert.match(act(run, { type: 'rotate', direction: 2 }).feedback, /direction/); + const peeking = createRun({ level: fixture(), run: 'peek', peek: true, budget: 10 }); + const seen = act(peeking, { type: 'peek', id: 't0' }); + assert.equal(seen.step.result.status, 'peek'); + assert.equal(seen.step.result.route, 'blocked'); + assert.equal(seen.step.result.blocker, 't1'); + assert.equal(metrics(peeking).peeks, 1); +}); + +test('a chained batch stops at the first failed tap and after a turn', () => { + const run = createRun({ level: fixture(), run: 'chain', budget: 20 }); + // t0 is blocked by t1: the chain must stop there and not tap t1. + let batch = actMany(run, [{ type: 'tap', id: 't0' }, { type: 'tap', id: 't1' }], { note: '连点试试' }); + assert.equal(batch.outcomes.length, 1); + assert.equal(batch.outcomes[0].step.result.status, 'blocked'); + assert.match(batch.stopped, /没成功/); + assert.equal(run.steps.length, 1); + assert.equal(run.steps[0].batch, 1); + assert.equal(run.steps[0].note, '连点试试'); + // A turn ends the chain even when it succeeds. + batch = actMany(run, [{ type: 'rotate', direction: 1 }, { type: 'tap', id: 't1' }]); + assert.equal(batch.outcomes.length, 1); + assert.match(batch.stopped, /旋转后/); + actMany(run, [{ type: 'undo' }]); + // The good order clears in one decision. + batch = actMany(run, [{ type: 'tap', id: 't1' }, { type: 'tap', id: 't0' }]); + assert.equal(batch.outcomes.length, 2); + assert.equal(batch.stopped, null); + assert.equal(run.status, 'cleared'); + const m = metrics(run); + assert.equal(m.steps, 5); + assert.equal(m.decisions, 4, 'four looks at the board: blocked chain, turn, undo, clearing chain'); +}); + +test('the agent self-reports its identity, and it survives into metrics', () => { + const run = createRun({ level: fixture(), run: 'who', model: 'claude-opus-5', agent: 'Claude Code · ultracode', budget: 5 }); + assert.equal(run.model, 'claude-opus-5'); + assert.equal(run.agent, 'Claude Code · ultracode'); + assert.equal(metrics(run).agent, 'Claude Code · ultracode'); + // Missing is fine — the banner falls back to the model alone. + assert.equal(createRun({ level: fixture(), run: 'anon' }).agent, ''); + // A runaway self-description must not become an unbounded banner. + assert.equal(createRun({ level: fixture(), run: 'long', agent: 'x'.repeat(500) }).agent.length, 120); +}); + +test('timing groups a batch into one decision and measures the gap before it', () => { + const run = createRun({ level: fixture(), run: 'timing', budget: 20, startedAt: new Date(1000).toISOString() }); + actMany(run, [{ type: 'tap', id: 't1' }, { type: 'tap', id: 't0' }]); + // Stamps come from the clock; rewrite them so the assertions are exact. + run.steps[0].at = 3000; run.steps[1].at = 3010; + const timeline = decisionTimeline(run); + assert.equal(timeline.length, 1, 'a chained batch is one decision'); + assert.equal(timeline[0].actions, 2); + assert.equal(timeline[0].from, 1); + assert.equal(timeline[0].ms, 2010, 'latency runs from startedAt to the batch commit'); + const m = metrics(run); + assert.equal(m.decisions, 1); + assert.equal(m.actionsPerDecision, 2); + assert.equal(m.timing.totalMs, 2010); + assert.equal(m.timing.medianDecisionMs, 2010); + assert.equal(m.timing.thinkMs, null, 'only the direct-API loop records model latency'); + // A run recorded before timestamps existed must not crash or invent numbers. + const legacy = createRun({ level: fixture(), run: 'legacy', budget: 20 }); + act(legacy, { type: 'tap', id: 't1' }); + delete legacy.steps[0].at; + assert.equal(metrics(legacy).timing.totalMs, null); + assert.equal(decisionTimeline(legacy)[0].ms, null); +}); + +test('budget and give-up end the run; rating is recorded', () => { + const run = createRun({ level: fixture(), run: 'budget', budget: 1 }); + act(run, { type: 'tap', id: 't0' }); + assert.equal(run.status, 'budget'); + assert.match(act(run, { type: 'tap', id: 't1' }).feedback, /已结束/); + const quit = createRun({ level: fixture(), run: 'quit', budget: 5 }); + act(quit, { type: 'giveup', reason: '看起来无解' }); + assert.equal(quit.status, 'gave-up'); + assert.equal(quit.reason, '看起来无解'); + assert.equal(metrics(quit).gaveUp, true); + const rating = rate(quit, { clarity: 2, stuck: 5, worst: '第 1 步' }); + assert.equal(rating.clarity, 2); assert.equal(rating.fun, null); + assert.equal(metrics(quit).rating.stuck, 5); +});