diff --git a/.claude/launch.json b/.claude/launch.json new file mode 100644 index 0000000..04ed786 --- /dev/null +++ b/.claude/launch.json @@ -0,0 +1,14 @@ +{ + "version": "0.0.1", + "configurations": [ + { + "name": "dev", + "runtimeExecutable": "npm", + "runtimeArgs": [ + "run", + "dev" + ], + "port": 3000 + } + ] +} diff --git a/labs/bench/README.md b/labs/bench/README.md new file mode 100644 index 0000000..7b78069 --- /dev/null +++ b/labs/bench/README.md @@ -0,0 +1,180 @@ +# 关卡实验台（Agent Bench） + +让任意 Agent **像人一样**玩关卡：只看盘面、只用 tap / rotate / undo（可选 peek）与棋盘交互，每一步写理由，结束后评分。你在观察页看它怎么玩，就能判断关卡对不对；多个 Agent 跑完再看汇总。 + +它不是求解器，也不是真人测试。用途是：**抓"体感无解"、找卡点、给关卡做相对排序**。绝对难度仍要少量真人校准（见文末）。 + +## 架构：一个 JSON 文件，三种接法 + +``` +Agent ──(shell)──▶ scripts/bench.mjs ──写──▶ bench/runs/<局>.json ◀──轮询── labs/bench/index.html（观察页） +Agent ──(API)───▶ scripts/bench-agent.mjs ──写──▶ 同上 +``` + +- `src/bench.js` 会话核心：`observe()` 给的是人眼在屏幕上看到的东西——**每枚棋子当前的路线**（依次经过哪些格、在哪里出界或闭环、路上压着谁 `[tX]`），加朝向和剩余数；**不给求解器，不给旋转后的预测**。它没有替 Agent 做的事：决定先后顺序、判断何时旋转和转哪边、发现擦碰（棋子有宽度，路线上没人也可能被邻格的棋子挡——反馈会告诉它被谁挡）。`act()` 经过真实的 `Game` 规则；`metrics()` 事后用求解器算指标。 +- 第一版观察只给每格的端口配对表，Haiku 只能靠坐标运算读路，结果是盲猜、连边上的棋子都不点。改成路线之后它 18 步清盘、3 次旋转全部有效，但 3 次空点全是擦碰——路线上没人，却被紧邻的棋子挡。擦碰不在格子数据里（它来自轨道曲线和棋子宽度的扫掠判定），Agent 推不出来；而人看得见线从棋子身边擦过。所以把它作为路线标记直接给出：`紧贴 tX（会擦到）`，用真实碰撞判定算，不用邻格猜。 +- **一次只报第一个阻挡者是不够的。** 最初 `graze` 只取扫掠碰到的第一个障碍，路上压着一枚棋子时，它身后的擦碰就被吞掉——全库 581 例「路上有阻挡」里有 113 例（约两成）在清掉挡路者之后会冒出从未提示过的擦碰。grok-4.6 玩 18 关的 4 次空点全部由此而来，它自己写的是「路线描述没提前写出这次擦碰」。现在 `route.grazes` 是数组，逐枚单独做扫掠判定，谁也遮不住谁，复测 0 例漏报。盘面读作 `路上有 t6，另外紧贴 t4（会擦到）`，含义是**列出来的全部清掉这枚才能走**。 +- 因此 `grazeDeadTaps` 的语义在修复前后不同：修复前测的是「系统没告诉它」，修复后测的是「它无视了提示」。跨这条分界线比较历史数据时要注意——按 `agent` 字段区分批次。 +- [`labs/bench/AGENT.md`](AGENT.md)：**给 Agent 读的唯一文件**——规则、读盘、命令、评分、示例都在里面。 +- `scripts/bench.mjs` 命令行：任何有 shell 的 Agent（Codex、Claude Code、Grok Build、人）一条命令一步。`start` 自动生成局名、自动起服务、打印观察页地址。 +- `scripts/bench-agent.mjs` 参考循环：任何只有 API 的模型（含弱模型、本地模型），纯文本进、一行 JSON 出，不需要 tool-use。 +- `labs/bench/`：观察页，用主游戏的渲染和动画重放每一步，可直播。 +- `scripts/bench-report.mjs`：按关汇总。 + +没有服务端，没有依赖。`npm run dev` 的静态服务器已经能提供 `bench/runs/*.json`。 + +## 最短路径：给 Agent 一句话 + +在仓库根目录启动任意 shell agent（Claude Code / Codex / Grok Build / DeepSeek Harness…），说一句： + +``` +读 labs/bench/AGENT.md，然后按它说的玩第 1 关。 +``` + +[`AGENT.md`](AGENT.md) 是**唯一**要给 Agent 的文件：规则、读盘方法、命令、评分表、完整示例都在里面。它读完就能开始；`start` 会自动生成局名、自动起本地服务、把观察页地址打印出来让它转告你。人类这边只要打开那个地址看着就行。 + +想换关就说"玩第 3 关"，想换风格就说"用 reader 人设"，想跑正式千关就说"加 `--set=levels/v0.5.json`"。 + +--- + +## 完整路径 + +按"你手上有什么"选： + +| 路径 | 需要什么 | 单局速度 | 用途 | +|---|---|---|---| +| **A. 规则机器人** | 什么都不需要 | 毫秒级 | 千关粗筛：规则能不能照着清、哪些关盲转多 | +| **B. Claude Code / Codex**（shell agent） | 订阅 | 精简协议 + 连点后约 1 分钟 | 看它怎么玩、评分、抓"体感无解" | +| C/D. 别的 harness / 便携包 | 对方能跑 Node | 同 B | Grok Build、DeepSeek Harness、任何沙箱 | +| E. 直连 API | API key | 每次决策 3–10 秒 | 有 key 时的批量主路径 | + +### A. 规则机器人：先把 1000 关扫一遍 + +```bash +BENCH_PROVIDER=mock node scripts/bench-batch.mjs --set=levels/v0.5.json --levels=all --concurrency=8 --prefix=bot-v05 +node scripts/bench-report.mjs +``` + +它只做协议里写的四条规则：路上没人不紧贴谁就点、否则右转。18 关 D5 原型 <1 秒跑完。它不评分，但它的**盲转数**是个便宜的结构探针——一路右转就能清的关和必须转回来的关，一眼分开。 + +### B. Claude Code / Codex：看它玩 + +单局——对 agent 说一句： + +``` +读 labs/bench/AGENT.md，然后按它说的玩第 1 关。 +``` + +多局并行——用非交互模式开多个进程，各玩一关： + +```bash +for L in 1 2 3 4 5 6; do + claude -p "读 labs/bench/AGENT.md，然后按它说的玩第 $L 关，--model 写 claude。" \ + --allowedTools "Read,Bash(node scripts/bench.mjs*)" & +done; wait +node scripts/bench-index.mjs && node scripts/bench-report.mjs --by=model +``` + +Codex 同理（`codex exec "…"`）。观察页 `刷新列表` 能看到正在跑的局；`?run=<局名>&live=1` 直播。 + +提速要点（已经做进 AGENT.md 和 CLI 里）：`AGENT.md` 一个文件读完就能开始；`start --no-rules` 不重复打印规则、自动生成局名、自动起服务；`act` 用逗号连点，一次看盘做一串。 + +实测（Opus 5，第 1 关，含子代理启动与读盘）：逐步 15 次决策 154 s → 连点 4 次决策 93 s → 精简协议 4 次决策 **82 s**。`claude -p` 非交互模式第 2 关：8 次决策 **76 s**。剩余时间大头是进程启动和第一次读盘（约 20–30 s）；每次决策约 8–10 s。要更快只能并行：`claude -p` 开 6 个进程，18 关约 4 分钟。 + +### C. 别的 agent harness（Grok Build / DeepSeek Harness / Codex …） + +**Grok Build（`grok`）和 DeepSeek Harness（`dsh`）都是本机 CLI，能执行 bash、能读写本地文件**，所以它们和 Claude Code 走同一条路：在仓库根目录启动，把 B 节那段提示给它。区别只是提示词里 `--model=` 写它自己的名字。 + +```bash +# Grok Build：curl -fsSL https://x.ai/cli/install.sh | bash +grok # 然后贴 B 节的提示；注意它默认不开沙箱，需要时用 workspace profile +# DeepSeek Harness：npx @deepseek-ai/dsh web （developer preview，接口会变） +``` + +如果对方拿不到这个仓库（另一台机器、另一个人、只能在自己沙箱里跑的 harness），发**独立实验台**给他（下一节）。 + +### D. 独立实验台：发给任何人 + +```bash +node scripts/bench-kit.mjs --zip # → bench/hex-bench/ 和 bench/hex-bench.zip +``` + +生成一个**自带观察页**的独立工程：`bench.mjs`（命令行）、`serve.mjs`（本地服务）、`report.mjs`（汇总）、`observer/`（观察页）、`src/`（引擎，按 import 自动追踪，不会漏）、`levels.json`、`AGENT.md`、`README.md`。约 1.5 MB，**只要 Node 20+，不联网、不装依赖、不需要原仓库**。 + +对方那边三步：`node serve.mjs` → 对 agent 说「读 `AGENT.md`，按它说的玩第 1 关」→ 打开 看着。包里的 `README.md` 就是写给这个人看的。 + +构建时会在生成的包里真跑一遍 list → start → act → rate → report 并检查观察页的依赖齐全，所以不会发出一个跑不起来的包。每一处路径改写都有断言：仓库里改了名字，构建会直接报错而不是悄悄发坏包。 + +玩完让对方把 `runs/` 发回来： + +```bash +node scripts/bench-import.mjs --from=<路径>/runs --tag=grok +node scripts/bench-report.mjs --by=model +``` + +`--tag` 会给局名加前缀，不同来源不会撞名。导入后在观察页照常回放。 + +（`scripts/bench-pack.mjs` 是它的前身——只有 7 个文件、没有观察页。已被 `bench-kit` 完全取代，可以删掉。） + +### E. 直连 API（有 key 时最快） + +不需要 shell、不需要 tool-use，模型每轮输出一行 JSON 或一个数组。预设了四个端点： + +```bash +BENCH_PROVIDER=anthropic ANTHROPIC_API_KEY=… node scripts/bench-batch.mjs --levels=1-18 --personas=tapper,reader --concurrency=6 +BENCH_PROVIDER=deepseek DEEPSEEK_API_KEY=… node scripts/bench-batch.mjs --levels=1-18 --model=deepseek-flash +BENCH_PROVIDER=xai XAI_API_KEY=… node scripts/bench-batch.mjs --levels=1-18 --model=grok-4.6 +BENCH_PROVIDER=openai OPENAI_API_KEY=… node scripts/bench-batch.mjs --levels=1-18 --model=gpt-4o-mini +BENCH_PROVIDER=openai BENCH_BASE_URL=http://localhost:11434/v1 node scripts/bench-batch.mjs --levels=1-18 --model=qwen2.5:7b # Ollama / vLLM +node scripts/bench-agent.mjs --run=x --level=1 --dry # 只看提示，不联网 +``` + +| 预设 | base URL | 环境变量 | 默认模型 | +|---|---|---|---| +| `anthropic` | `https://api.anthropic.com` | `ANTHROPIC_API_KEY` | `claude-sonnet-5` | +| `deepseek` | `https://api.deepseek.com` | `DEEPSEEK_API_KEY` | `deepseek-flash` | +| `xai` | `https://api.x.ai/v1` | `XAI_API_KEY` | `grok-4.6` | +| `openai` | `https://api.openai.com/v1` | `OPENAI_API_KEY` | `gpt-4o-mini` | + +模型标识符会变，用 `--model` 覆盖（例如 DeepSeek 的 `deepseek-chat` / `deepseek-reasoner` 已于 2026-07-24 下线，现在是 `deepseek-flash` / `deepseek-v4-pro`）。请求体只用 `model` / `messages` / `max_tokens` / `temperature`——DeepSeek 不接受 `frequency_penalty` 和 `presence_penalty`。 + +这条路径的**网络部分本仓库没有 key，未实测**；循环、解析、连点、provider 路由、索引重建都用规则机器人跑通了。 + +## 速度：连点 + +一步一个 LLM 往返是最大的开销（一局 15 步约 2.5 分钟，人玩这关不到 1 分钟）。解决办法和人一样——**看清一次盘面就连着点几枚**： + +```bash +node scripts/bench.mjs act --run=x tap t4, tap t5, tap t8, rotate right --note="三枚边上的，清完就转" +``` + +链在第一个没成功的动作和任何一次旋转之后停下并返回新盘面，所以它不会"闭眼连点"到错的地方。API 循环里对应输出一个 JSON 数组。指标里 `decisions`（看盘次数）和 `steps`（动作数）分开记：同样 15 步，连点大约 5–6 次决策。 + +## 怎么看 + +```bash +npm run dev +``` + +- 回放：`http://localhost:4173/labs/bench/index.html?run=<局名>`——下拉选局，点右侧实况里的任意一步即可回看那一刻的棋盘，「↩ 回到实时」跟回 Agent。右侧竖排是计时板与 Agent 每一步的理由，指标和评分在页面下方。棋盘上方的横幅显示当前是谁在玩（`--model` / `--agent`）。 +- 直播：勾选「直播」或 URL 加 `&live=1`，页面每秒读一次文件，Agent 每走一步这里就动一步。 +- 汇总：`node scripts/bench-report.mjs`（写 `bench/report.json`）。带标记的关（有放弃 / 有人觉得无解 / 盲转 ≥2）先看。 + +## 指标（事后算，Agent 看不到） + +| 指标 | 含义 | 对应 §08 | +|---|---|---| +| 单局耗时 / 每次决策 | 墙钟时间，含 agent 自己的进程启动；`decisions` 与 `steps` 分开记 | — | +| 动作 ÷ 棋子 | 效率；猴子玩家全库 1.67 | — | +| 最长连续空点 | 点了没动的最长连续次数；某关中位 ≥3 = 可读性问题 | 连续空点 | +| 盲转 / 转错方向 | 一次旋转什么都没打开 / 另一边本来能打开 | 卡住停留、旋转效率 | +| 进入卡住状态 | 动作后无棋可走的次数 | — | +| 多转 | 清盘用的旋转数 − 最少旋转数 | 旋转效率 | +| 放弃 + stuck 评分 | "体感无解"的直接信号 | 逐关会话终止率 | + +## 边界 + +- Agent 不是人：推理贵、感知弱。用它做**相对**排序和卡点检测；不要把 clear rate 当真人通关率。 +- 观察是结构化文本，不是像素。它测不出"看不清"。要测可读性，下一步是纯截图模式（让模型看图点坐标），本仓库尚未实现。 +- 没有动画时长、没有连点队列：一步一步来，等价于一个不急的玩家。 +- 校准：同一批关让 5–10 个真人玩一次，把真人的空点、放弃、评分与 Agent 汇总对照，找出 Agent 系统性偏差，再用 Agent 去筛更多关。 diff --git a/labs/bench/app.js b/labs/bench/app.js new file mode 100644 index 0000000..e9e8772 --- /dev/null +++ b/labs/bench/app.js @@ -0,0 +1,331 @@ +import { Game, trace } from '../../src/core.js'; +import { metrics, decisionTimeline, describeAction, describeResult, RATING_QUESTIONS } from '../../src/bench.js'; +import { layout } from '../../src/geometry.js'; +import { CanvasPainter, drawScene } from '../../src/renderer.js'; +import { TILE_ATLAS } from '../../src/sprites.js'; +import { exitTiming } from '../../src/exit-animation.js'; +import { MOTION } from '../../src/design-tokens.js'; + +// Where the run files and the tile atlas live, relative to this page. The +// standalone kit builder (scripts/bench-kit.mjs) rewrites these two lines, so +// keep them as plain string literals. +const DATA = '../../bench/runs', ASSET_BASE = '../../'; + +const $ = id => document.getElementById(id); +const canvas = $('board'), context = canvas.getContext('2d'), painter = new CanvasPainter(context); +const reduced = matchMedia('(prefers-reduced-motion: reduce)').matches; +let runObj = null, game = null, cursor = 0, view, width, height, animation = null, turning = null, hint = null, path = null, frame = 0, liveTimer = null, clockFrame = 0; + +// ---------- data ---------- +// Data always refreshes; the checkbox only decides whether the cursor chases +// the newest step. Hiding the refresh behind a toggle meant a page opened from +// the dropdown sat at 0/0 while the agent played. +const following = () => $('live').checked; +// The board column carries no running commentary — the feed already says what +// happened, step by step. This line exists only for "the file is gone". +function notice(text) { const n = $('notice'); n.textContent = text || ''; n.hidden = !text; } +async function loadIndex({ keepSelection = false } = {}) { + const res = await fetch(`${DATA}/index.json?t=` + Date.now()).catch(() => null); + let index = []; + try { index = res && res.ok ? await res.json() : []; } catch { return null; } + index.sort((a, b) => (b.updatedAt || '').localeCompare(a.updatedAt || '')); + const markup = index.length ? index.map(e => ``).join('') : ''; + // Rebuilding the list on every poll would fight the open dropdown; only + // touch it when the options actually changed. + if (markup !== $('run').innerHTML) { + const selected = keepSelection ? $('run').value : null; + $('run').innerHTML = markup; + if (selected && index.some(e => e.run === selected)) $('run').value = selected; + } + return index; +} +let missing = 0; +async function loadRun(name, { keepCursor = false, fromStart = false } = {}) { + let fresh; + try { + const res = await fetch(`${DATA}/${name}.json?t=` + Date.now()); + if (!res.ok) { + // A stale index entry would otherwise 404 once a second forever. + // Drop the board with the data: leaving `game` behind meant the next + // resize redrew a run that no longer exists and read `runObj.steps`. + if (++missing >= 3) { runObj = null; game = null; view = null; missing = 0; notice(`${DATA}/${name}.json 不见了（可能已删除）——从列表里另选一局。`); await loadIndex(); } + else if (!keepCursor) notice(`读不到 ${DATA}/${name}.json`); + return; + } + // The agent may be mid-write; a torn read just means "try again next tick". + fresh = await res.json(); + missing = 0; notice(''); + } catch { return; } + const sameRun = runObj && runObj.run === fresh.run; + const changed = !sameRun || runObj.steps.length !== fresh.steps.length || runObj.status !== fresh.status || runObj.rating !== fresh.rating; + runObj = fresh; + if (changed) { renderSteps(); renderMetrics(); } + // Hopping to a fresh run: watch it from the opening so the animations play. + if (!sameRun || !keepCursor) rebuild(fromStart ? 0 : runObj.steps.length); + else pump(); + // Keep the dropdown and the address bar on the run being watched. An auto-hop + // lands on a run the select may not list yet, and a silent mismatch there + // labels the board with somebody else's game. + if (!sameRun) { + if (![...$('run').options].some(o => o.value === runObj.run)) await loadIndex(); + $('run').value = runObj.run; + const url = new URL(location.href); url.searchParams.set('run', runObj.run); history.replaceState(null, '', url); + } + syncUi(); + cancelAnimationFrame(clockFrame); paintClock(); + if (fromStart) pump(); +} +// Following the agent: keep playing whatever steps have arrived until the +// cursor is caught up. draw() calls back here when an animation finishes, so a +// batch of several steps plays through instead of stalling after one. +function pump() { + if (!following() || !runObj) return; + if (animation || turning) return; + if (cursor < runObj.steps.length) advance(); +} + +// ---------- replay ---------- +function rebuild(n) { + cancelAnimationFrame(frame); animation = null; turning = null; hint = null; path = null; + game = new Game(runObj.level); cursor = 0; + for (; cursor < n && cursor < runObj.steps.length; cursor++) applyModel(runObj.steps[cursor]); + resize(); syncUi(); +} +function applyModel(step) { + const a = step.action; + if (a.type === 'tap') game.tap(a.id); + else if (a.type === 'rotate') game.rotate(a.direction); + else if (a.type === 'undo') game.undo(); +} +// Advance one step with the real animations. +function advance() { + if (!runObj || cursor >= runObj.steps.length || animation || turning) return; + const step = runObj.steps[cursor]; const a = step.action; + hint = null; path = null; + if (a.type === 'tap') { + const result = game.tap(a.id); + if (result.status === 'exit') animation = { route: result, start: performance.now(), duration: reduced ? 100 : exitTiming(result).duration }; + else if (result.travelPoints) animation = { route: result, start: performance.now(), duration: reduced ? 150 : MOTION.collisionMs }; + } else if (a.type === 'rotate') { + game.rotate(a.direction); + turning = { direction: a.direction, start: performance.now(), duration: reduced ? 100 : MOTION.rotationMs }; + } else if (a.type === 'undo') game.undo(); + else if (a.type === 'peek') { const tile = game.tiles.find(t => t.id === a.id); if (tile) { hint = tile.id; path = trace(game.level, tile, game.tiles); } } + cursor++; + syncUi(); draw(); +} +function syncUi() { + if (!game || !runObj) return; + const behind = runObj.steps.length - cursor; + $('feed-state').textContent = !following() ? '已暂停跟随' : pinned ? '已固定这一局' : runObj.status === 'playing' ? (behind > 0 ? `追 ${behind} 步` : '等待下一步') : '本局已结束'; + $('feed-state').style.color = following() && runObj.status === 'playing' ? 'var(--live-ink)' : 'var(--ink3)'; + $('who-model').textContent = runObj.model || 'unknown'; + $('who-agent').textContent = runObj.agent || ''; + $('who-level').textContent = `${runObj.level.id} · ${runObj.level.cells.length} 格 · ${runObj.level.tiles.length} 枚 · 人设 ${runObj.persona}`; + // The jump-back button is the only replay control left: it appears exactly + // when the view is not tracking the agent's live frontier, and says which + // part of that it will resume — the newest step, or the newest run. + const drifted = !following(), locked = pinned && following(); + $('end').hidden = !drifted && !locked; + $('end').textContent = pinned ? '↩ 跟回最新一局' : '↩ 回到实时'; + document.querySelectorAll('#steps li').forEach((li, i) => li.classList.toggle('current', i === cursor - 1)); + scrollFeedTo(document.querySelector('#steps li.current')); + renderScoreboard(); +} + +// Scroll the feed, never the page. scrollIntoView({block:'nearest'}) walks up +// every scroll container, so on a phone — where the feed sits below the fold — +// each new step yanked the clock off the top of the screen. +function scrollFeedTo(li) { + if (!li) return; + const box = $('steps'), b = box.getBoundingClientRect(), r = li.getBoundingClientRect(); + if (r.top < b.top) box.scrollTop += r.top - b.top; + else if (r.bottom > b.bottom) box.scrollTop += r.bottom - b.bottom; +} + +// ---------- scoreboard ---------- +const STATUS = { playing: ['进行中', 'playing'], cleared: ['通关', 'cleared'], 'gave-up': ['放弃', 'gaveup'], budget: ['预算用尽', 'budget'] }; +const pad = (n, w = 2) => String(n).padStart(w, '0'); +// A stopwatch, not a duration: minutes:seconds.milliseconds, always the same width. +function stopwatch(ms) { + if (ms == null || ms < 0) return '0:00.000'; + const m = Math.floor(ms / 60000), s = Math.floor(ms % 60000 / 1000); + return `${m}:${pad(s)}.${pad(Math.floor(ms % 1000), 3)}`; +} +// metrics() re-solves the level, which is far too heavy to run on every clock +// frame; the numbers only change when a step lands. +let metricsCache = { key: null, value: null }; +function runMetrics() { + const key = `${runObj.run}:${runObj.steps.length}:${runObj.status}:${runObj.rating ? 1 : 0}`; + if (metricsCache.key !== key) metricsCache = { key, value: metrics(runObj) }; + return metricsCache.value; +} +function renderScoreboard() { + if (!runObj) return; + const m = runMetrics(), total = runObj.level.tiles.length; + const last = runObj.steps[runObj.steps.length - 1]; + const left = last ? last.remaining : total; + // Rewound, every number here still describes the whole run while the board + // beside it shows step N. Label the mismatch rather than let them contradict. + const rewound = !following() && cursor < runObj.steps.length; + const [label, cls] = rewound + ? [`回看 ${cursor} / ${runObj.steps.length}`, 'rewound'] + : STATUS[runObj.status] || [runObj.status, '']; + document.querySelector('.scoreboard').classList.toggle('rewound', rewound); + $('sb-steps').textContent = m.steps; + $('sb-budget').textContent = `/ ${runObj.budget}`; + $('sb-decisions').textContent = m.decisions; + $('sb-per').textContent = m.actionsPerDecision ? `· ${m.actionsPerDecision.toFixed(1)} 步/次` : '—'; + $('sb-cleared').textContent = total - left; + $('sb-total').textContent = `/ ${total}`; + $('sb-bar').style.width = `${Math.round((total - left) / Math.max(1, total) * 100)}%`; + $('sb-dead').textContent = m.deadTaps; + $('sb-rot').textContent = `· ${m.rotations} 转${m.minimumRotations != null ? ` / 最少 ${m.minimumRotations}` : ''}`; + $('sb-status').textContent = label; + $('sb-status').className = `pill ${cls}`; +} +// The clock is the point of the page: it runs while the agent is still playing, +// and freezes on the final time the moment the run ends. +function paintClock() { + clockFrame = 0; + if (!runObj) return; + const started = runObj.startedAt ? Date.parse(runObj.startedAt) : null; + const last = runObj.steps[runObj.steps.length - 1]; + const lastAt = last?.at ?? started; + const live = runObj.status === 'playing'; + const now = Date.now(); + $('clock').textContent = stopwatch(started == null ? null : (live ? now : lastAt) - started); + if (live && lastAt != null) { + const thinking = now - lastAt; + $('think').textContent = thinking >= 60000 ? stopwatch(thinking) : `${(thinking / 1000).toFixed(1)}s`; + $('think').classList.toggle('hot', thinking > 15000); + $('think-label').textContent = '本步思考'; + } else { + $('think').textContent = last ? stopwatch(runMetrics().timing.maxDecisionMs) : '—'; + $('think').classList.remove('hot'); + $('think-label').textContent = '最长一步'; + } + if (live) clockFrame = requestAnimationFrame(paintClock); +} +const secs = ms => ms == null ? '—' : ms >= 10000 ? `${(ms / 1000).toFixed(0)} s` : ms >= 1000 ? `${(ms / 1000).toFixed(1)} s` : `${ms} ms`; +function renderSteps() { + // One badge per decision: how long the agent took before committing this batch. + const latency = new Map(decisionTimeline(runObj).map(d => [d.from, d])); + $('steps').innerHTML = runObj.steps.map(s => { + const d = latency.get(s.n); + const badge = d ? `${secs(d.ms)}${d.actions > 1 ? ` · 连做 ${d.actions} 步` : ''}` : ''; + return `
  • ${badge}${describeAction(s.action)} → ${describeResult(s.result)}${s.note ? `${s.note}` : ''}
    • `; + }).join(''); + document.querySelectorAll('#steps li').forEach(li => li.onclick = () => { const n = Number(li.dataset.n); follow(false); rebuild(n); }); +} +// The list is one tab stop; the arrows walk it. Without this the step rows are +// mouse-only, and removing the slider left keyboard users with no replay at all. +$('steps').onkeydown = e => { + if (!runObj) return; + const last = runObj.steps.length; + const to = { ArrowDown: cursor + 1, ArrowRight: cursor + 1, ArrowUp: cursor - 1, ArrowLeft: cursor - 1, Home: 0, End: last }[e.key]; + if (to === undefined) return; + e.preventDefault(); + const n = Math.max(0, Math.min(last, to)); + if (n === last) { follow(true); return; } + follow(false); rebuild(n); +} +function renderMetrics() { + const m = runMetrics(); + const t = m.timing; + const perPiece = t.totalMs != null ? t.totalMs / runObj.level.tiles.length : null; + $('timing').innerHTML = [ + ['总耗时', secs(t.totalMs)], ['决策次数', m.actionsPerDecision != null ? `${m.decisions}（${m.actionsPerDecision.toFixed(1)} 步/次）` : `${m.decisions}`], + ['每次决策', `中位 ${secs(t.medianDecisionMs)} · 最长 ${secs(t.maxDecisionMs)}`], + ['开局读盘', secs(t.firstDecisionMs)], ['每枚棋子', secs(perPiece == null ? null : Math.round(perPiece))], + ...(t.thinkMs != null ? [['其中模型思考', secs(t.thinkMs)]] : []), + ].map(([k, v]) => `
    ${k}
    ${v}
    `).join(''); + const rows = [['状态', m.status], ['步数 / 预算', `${m.steps} / ${runObj.budget}`], ['动作 ÷ 棋子', m.actionsPerPiece?.toFixed(2)], ['点击 / 离场 / 空点', `${m.taps} / ${m.exits} / ${m.deadTaps}`], ['其中擦碰空点', m.grazeDeadTaps], ['最长连续空点', m.longestDeadStreak], ['旋转 / 有效 / 盲转', `${m.rotations} / ${m.effectiveRotations} / ${m.blindRotations}`], ['转错方向', m.wrongWayRotations], ['最长连续盲转', m.longestBlindStreak], ['进入卡住状态', m.stuckEntries], ['首次离场在第', m.stepsToFirstExit ?? '—'], ['撤销 / 查看', `${m.undos} / ${m.peeks}`], ['最少旋转（求解器）', m.minimumRotations ?? '—']]; + $('metrics').innerHTML = rows.map(([k, v]) => `
    ${k}
    ${v}
    `).join(''); + $('rating').innerHTML = runObj.rating ? RATING_QUESTIONS.map(q => `
    ${q.key}
    ${runObj.rating[q.key] ?? '—'}
    `).join('') : '
    尚未评分
    —
    '; + if (runObj.status === 'gave-up') $('rating').insertAdjacentHTML('afterbegin', `
    放弃原因
    ${runObj.reason || '—'}
    `); +} + +// ---------- drawing ---------- +function draw(now = performance.now()) { + cancelAnimationFrame(frame); + if (!view || !game || !runObj) return; + context.clearRect(0, 0, width, height); + const state = { hint, path, reduced }; + if (animation) { const p = Math.min(1, (now - animation.start) / animation.duration); state.animation = { route: animation.route, progress: p }; if (p >= 1) animation = null; } + if (turning) { const p = Math.min(1, (now - turning.start) / turning.duration); state.boardAngle = -turning.direction * Math.PI / 3 * Math.pow(1 - p, 3); if (p >= 1) turning = null; } + drawScene(painter, game.level, game.tiles, view, state); + if (animation || turning) frame = requestAnimationFrame(draw); + else if (following() && cursor < runObj.steps.length) setTimeout(pump, 120); +} +function resize() { + if (!game) return; + const rect = canvas.getBoundingClientRect(), dpr = Math.min(devicePixelRatio || 1, 3); + width = rect.width; height = rect.height; + canvas.width = Math.round(width * dpr); canvas.height = Math.round(height * dpr); + context.setTransform(dpr, 0, 0, dpr, 0, 0); + view = layout(game.level.cells, width, height); draw(); +} + +// ---------- controls ---------- +$('run').onchange = () => { pinned = true; loadRun($('run').value); }; +$('reload').onclick = async () => { const index = await loadIndex(); if (runObj) $('run').value = runObj.run; else if (index?.[0]) loadRun(index[0].run); }; +// Polling is unconditional — a page opened from the dropdown must still track a +// running game. `follow` only decides whether the page chases the agent: the +// newest step within this run, and the agent's next run once this one is done. +function follow(on) { + $('live').checked = on; + if (on) pinned = false; + if (on && runObj) rebuild(runObj.steps.length); + syncUi(); +} +// Watching "the agent", not "one run": when our run has finished (or has gone +// quiet while another is clearly active), hop to whatever it started next. +const STALE_MS = 20000; +let lastChangeAt = Date.now(); +// Picking a run from the dropdown pins it: auto-hop must not yank the viewer +// away from a game they deliberately opened. 「跟随最新」releases the pin. +let pinned = false; +function nextRun(index) { + if (pinned || !following() || !runObj || !index?.length) return null; + const mine = runObj.status !== 'playing' ? null : Date.now() - lastChangeAt; + if (mine !== null && mine < STALE_MS) return null; + const candidate = index.filter(e => e.run !== runObj.run && e.status === 'playing') + .sort((a, b) => (b.updatedAt || '').localeCompare(a.updatedAt || ''))[0]; + if (!candidate) return null; + return (candidate.updatedAt || '') > (runObj.updatedAt || '') ? candidate.run : null; +} +let tick = 0; +liveTimer = setInterval(async () => { + if (runObj) { + const before = runObj.steps.length; + await loadRun(runObj.run, { keepCursor: true }); + if (runObj && runObj.steps.length !== before) lastChangeAt = Date.now(); + } + if (tick++ % 2 === 0) { + const index = await loadIndex({ keepSelection: true }); + const hop = nextRun(index); + if (hop) { + await loadRun(hop, { fromStart: true }); + lastChangeAt = Date.now(); + } + } +}, 1000); +$('live').onchange = () => follow($('live').checked); +// Clicking a step in the feed is the whole replay control; this puts it back. +$('end').onclick = () => { follow(true); }; +const atlas = new Image(); atlas.onload = () => { painter.atlas = atlas; draw(); }; atlas.src = ASSET_BASE + TILE_ATLAS; +new ResizeObserver(resize).observe(canvas); + +const index = await loadIndex(); +const wanted = new URLSearchParams(location.search).get('run'); +if (wanted && index.some(e => e.run === wanted)) { + $('run').value = wanted; await loadRun(wanted); + // A link to a finished run is a replay link: don't let auto-hop drag the + // viewer off to whatever happens to be playing. A link to a live run still + // follows the agent into its next game, which is what `start` prints it for. + if (runObj && runObj.status !== 'playing') pinned = true; +} +else if (index[0]) await loadRun(index[0].run); +// live=1 仍然接受（start 打印的链接带它），但跟随本来就是默认开的。 +if (new URLSearchParams(location.search).get('live') === '0') follow(false); diff --git a/labs/bench/index.html b/labs/bench/index.html new file mode 100644 index 0000000..1534e70 --- /dev/null +++ b/labs/bench/index.html @@ -0,0 +1,104 @@ + + + + + + 关卡实验台 · Agent 观察 + + + +
    +

    关卡实验台 · Agent 观察

    + ← 返回主线 +
    + + + +
    + AGENT BENCH +
    + +
    + +
    +
    +
    +
    + — + +
    + +
    +
    + + +
    + + +
    + +
    +
    +

    耗时明细

    +
    +
    +
    +

    指标

    +
    +
    +
    +

    Agent 评分

    +
    +
    +
    +
    + + + + + diff --git a/labs/bench/style.css b/labs/bench/style.css new file mode 100644 index 0000000..0d03d58 --- /dev/null +++ b/labs/bench/style.css @@ -0,0 +1,169 @@ +:root{ + /* ink3 is the smallest type on the page; it has to clear 4.5:1 on white. */ + --ink:#1a2430; --ink2:#516678; --ink3:#647385; --ink4:#5a6974; + /* One source of truth for the two-column width, so the panels below the + stage line up with it instead of running the full window. */ + --rail:clamp(270px,25vw,340px); + --boardmax:calc((100svh - 205px) * .92); + --stage-w:calc(var(--boardmax) + var(--rail) + 16px); + --page:#eceff2; --card:#fff; --line:#d6dee5; --line2:#c2ccd6; + /* --live fills bars and the dark board; --live-ink is the readable text version. */ + --accent:#3f6d95; --live:#2ea36a; --live-ink:#1f7d50; --warn:#c2762a; --bad:#b8412b; + --board-bg:#11181f; --board-ink:#e8eef4; --board-dim:#8798a8; + font-family:system-ui,-apple-system,"PingFang SC",sans-serif;color:var(--ink);background:var(--page);font-synthesis:none; +} +*{box-sizing:border-box} +body{margin:0;padding-bottom:28px} +h1,h2{margin:0} +code{font-family:ui-monospace,Menlo,monospace;font-size:.92em;background:#e2e8ee;padding:1px 4px;border-radius:3px} + +/* ---------- top bar ---------- */ +.topbar{display:flex;align-items:center;gap:14px;padding:10px 20px;background:var(--card);border-bottom:1px solid var(--line);flex-wrap:wrap} +.back{color:var(--ink2);text-decoration:none;font-size:13px;white-space:nowrap} +.badge{font-size:10px;letter-spacing:.14em;color:var(--ink3);border:1px solid var(--line2);border-radius:20px;padding:5px 11px;margin-left:auto;white-space:nowrap} +.picker{display:flex;align-items:center;gap:9px;flex:1;min-width:280px;font-size:12px;color:var(--ink2)} +.picker select{font:13px system-ui;padding:6px 9px;border:1px solid var(--line2);border-radius:8px;background:#fff;color:var(--ink);flex:1;min-width:0;max-width:560px} +.toggle{display:inline-flex;align-items:center;gap:6px;cursor:pointer;white-space:nowrap} +input[type=checkbox]{accent-color:var(--accent)} +button{font:14px system-ui;color:var(--ink2);background:#f7fafc;border:1px solid var(--line2);border-radius:10px;padding:7px 12px;cursor:pointer} +button:hover{background:#fff;color:var(--ink)} +button:disabled{opacity:.4;cursor:default} +button.small{font-size:12px;padding:5px 10px;border-radius:8px} +button:focus-visible,a:focus-visible,select:focus-visible,input:focus-visible,.steps:focus-visible{outline:3px solid var(--accent);outline-offset:2px} +/* Reachable by screen readers, invisible on screen. */ +.sr-only{position:absolute;width:1px;height:1px;padding:0;margin:-1px;overflow:hidden;clip-path:inset(50%);white-space:nowrap;border:0} + +/* ---------- stage: board left, live rail right ---------- */ +/* The board is a portrait hexagon, so its card is height-bound: cap the board + column near the height it can actually use and centre the pair, instead of + stretching a card whose middle 40% would be empty. */ +.stage{display:grid;grid-template-columns:minmax(0,var(--boardmax)) var(--rail); + gap:16px;padding:16px 20px 0;align-items:stretch;justify-content:center} +.board-col{background:var(--card);border:1px solid var(--line);border-radius:14px;padding:14px;display:flex;flex-direction:column;min-width:0} +/* The canvas is taken out of flow on purpose. Left in flow, `height:100%` + degrades to auto during intrinsic sizing and the box falls back to the + canvas *attribute* ratio — which resize() writes from this very box. That + loop let a detour through a short window permanently inflate the card. */ +.board-wrap{flex:1;width:100%;position:relative;min-height:clamp(340px,calc(100svh - 235px),760px)} +canvas{position:absolute;inset:0;width:100%;height:100%;display:block} +/* Only surfaces when a run's JSON can't be read — otherwise it takes no space. */ +.notice{margin:10px 0 0;text-align:center;font-size:13px;line-height:1.6;color:var(--bad)} + +/* 谁在玩：棋盘正上方的身份横幅 */ +.who{display:flex;align-items:baseline;justify-content:space-between;gap:12px;flex-wrap:wrap; + padding:0 2px 10px;margin-bottom:10px;border-bottom:1px solid var(--line)} +.who-id{display:flex;align-items:baseline;gap:10px;flex-wrap:wrap;min-width:0} +.who-id b{font-size:19px;font-weight:700;letter-spacing:-.01em} +/* Agents self-report free text, so assume the worst: a slug or user-agent + string with no spaces has to break rather than push the page sideways. */ +.who-agent{font-size:13px;color:var(--accent);font-weight:600;min-width:0;overflow-wrap:anywhere} +/* Hang the separator off the model name. As ::before on the agent it wrapped + to the start of the next line and read like a bullet. */ +.who-id:has(.who-agent:not(:empty)) b::after{content:'·';margin-left:10px;color:var(--ink3);font-weight:400} +.who-level{font-size:11.5px;color:var(--ink3);font-variant-numeric:tabular-nums} + +.rail{display:flex;flex-direction:column;gap:16px;min-width:0;min-height:0} + +/* ---------- scoreboard: a vertical panel beside the board ---------- */ +.scoreboard{background:var(--board-bg);color:var(--board-ink);border-radius:14px;padding:16px; + display:flex;flex-direction:column;gap:13px;flex:0 0 auto;transition:opacity .2s} +/* Rewound: the clock and the totals belong to the whole run, so they no longer + describe the board sitting right next to them. Say so instead of lying. */ +.scoreboard.rewound{opacity:.55} +@media(prefers-reduced-motion:reduce){.scoreboard{transition:none}} +.scoreboard .clock{display:flex;flex-direction:column;gap:4px;min-width:0} +.scoreboard .k{font-size:10px;letter-spacing:.12em;color:var(--board-dim);text-transform:uppercase;white-space:nowrap} +.scoreboard b{font-family:ui-monospace,SFMono-Regular,Menlo,monospace;font-variant-numeric:tabular-nums;font-weight:600;line-height:1;font-size:19px} +.scoreboard small{font-size:11px;color:var(--board-dim);margin-left:4px;font-family:ui-monospace,Menlo,monospace} +.clock.primary b{font-size:clamp(32px,3.1vw,42px);letter-spacing:-.02em} +.clock.think b{font-size:23px;color:#f0c674} +.tick.hot{color:#ffcf6b} +/* Each readout is a wrapping row: the label owns a line, the number keeps its + suffix beside it. Stacking all three (a flex column blockifies `inline`) + cost a third line per cell and made the panel needlessly tall. */ +.scoreboard .stats{display:grid;grid-template-columns:1fr 1fr;gap:12px 12px;border-top:1px solid #27323d;padding-top:13px} +.scoreboard .cell{display:flex;flex-wrap:wrap;align-items:baseline;min-width:0} +.scoreboard .cell .k{flex:0 0 100%;margin-bottom:4px} +.scoreboard .bar{height:5px;background:#27323d;border-radius:3px;overflow:hidden} +.scoreboard .bar i{display:block;height:100%;background:var(--live);width:0;transition:width .3s} +.pill{align-self:flex-start;font-size:12px;font-weight:600;letter-spacing:.04em;padding:7px 14px;border-radius:20px;white-space:nowrap;background:#25303b;color:var(--board-dim)} +.pill.rewound{background:#2b3742;color:#c7d4de} +.pill.playing{background:rgba(46,163,106,.18);color:#5ed69b} +.pill.playing::before{content:'●';margin-right:6px;animation:blink 1.4s infinite} +.pill.cleared{background:rgba(46,163,106,.25);color:#7fe6b4} +.pill.gaveup{background:rgba(184,65,43,.22);color:#ff9b85} +.pill.budget{background:rgba(194,118,42,.22);color:#f3c98a} +@keyframes blink{50%{opacity:.25}} + +/* ---------- live feed ---------- */ +.feed-col{background:var(--card);border:1px solid var(--line);border-radius:14px;padding:14px;display:flex;flex-direction:column;flex:1 1 auto;min-height:200px} +/* The button and the live state are siblings of the heading, not inside it — + nested, they became part of the heading's accessible name. */ +.feed-head{display:flex;align-items:center;gap:8px;flex-wrap:wrap;margin-bottom:10px} +.feed-col h2{font-size:11px;letter-spacing:.16em;color:var(--ink3);text-transform:uppercase} +.feed-state{font-size:10.5px;letter-spacing:.02em;color:var(--live-ink);font-weight:600} +.back-live{margin-left:auto;color:var(--accent);border-color:var(--accent)} +.steps{margin:0;padding:0;list-style:none;overflow-y:auto;flex:1 1 0;min-height:0;font-size:12.5px;counter-reset:step} +.steps li{padding:8px 10px 8px 32px;border-bottom:1px solid #eef2f5;position:relative;cursor:pointer;line-height:1.55;counter-increment:step} +.steps li::before{content:counter(step);position:absolute;left:8px;top:8px;font-family:ui-monospace,Menlo,monospace;font-size:10.5px;color:var(--ink3)} +.steps li:last-child{border-bottom:none} +.steps li.lead{border-top:1px solid var(--line);margin-top:2px} +.steps li:first-child.lead{border-top:none;margin-top:0} +.steps li b{font-weight:600} +.steps li i{display:block;color:var(--ink2);font-style:normal;font-size:11.5px;margin-top:2px} +.steps li em{float:right;margin-left:8px;font-style:normal;font-size:10.5px;color:var(--accent);background:#eaf1f7;border-radius:9px;padding:1px 7px;font-variant-numeric:tabular-nums;font-weight:600} +.steps li.current{background:#eaf1f7;box-shadow:inset 3px 0 0 var(--accent)} +.steps li:hover{background:#f2f6f9} + +/* ---------- details below ---------- */ +.details{display:grid;grid-template-columns:repeat(auto-fit,minmax(260px,1fr));gap:14px;padding:16px 20px 0; + width:100%;max-width:calc(var(--stage-w) + 40px);margin-inline:auto} +.panel{background:var(--card);border:1px solid var(--line);border-radius:14px;padding:14px} +.panel h2{font-size:11px;letter-spacing:.16em;color:var(--ink3);text-transform:uppercase;margin-bottom:10px} +.kv{margin:0;font-size:12.5px} +.kv>div{display:flex;justify-content:space-between;gap:10px;padding:5px 0;border-bottom:1px dashed #e4eaef} +.kv>div:last-child{border-bottom:none} +.kv dt{color:var(--ink3)} +.kv dd{margin:0;font-variant-numeric:tabular-nums;text-align:right;color:var(--ink);font-weight:600} +.kv .giveup dd{color:var(--bad);text-align:left;font-weight:400} + +footer{text-align:center;margin:20px auto 0;max-width:760px;padding:0 20px;font-size:11px;color:var(--ink4);line-height:1.7} + +/* The rail needs ~260px to stay readable; below that it goes under the board, + where the clock turns into a wide two-up row instead of a tall column. */ +@media(max-width:820px){ + .stage{grid-template-columns:1fr;align-items:start} + /* Stacked, the card is full width, so the board should be tall enough to + use it — 62vw left a 603px canvas holding a 330px board. */ + .board-wrap{min-height:clamp(280px,min(96vw,64svh),680px)} + .scoreboard{display:grid;grid-template-columns:auto minmax(0,1fr);align-items:center;gap:12px 18px} + .scoreboard .clock.primary{grid-column:1} + .scoreboard .clock.think{grid-column:2} + .clock.primary b{font-size:38px} + .scoreboard .stats{grid-column:1 / -1;grid-template-columns:repeat(4,minmax(0,1fr))} + .scoreboard .bar,.pill{grid-column:1 / -1} + /* Stacked, the rail's height comes from its content, so `flex:1 1 0` resolved + to zero and the list vanished — taking the only way to replay a step with + it. Give the list its own height instead of a share of a leftover. */ + .feed-col{min-height:0} + .steps{flex:0 0 auto;max-height:min(340px,52svh)} +} +@media(max-width:520px){ + .topbar{padding:8px 12px;gap:8px} + .picker{min-width:100%;order:3} + .picker select{max-width:none} + .stage,.details{padding-left:12px;padding-right:12px} + .board-col,.feed-col,.panel{padding:12px} + .scoreboard{padding:14px;gap:11px 14px} + .clock.primary b{font-size:34px} + .clock.think b{font-size:20px} + .scoreboard .stats{grid-template-columns:1fr 1fr} + .who-id b{font-size:17px} +} +/* Below ~400px the two clocks no longer fit on one row: 「本步思考」 gets about + 94px and its mono value needs 96. Stack them rather than clip the digits. */ +@media(max-width:400px){ + .scoreboard .clock.primary,.scoreboard .clock.think{grid-column:1 / -1} + .clock.think b{font-size:22px} +} +@media(prefers-reduced-motion:reduce){.pill.playing::before{animation:none}.scoreboard .bar i{transition:none}}