diff --git a/README.md b/README.md index 8d006ec..8b9e8b4 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,8 @@ KataGo 是事实裁判，LLM 是讲棋老师。GoAgent 的目标是让学生不 ### 多模态讲棋 +**为了获得最佳讲棋效果，建议使用支持图片输入的多模态模型。** 当前讲棋流程会核验棋盘图片，并结合精确棋谱坐标与 KataGo 证据解释局面；图片中的观察不能覆盖棋谱事实。 + 当前手分析会把这些信息组合给用户配置的多模态 LLM： - 当前棋盘截图。 diff --git a/README_EN.md b/README_EN.md index 55d630c..dd005c1 100644 --- a/README_EN.md +++ b/README_EN.md @@ -153,6 +153,8 @@ pnpm dist:linux ## AI Teacher Connections +**For the best teaching results, we recommend a multimodal model that accepts images.** The current teaching workflow validates board images alongside exact SGF coordinates and KataGo evidence; visual observations must not override the game record. + - **API key**: Continue using any OpenAI-compatible multimodal model service. - **ChatGPT sign-in**: Choose “Sign in with ChatGPT” under **Settings → AI Teacher**. GoAgent uses the official Codex App Server for sign-in, model discovery, and requests, and can use models in the active ChatGPT plan that accept board images. - ChatGPT sign-in uses a GoAgent-specific Codex App Server data directory. It does not read or modify Codex CLI or Codex Desktop sign-in. GoAgent application code does not read, copy, or print OAuth tokens. diff --git a/docs/COMMENTARY_CONCEPT_EXPERIMENTS.md b/docs/COMMENTARY_CONCEPT_EXPERIMENTS.md new file mode 100644 index 0000000..c42678e --- /dev/null +++ b/docs/COMMENTARY_CONCEPT_EXPERIMENTS.md @@ -0,0 +1,88 @@ +# 围棋解说机制与概念识别实验 + +本次改动将精确棋谱、可验证的棋形与棋块关系、同局面 KataGo 比较和简短输出策略接入实际老师流程。目标是减少错误的棋理断言，并让当前手讲解直接表达主要目的。 + +**为了获得最佳讲棋效果，建议使用支持图片输入的多模态模型。** 当前流程会核验棋盘图片，并将精确坐标和引擎证据交给模型。结构化证据本身不依赖视觉识别，但当前程序尚未提供经过验证的纯文本讲棋路径；本轮也未进行相同输入条件下有图与无图的隔离对照，因此这一建议不是多模态收益的定量结论。 + +## 讲棋机制 + +### 棋形、目的与结果分层 + +- 相对尖、跳、飞、直接接触、空走廊及阻挡属于可核对的棋盘事实；保留多个己棋锚点与真实颜色。 +- 争头、限制出头、分割联系、己方出头及沿边展开属于带目标与反证的目的假设。 +- 已经封锁、已经分断、死活、先手与强制应手需要独立证据；不能从几何、局部模式分数、气数或 ownership 自动推出。 +- 本地知识检索移除全盘绝对坐标巧合及 PV 锚点加分。题型相似只提示应检查什么，不授权将题库答案当作当前局面的合法变化。 + +### 棋块关系与实际落子效果 + +`boardGroupRelations.ts` 提供完整黑白坐标、真实正交连通串、潜在联系及区域目的候选。潜在联系不等于已经连接，正交串不等于战略整龙。 + +`ownDevelopment.ts` 比较己方受压方向、开放走廊与前端锚点，帮助区分自身出头和攻击对方。同一己串有跳与背侧飞关系时，不能任选一个锚点给整手命名。自身发展候选也不证明棋块未活或已经安定。 + +`moveBoardEffects.ts` 模拟落子、提子、连接和气的变化。局部单子劫循环以结构证据描述；反事实回提只检验棋盘结构，不验证历史劫禁或超级劫。粘劫、收官和连接不能自动称作弱棋补强。 + +### 全局价值与具体着手评价 + +`globalPriorityEvidence.ts` 比较同一落子前局面的首选与实战，区分普通根候选和单独 forced 实战查询，按行棋方统一视角。不能用落子前后 root 估值相减替代同局面选择损失。 + +远处阈值随棋盘尺寸变化；明显差距采用至少3目或非饱和胜率区间内8个百分点，并核验根搜索深度、候选搜索次数与教学就绪状态。单独 forced 实战查询次数不能抬高普通根搜索质量。 + +本地存在接近首选的替代点时，不把具体实战点的损失归因于整个局部不重要。具体实战点有可靠显著损失时，即使首选不满足远处条件，也应简短指出更好的选择及已核验用途。 + +### 证据传递与简短输出 + +- 修复 KataGo 顶层 root ownership 读取，兼容旧 remote nested 输出并严格检查尺寸与数值范围。 +- 核对棋局、手数、落子前时点和试下分支。根候选使用同一落子前棋盘，PV 后续必须另行重放。 +- 显式要求加深时核对实际根搜索预算，防止浅层 prefetched 分析覆盖加深请求；试下使用实际试下分析入口。 +- 完整坐标、全局比较、紧凑候选发展与关键落子效果优先序列化，减少工具消息18k字符预算截断导致的事实遗漏。 +- 保留合法 assistant tool_calls、对应工具成功/失败结果和后续图片。Codex 自动工具当前手路径在首次模型请求前准备真实棋盘图，减少动态图片传递的不可靠情况。 +- 默认当前手解说一句话、约20–60中文字，选择主要有据目的。小差距省略推荐比较，内部核验深度不要求最终长篇。明确详解或变化请求仍可展开。 + +## 实验路线与选择 + +| 路线 | 实际实验 | 结论 | +| --- | --- | --- | +| 相对几何、关系与 KataGo 证据 | 真实规则执行、棋盘模拟及老师工具集成 | 已接入本次候选实现，仍需围棋语义审阅 | +| 棋盘线性模型 / 小 CNN | 6000条评论局面，按完整棋谱组隔离训练、验证和测试 | 评论词语弱标签精确率不足，未接入产品 | +| 棋盘 + KataGo ownership/policy | 同600局面配对探索 | 增益不稳定，未接入产品 | +| 隐藏层 probe | 核验论文、原代码与普通 KataGo 接口 | 未训练；需要适配引擎和专家标签 | + +数据准备默认固定源提交并下载1000盘 SGF，获得32,760个合法主线评论局面。评论关键词不是当前手意图金标，包含假设变化、否定与省略等噪声；实验输入也不等同于产品的落子前目的解释。 + +平衡 CNN 探索的宏精确率约0.0403、宏 F1 约0.0722，不能因能输出术语就用于产品命名。同600局面探索中，加入 KataGo 的 CNN F1 从0.0533到0.0633，但按棋谱组 bootstrap 的差异区间跨0，不能认定稳定胜出。完整数据、模型权重及原评论不随源码或发行包分发。 + +来源与许可： + +- [ACL 2022论文](https://aclanthology.org/2022.acl-short.90/)、[原始数据代码](https://github.com/AndreHe02/go)、[原 probe 代码](https://github.com/AndreHe02/go-probe)。 +- [GTL版权说明](https://gtl.xmp.net/about/site)：原评论属于评审者，本次仅作私人研究。 +- [KataGo Analysis API](https://github.com/lightvector/KataGo/blob/master/docs/Analysis_Engine.md)、[模型许可](https://katagotraining.org/network_license/)。 +- [MasterMind](https://github.com/opendilab/Mastermind) 的学术用途及书籍来源限制未澄清，本次未使用其数据训练。 + +## 验证与人工审阅 + +改动已通过392项项目测试、类型检查和生产构建。覆盖颜色/对称/阻挡反例、占据与缺盘弃权、坐标完整性、同根比较、当前方视角、浅搜索保护、试下加深、工具历史、初始图片和实际棋盘效果。工程测试通过不代表解说语义完全正确；打包发行尚未验证。 + +真实老师评测使用用户选择的 ChatGPT GPT-6 Luna，匿名 SGF 只包含尺寸、贴目、规则、初始棋子和完整真实落子历史；不传入源评论、专家术语或反馈。运行记录核验实际加载构建、模型/provider、棋谱、提示、设置、棋盘和图片身份，保存原文、失败及各次运行；provider temperature/effort 使用默认值。缺失棋谱规则时明确记录日本规则 fallback。 + +人工审阅发现并推动了全局重点、棋块目的、简短表达及己方出头等修正。新增随机样本先固定抽样再生成，不依据模型结果换样。反馈用于修改后，相应局面是开发回归样本，不能用于独立总体准确率统计。 + +最后三局回归原文已转为自身发展、粘劫及跳出，并能解释推荐连接点；用户认为可接受。仍有次要推断和泛化提醒可省略，尚未实现任意自然语言颜色/占据/战术断言的全面语义核验。三局运行流程完成不等于泛化准确率。 + +审阅工具显示完整实际原文与落子前后棋盘。填写反馈自动暂存在浏览器，点击“保存反馈到项目”才写入项目；浏览器保存网页或下载 JSON 备份不等于提交。工具默认只监听 loopback。源码不包含本机评测产物、用户原始反馈或认证数据。 + +## 复现入口 + +```sh +pnpm test +pnpm typecheck +pnpm build +python -m pip install -r scripts/experiments/requirements-comment-concepts.txt +python scripts/experiments/prepare_comment_concepts.py --games 1000 +python scripts/experiments/train_comment_concepts.py --data .tmp/concept-data/comments.jsonl --out .tmp/concept-experiments/board6000 --max-samples 6000 --epochs 8 +pnpm eval:move-concepts +python scripts/serve_concept_review.py --directory .tmp/concept-review --port 8847 +``` + +引擎实验需要自行提供 KataGo binary、model 与配置路径，不下载或提交本机二进制。`collect_katago_concepts.py --help` 与 `collect_review_analysis.py --help` 列出参数。 + +真实老师、盲评与短句预览入口为 `eval_commentary_pipeline.mjs`、`build_commentary_blind_review.mjs` 和 `build_commentary_short_preview.mjs`。实际评测需要运行配置好模型、KataGo 与 remote debugging 的隔离 GoAgent 实例及匿名 fixtures；查看各脚本 `--help`。筛选参数为 `--onlyids`。生产程序不依赖这些评测脚本。 diff --git a/package.json b/package.json index ff485f4..a065fff 100644 --- a/package.json +++ b/package.json @@ -22,7 +22,7 @@ "typecheck": "tsc --noEmit -p tsconfig.node.json && tsc --noEmit -p tsconfig.web.json", "preview": "node scripts/start_dev_app.mjs --skip-build", "lint": "pnpm typecheck", - "test": "node --test tests/*.test.mjs", + "test": "node --import tsx --test tests/*.test.mjs", "check": "pnpm test && pnpm typecheck && pnpm build", "eval:teacher": "node scripts/eval_teaching_accuracy.mjs", "eval:claims": "node scripts/eval_teacher_claims.mjs", @@ -30,6 +30,7 @@ "check:knowledge-sources": "node scripts/check_knowledge_sources.mjs", "eval:knowledge-coverage": "node scripts/eval_knowledge_coverage.mjs", "eval:shape-recognition": "node scripts/eval_shape_recognition.mjs", + "eval:move-concepts": "node scripts/eval_move_concepts.mjs", "eval:move-range": "node scripts/eval_move_range.mjs", "eval:vision-evidence": "node scripts/eval_vision_evidence.mjs", "eval:katago-trace": "node scripts/eval_katago_trace.mjs", diff --git a/scripts/build_commentary_blind_review.mjs b/scripts/build_commentary_blind_review.mjs new file mode 100644 index 0000000..02faac7 --- /dev/null +++ b/scripts/build_commentary_blind_review.mjs @@ -0,0 +1,267 @@ +#!/usr/bin/env node +import assert from 'node:assert/strict' +import { readFileSync, writeFileSync, mkdirSync, existsSync, realpathSync } from 'node:fs' +import { resolve, join, relative, dirname, isAbsolute } from 'node:path' +import { fileURLToPath } from 'node:url' +import { createHash, randomInt } from 'node:crypto' + +const root = fileURLToPath(new URL('../', import.meta.url)) +const hash = value => createHash('sha256').update(value).digest('hex') +const read = path => JSON.parse(readFileSync(path, 'utf8')) +const canonical = value => JSON.stringify(value, (_, v) => v && typeof v === 'object' && !Array.isArray(v) ? Object.fromEntries(Object.keys(v).sort().map(k => [k, v[k]])) : v) +const escape = value => String(value).replace(/[&<>"']/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c]) +const columns = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +const within = (parent, child) => { const path = relative(parent, child); return path !== '' && !path.startsWith('..') && !isAbsolute(path) } + +function secureOutput(output, inputs) { + const temporary = resolve(root, '.tmp') + assert.ok(within(temporary, output), 'Output must be a child directory of this repository .tmp, not .tmp itself') + for (const input of inputs) assert.ok(output !== input && !within(output, input) && !within(input, output), 'Output must be separate from input directories') + let ancestor = output + while (!existsSync(ancestor)) ancestor = dirname(ancestor) + assert.ok(within(realpathSync(temporary), realpathSync(ancestor)) || realpathSync(temporary) === realpathSync(ancestor), 'Output ancestor escapes .tmp through a link') + if (existsSync(output)) assert.equal(realpathSync(output).toLowerCase(), output.toLowerCase(), 'Linked output directories are unsupported') +} + +function point(raw, size) { + if (/^pass$/i.test(raw)) return null + const match = /^([A-HJ-Z])([1-9]\d?)$/.exec(raw) + assert.ok(match, 'Invalid board point') + const x = columns.indexOf(match[1]); const y = Number(match[2]) - 1 + assert.ok(x < size && y < size, 'Point outside board') + return [x, y] +} +const key = (x, y) => `${x},${y}` +function neighbors(x, y, size) { return [[x - 1, y], [x + 1, y], [x, y - 1], [x, y + 1]].filter(([a, b]) => a >= 0 && b >= 0 && a < size && b < size) } +function group(board, origin, size) { + const color = board.get(origin); const stones = new Set([origin]); const pending = [origin]; const liberties = new Set() + while (pending.length) { + const [x, y] = pending.pop().split(',').map(Number) + for (const [a, b] of neighbors(x, y, size)) { + const p = key(a, b) + if (!board.has(p)) liberties.add(p) + else if (board.get(p) === color && !stones.has(p)) { stones.add(p); pending.push(p) } + } + } + return { stones, liberties } +} +function play(board, color, coordinate, size) { + if (!coordinate) return + const [x, y] = coordinate; const p = key(x, y) + assert.ok(!board.has(p), 'Replay found occupied move') + board.set(p, color) + for (const [a, b] of neighbors(x, y, size)) { + const q = key(a, b) + if (board.has(q) && board.get(q) !== color) { + const enemy = group(board, q, size) + if (!enemy.liberties.size) for (const stone of enemy.stones) board.delete(stone) + } + } + assert.ok(group(board, p, size).liberties.size, 'Replay found suicide; unsupported fixture rules') +} +function boardList(board) { + return [...board].map(([p, color]) => { const [x, y] = p.split(',').map(Number); return { point: `${columns[x]}${y + 1}`, color } }).sort((a, b) => a.point.localeCompare(b.point)) +} + +export function fixtureBoards(fixture, source, sgf) { + const size = fixture.boardSize + assert.equal(Number(/SZ\[(\d+)\]/.exec(sgf)?.[1]), size) + assert.equal(hash(sgf.replace(/\r?\n$/, '')), fixture.sha256, 'Fixture content hash changed') + assert.ok([...sgf.matchAll(/([A-Z]+)\[/g)].every(match => ['SZ', 'KM', 'RU', 'AB', 'AW', 'B', 'W'].includes(match[1])), 'Nonanonymous SGF metadata') + const decode = value => value ? [value.charCodeAt(0) - 97, size - 1 - (value.charCodeAt(1) - 97)] : null + const board = new Map() + for (const setup of sgf.matchAll(/A([BW])((?:\[[a-y]{2}\])+)/g)) for (const value of setup[2].matchAll(/\[([a-y]{2})\]/g)) { + const [x, y] = decode(value[1]); assert.ok(x < size && y >= 0, 'Invalid setup stone') + assert.ok(!board.has(key(x, y)), 'Duplicate setup stone'); board.set(key(x, y), setup[1]) + } + const moves = [...sgf.matchAll(/;([BW])\[([a-y]{2}|)\]/g)].map(match => [match[1], decode(match[2])]) + assert.equal(moves.length, fixture.moveNumber, 'SGF/history mismatch') + assert.equal(source.moveNumber, fixture.moveNumber) + for (const [color, coordinate] of moves.slice(0, -1)) play(board, color, coordinate, size) + const before = boardList(board) + assert.ok(Array.isArray(source.boardSnapshot), 'Source has no complete snapshot') + const snapshot = new Map() + for (const stone of source.boardSnapshot) { + assert.ok(['B', 'W'].includes(stone.color)); const [x, y] = point(stone.point, size) + assert.ok(!snapshot.has(key(x, y)), 'Duplicate snapshot stone'); snapshot.set(key(x, y), stone.color) + } + assert.equal(canonical(before), canonical(boardList(snapshot)), 'SGF replay does not equal source before-move snapshot') + assert.equal(moves.at(-1)?.[0], source.playerColor) + assert.equal(canonical(moves.at(-1)?.[1]), canonical(point(source.playedMove, size)), 'Played move differs from source') + play(board, source.playerColor, moves.at(-1)[1], size) + const recentMoves = moves.slice(0, -1).slice(-8).map(([color, coordinate], index, list) => ({ + moveNumber: moves.length - list.length + index, color, + point: coordinate ? `${columns[coordinate[0]]}${coordinate[1] + 1}` : 'pass' + })) + return { boardSize: size, moveNumber: fixture.moveNumber, playerColor: source.playerColor, playedMove: source.playedMove, before, after: boardList(board), recentMoves } +} + +export function validatePair(fixture, candidate, baseline) { + for (const [name, result] of [['candidate', candidate], ['baseline', baseline]]) { + assert.equal(result?.success, true, 'Both real results must be successful') + assert.equal(result.phase, 'formal', 'Only formal evaluations can enter blind review') + assert.equal(result.variant, name, 'Input result is in the wrong directory') + assert.equal(result.id, fixture.id) + assert.equal(result.model, 'gpt-6-luna') + assert.equal(result.provider, 'codex-app-server', 'Mock and other providers are excluded') + assert.ok(result.result?.markdown?.trim(), 'Missing complete model commentary') + assert.equal(result.identity?.fixtureHash, fixture.sha256) + assert.equal(result.identity?.model, result.model) + assert.equal(result.identity?.provider, result.provider) + assert.equal(result.identity?.variant, name) + assert.equal(result.identity?.phase, 'formal') + assert.match(result.profileHash ?? '', /^[a-f0-9]{64}$/, 'Missing semantic student profile hash') + for (const field of ['bundleHash', 'runtimeHash', 'engineConfigHash']) { + assert.match(result[field] ?? '', /^[a-f0-9]{64}$/, `Missing ${field}`) + assert.equal(result.identity[field], result[field]) + } + for (const field of ['promptHash', 'settingsHash']) assert.match(result.identity[field] ?? '', /^[a-f0-9]{64}$/) + assert.equal(result.request?.moveNumber, fixture.moveNumber) + assert.equal(result.request?.mode, 'current-move') + assert.equal(hash(result.request.prompt), result.identity.promptHash) + assert.equal(hash(JSON.stringify({ app: result.applicationSettings, request: result.requestSettings })), result.identity.settingsHash) + assert.ok(result.requestSettings && result.applicationSettings && result.inferenceSettings, 'Missing reproducible settings') + assert.equal(result.analysis?.moveNumber, fixture.moveNumber, 'Analysis move number mismatch') + assert.equal(result.analysis?.boardSize, fixture.boardSize, 'Analysis board size mismatch') + assert.equal(result.analysis?.gameId, result.request.gameId, 'Analysis belongs to another game') + assert.equal(result.boardVerification?.verified, true, 'Missing imported-board verification') + for (const phase of ['before', 'after']) assert.equal(result.boardVerification[phase]?.length, fixture.boardSize ** 2, 'Incomplete verified board') + } + assert.equal(candidate.profileHash, baseline.profileHash, 'Semantic student profile changed between paired runs') + for (const phase of ['before', 'after']) assert.equal(canonical(candidate.boardVerification[phase]), canonical(baseline.boardVerification[phase]), 'Verified boards differ between paired runs') + for (const field of ['fixtureHash', 'runtimeHash', 'engineConfigHash', 'promptHash', 'settingsHash']) assert.equal(candidate.identity[field], baseline.identity[field], `Pair ${field} mismatch`) + for (const field of ['requestSettings', 'applicationSettings', 'inferenceSettings']) assert.equal(canonical(candidate[field]), canonical(baseline[field]), `Pair ${field} mismatch`) + const cleanRequest = request => Object.fromEntries(Object.entries(request).filter(([name]) => name !== 'runId')) + assert.equal(canonical(cleanRequest(candidate.request)), canonical(cleanRequest(baseline.request)), 'Pair request mismatch') +} + +function verifyReplayEvidence(boards, result) { + for (const phase of ['before', 'after']) { + const flat = Array(boards.boardSize ** 2).fill(0) + for (const stone of boards[phase]) { const [x, y] = point(stone.point, boards.boardSize); flat[y * boards.boardSize + x] = stone.color === 'B' ? 1 : -1 } + assert.equal(canonical(flat), canonical(result.boardVerification[phase]), 'Verified result board differs from anonymous SGF replay') + } +} + +export function runStatus(result) { + if (!result || result.phase !== 'formal') return { attempted: false, passed: false, text: '尚无正式运行记录' } + if (result.model !== 'gpt-6-luna' || result.provider !== 'codex-app-server') return { attempted: false, passed: false, text: '记录不符合本次评测条件' } + if (result.success === true) return { attempted: true, passed: true, text: '整链验收已通过' } + if (result.result?.markdown?.trim()) return { attempted: true, passed: false, text: /evaluator interruption|Recovery incomplete|评测中断/i.test(result.error ?? '') ? '讲解已返回，但评测中断、验收证据不完整' : '讲解已返回，但整链验收未通过' } + return { attempted: true, passed: false, text: '运行或整链验收未通过' } +} + +function statusPanel(data) { + const summary = data.summary ?? { plannedPositions: data.cases.length / 2, completePairs: data.cases.length / 2, unpairedPositions: 0, expectedRuns: data.cases.length, passedRuns: data.cases.length, unpassedRuns: 0, pendingRuns: 0 } + const failures = (data.positions ?? []).filter(item => item.state !== 'complete').map(item => `
  • ${escape(item.id)}：${item.statuses.map(escape).join('；')}。${escape(item.state === 'incompatible' ? '配对条件或棋盘核对未通过' : '缺少完整合格配对')}。
    • `).join('') + return `

    本轮覆盖情况

    预定 ${summary.plannedPositions} 局 · 完整配对 ${summary.completePairs} 局 · 未配对 ${summary.unpairedPositions} 局。

    两组运行合计 ${summary.expectedRuns} 份：整链验收通过 ${summary.passedRuns} 份，未通过或评测中断 ${summary.unpassedRuns} 份，尚无合格正式记录 ${summary.pendingRuns} 份。

    下面只评完整配对的讲解。未配对局不计语义准确率，排除失败也不能说明讲解更准确。${summary.completePairs === 0 ? '目前没有完整配对，流水线尚未完成。' : ''}

    ${failures ? `
    未配对局状态（不区分两组）
    ` : ''}
    ` +} + +function boardSvg(entry, stones, phase) { + const step = 22; const offset = 32; const end = offset + (entry.boardSize - 1) * step; const extent = end + 32 + let svg = `` + for (let i = 0; i < entry.boardSize; i++) { + const p = offset + i * step + svg += `${columns[i]}${entry.boardSize - i}` + } + const locate = raw => { const [x, y] = point(raw, entry.boardSize); return [offset + x * step, offset + (entry.boardSize - 1 - y) * step] } + for (const stone of stones) { const [x, y] = locate(stone.point); svg += `` } + if (point(entry.playedMove, entry.boardSize)) { const [x, y] = locate(entry.playedMove); svg += `` } + return svg + '' +} + +export function reviewHtml(publicData) { + const options = values => `${values.map(v => ``).join('')}` + const entries = new Map(publicData.cases.map(entry => [entry.id, entry])) + const ids = publicData.cases.filter(entry => entry.id.endsWith(':A')).map(entry => entry.id.slice(0, -2)) + const cards = statusPanel(publicData) + ids.map((id, index) => { + const entry = entries.get(`${id}:A`) + const panels = ['A', 'B'].map(letter => { const result = entries.get(`${id}:${letter}`); return `

    讲解 ${letter}

    完整讲解原文
    ${escape(result.markdown)}
    ` }).join('') + const recent = (entry.recentMoves ?? []).map(move => `第 ${move.moveNumber} 手 ${move.color === 'B' ? '黑' : '白'} ${move.point === 'pass' ? '停着' : move.point}`).join(' → ') + return `

    ${index + 1}. ${id} · 第 ${entry.moveNumber} 手 · ${entry.playerColor === 'B' ? '黑' : '白'} ${entry.playedMove}

    ${boardSvg(entry, entry.before, 'before')}

    红圈标记本手；停着无落点圈。

    请分别评 A、B 两段完整讲解。可以展开或收起原文，棋盘可切换到落子后查看提子结果。

    落子前最近手顺

    ${escape(recent || '没有此前着手。')}

    ${panels}
    ` + }).join('') + const storageKey = `goagent-commentary-blind-v1-${publicData.reviewSetId}` + return `围棋讲解盲评

    围棋讲解盲评

    模型：${publicData.model} · ${ids.length} 局，两段讲解均来自同一模型。

    建议先看还没审过的局面。先对照棋盘判断事实，再看讲解是否解释清楚关系、目的与风险；不必因为术语多就给高分。不确定时可直接标“不确定”。

    ${cards}
    ` +} + +export function main(args = process.argv.slice(2)) { + const option = (name, fallback) => { const at = args.indexOf(`--${name}`); if (at < 0) return fallback; assert.ok(args[at + 1] && !args[at + 1].startsWith('--'), `Missing --${name} value`); return args[at + 1] } + if (args.includes('--help')) { console.log('node scripts/build_commentary_blind_review.mjs [--input .tmp/commentary-live-v2] [--cases .tmp/concept-review-round2/cases.json] [--out .tmp/commentary-blind-review-v2] [--onlyids position-1,position-2]'); return } + const input = resolve(root, option('input', '.tmp/commentary-live-v2')) + const output = resolve(root, option('out', '.tmp/commentary-blind-review-v2')) + const sourcePath = resolve(root, option('cases', '.tmp/concept-review-round2/cases.json')) + const privateDirectory = resolve(root, '.tmp/.commentary-blind-private') + secureOutput(output, [input, dirname(sourcePath), privateDirectory]) + secureOutput(privateDirectory, [input, output, dirname(sourcePath)]) + const mappingPath = join(privateDirectory, `${hash(output.toLowerCase()).slice(0, 24)}.json`) + const mapping = existsSync(mappingPath) ? read(mappingPath) : { schemaVersion: 1, reviewSetId: hash(output.toLowerCase()).slice(0, 24), pairs: {} } + assert.equal(mapping.schemaVersion, 1) + const manifest = read(join(input, 'fixtures/manifest.json')) + const source = new Map(read(sourcePath).cases.map(entry => [entry.id, entry])) + const requested = option('onlyids', '').split(',').filter(Boolean) + assert.ok(requested.every(id => /^position-[1-9]\d*$/.test(id) && manifest.cases.some(item => item.id === id)), 'Unknown anonymous position ID') + assert.equal(new Set(requested).size, requested.length, 'Duplicate position IDs') + const selected = requested.length ? requested.map(id => manifest.cases.find(item => item.id === id)) : manifest.cases + const cases = []; const rejected = []; const positions = [] + let passedRuns = 0; let attemptedRuns = 0 + const artifact = path => { try { return read(path) } catch { return null } } + for (const fixture of selected) { + const candidate = artifact(join(input, 'candidate', `${fixture.id}.json`)); const baseline = artifact(join(input, 'baseline', `${fixture.id}.json`)) + const statuses = [runStatus(candidate), runStatus(baseline)] + passedRuns += statuses.filter(status => status.passed).length + attemptedRuns += statuses.filter(status => status.attempted).length + const positionStatus = { id: fixture.id, state: 'unpaired', statuses: statuses.map(status => status.text).sort() } + positions.push(positionStatus) + try { + assert.match(fixture.id, /^position-[1-9]\d*$/) + validatePair(fixture, candidate, baseline) + const original = source.get(fixture.localCaseId); assert.ok(original, 'Missing source snapshot') + assert.ok(within(join(input, 'fixtures'), resolve(fixture.filePath)), 'Fixture outside input fixture directory') + const boards = fixtureBoards(fixture, original, readFileSync(fixture.filePath, 'utf8')) + verifyReplayEvidence(boards, candidate); verifyReplayEvidence(boards, baseline) + const fingerprint = hash(canonical({ fixtureHash: fixture.sha256, candidate: candidate.identity, baseline: baseline.identity, profiles: [candidate.profileHash, baseline.profileHash], texts: [candidate.result.markdown, baseline.result.markdown], boards })) + const prior = mapping.pairs[fixture.id] + assert.ok(!prior || prior.fingerprint === fingerprint, 'Pair changed after assignment; use a separate --out directory') + const a = prior?.a ?? (randomInt(2) ? 'candidate' : 'baseline') + assert.ok(['candidate', 'baseline'].includes(a)) + mapping.pairs[fixture.id] = { a, fingerprint } + const texts = a === 'candidate' ? [candidate.result.markdown, baseline.result.markdown] : [baseline.result.markdown, candidate.result.markdown] + for (const [index, letter] of ['A', 'B'].entries()) cases.push({ id: `${fixture.id}:${letter}`, ...boards, markdown: texts[index], review: { geometryCorrect: '', useful: '', intendedTerm: '', notes: '' } }) + positionStatus.state = 'complete' + } catch (error) { + if (statuses.every(status => status.passed)) positionStatus.state = 'incompatible' + rejected.push({ id: fixture.id, reason: error.message }) + } + } + assert.ok(attemptedRuns > 0, 'No formal real records; no review output generated') + if (existsSync(join(output, 'feedback.json'))) { + const existing = read(join(output, 'feedback.json')) + assert.ok(existing.cases.every(entry => cases.some(item => item.id === entry.id)), 'Existing feedback includes omitted pairs; select them too or use a new output directory') + } + const summary = { plannedPositions: selected.length, completePairs: cases.length / 2, unpairedPositions: selected.length - cases.length / 2, + expectedRuns: selected.length * 2, passedRuns, unpassedRuns: attemptedRuns - passedRuns, pendingRuns: selected.length * 2 - attemptedRuns } + const publicData = { schemaVersion: 1, model: 'gpt-6-luna', reviewSetId: mapping.reviewSetId, summary, positions, cases } + const json = JSON.stringify(publicData, null, 2) + for (const id of source.keys()) assert.ok(!json.includes(id), 'Original case identifier in model commentary; cannot publish anonymously') + assert.ok(!/"(?:variant|localCaseId|labels|weakMentions|feedback|comment)"\s*:/.test(json), 'Private metadata in public cases') + const html = reviewHtml(publicData) + mkdirSync(privateDirectory, { recursive: true }); writeFileSync(mappingPath, JSON.stringify(mapping, null, 2) + '\n') + mkdirSync(output, { recursive: true }) + for (const name of ['cases.json', 'review.html']) assert.ok(!existsSync(join(output, name)) || realpathSync(join(output, name)).toLowerCase() === join(output, name).toLowerCase(), 'Linked output files are unsupported') + writeFileSync(join(output, 'cases.json'), json + '\n'); writeFileSync(join(output, 'review.html'), html) + console.log(JSON.stringify({ output, summary, rejected })) +} +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { main() } catch (error) { console.error(error.message); process.exitCode = 1 } +} diff --git a/scripts/build_commentary_short_preview.mjs b/scripts/build_commentary_short_preview.mjs new file mode 100644 index 0000000..9b3466c --- /dev/null +++ b/scripts/build_commentary_short_preview.mjs @@ -0,0 +1,122 @@ +#!/usr/bin/env node +import assert from 'node:assert/strict' +import { readFileSync, writeFileSync, mkdirSync, existsSync, realpathSync } from 'node:fs' +import { resolve, join, dirname, relative, isAbsolute } from 'node:path' +import { fileURLToPath } from 'node:url' +import { createHash } from 'node:crypto' +import { fixtureBoards } from './build_commentary_blind_review.mjs' + +const root = fileURLToPath(new URL('../', import.meta.url)) +const read = path => JSON.parse(readFileSync(path, 'utf8')) +const hash = text => createHash('sha256').update(text).digest('hex') +const escape = value => String(value).replace(/[&<>"']/g, c => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[c]) +const columns = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +const within = (parent, child) => { const path = relative(parent, child); return path !== '' && !path.startsWith('..') && !isAbsolute(path) } + +function boardSvg(entry, phase) { + const step = 22, offset = 32, end = offset + (entry.boardSize - 1) * step, extent = end + 32 + const locate = point => [offset + columns.indexOf(point[0]) * step, offset + (entry.boardSize - Number(point.slice(1))) * step] + let svg = `` + for (let i = 0; i < entry.boardSize; i++) { + const p = offset + i * step + svg += `${columns[i]}${entry.boardSize - i}` + } + for (const stone of entry[phase]) { + const [x, y] = locate(stone.point) + svg += `` + } + if (!/^pass$/i.test(entry.playedMove)) { + const [x, y] = locate(entry.playedMove) + svg += `` + } + return svg + '' +} + +export function reviewHtml(data) { + const cards = data.cases.map((entry, index) => `

    ${index + 1}. 第 ${entry.moveNumber} 手 · ${entry.playerColor === 'B' ? '黑' : '白'} ${escape(/^pass$/i.test(entry.playedMove) ? '停着' : entry.playedMove)}

    ${boardSvg(entry, 'before')}

    红圈标记本手落点。

    讲解原文

    ${escape(entry.markdown)}
    留下反馈（可选）
    `).join('') + return `围棋短讲解预览

    围棋短讲解预览

    GPT-6 Luna 实际输出 · ${data.cases.length} 个局面

    对照棋盘看本手讲解，点击“落子后”查看变化。

    正在读取已保存的反馈…

    ${cards}
    ` +} + +export function main(args = process.argv.slice(2)) { + if (args.includes('--help')) { console.log('node scripts/build_commentary_short_preview.mjs [--ids position-19,position-20,position-8,position-16] [--result-dir PATH (repeatable)] [--out .tmp/commentary-short-preview]'); return } + const options = new Map() + for (let i = 0; i < args.length; i += 2) { + assert.ok(['--ids', '--result-dir', '--out', '--manifest', '--source'].includes(args[i]), `Unknown option ${args[i]}`) + assert.ok(args[i + 1] && !args[i + 1].startsWith('--'), `Missing ${args[i]} value`) + assert.ok(args[i] === '--result-dir' || !options.has(args[i]), `Duplicate ${args[i]}`) + options.set(args[i], [...(options.get(args[i]) ?? []), args[i + 1]]) + } + const option = (name, fallback) => options.get(name)?.[0] ?? fallback + const ids = option('--ids', 'position-19,position-20,position-8,position-16').split(',') + assert.ok(ids.length >= 1 && ids.length <= 6 && new Set(ids).size === ids.length && ids.every(id => /^position-[1-9]\d*$/.test(id)), 'Select 1–6 unique position IDs') + const manifestPath = resolve(root, option('--manifest', '.tmp/commentary-live-v5/fixtures/manifest.json')) + const sourcePath = resolve(root, option('--source', '.tmp/concept-review-round2/cases.json')) + const resultDirs = (options.get('--result-dir') ?? ['.tmp/commentary-short-preview-results/candidate', '.tmp/commentary-live-v5/candidate']).map(path => resolve(root, path)) + const output = resolve(root, option('--out', '.tmp/commentary-short-preview')) + const temporary = resolve(root, '.tmp') + assert.ok(within(temporary, output), 'Output must be a child of repository .tmp') + for (const input of [dirname(manifestPath), dirname(sourcePath), ...resultDirs]) assert.ok(output !== input && !within(output, input) && !within(input, output), 'Output must be separate from inputs') + let ancestor = output + while (!existsSync(ancestor)) ancestor = dirname(ancestor) + assert.ok(realpathSync(ancestor) === realpathSync(temporary) || within(realpathSync(temporary), realpathSync(ancestor)), 'Output ancestor escapes .tmp') + const fixtures = read(manifestPath).cases + const source = new Map(read(sourcePath).cases.map(entry => [entry.id, entry])) + const cases = ids.map(id => { + const fixture = fixtures.find(entry => entry.id === id) + assert.ok(fixture, `Unknown fixture ${id}`) + const resultPath = resultDirs.map(dir => join(dir, `${id}.json`)).find(existsSync) + assert.ok(resultPath, `Result not ready: ${id}`) + const result = read(resultPath) + assert.equal(result.success, true, `Unsuccessful result ${id}`) + assert.equal(result.id, id) + assert.equal(result.model, 'gpt-6-luna') + assert.equal(result.provider, 'codex-app-server') + assert.equal(result.variant, 'candidate') + assert.equal(result.identity?.fixtureHash, fixture.sha256) + assert.equal(result.request?.moveNumber, fixture.moveNumber) + assert.ok(typeof result.result?.markdown === 'string' && result.result.markdown.trim(), `Missing original commentary ${id}`) + assert.equal(result.boardVerification?.verified, true) + const fixturePath = resolve(fixture.filePath) + assert.ok(within(dirname(manifestPath), fixturePath), 'Fixture outside manifest directory') + const original = source.get(fixture.localCaseId) + assert.ok(original, `Missing source ${id}`) + const boards = fixtureBoards(fixture, original, readFileSync(fixturePath, 'utf8')) + for (const phase of ['before', 'after']) { + const flat = Array(boards.boardSize ** 2).fill(0) + for (const stone of boards[phase]) flat[(Number(stone.point.slice(1)) - 1) * boards.boardSize + columns.indexOf(stone.point[0])] = stone.color === 'B' ? 1 : -1 + assert.deepEqual(result.boardVerification[phase], flat, `Result board differs from fixture ${id}`) + } + return { id, ...boards, markdown: result.result.markdown } + }) + const data = { schemaVersion: 1, model: 'gpt-6-luna', reviewSetId: hash(output.toLowerCase()).slice(0, 24), cases } + for (const name of ['cases.json', 'review.html', 'feedback.json']) { + const path = join(output, name) + assert.ok(!existsSync(path) || realpathSync(path).toLowerCase() === path.toLowerCase(), 'Linked output files unsupported') + } + if (existsSync(join(output, 'cases.json'))) { + for (const previous of read(join(output, 'cases.json')).cases) assert.deepEqual(cases.find(entry => entry.id === previous.id), previous, 'Existing preview changed; use a new --out directory') + } + if (existsSync(join(output, 'feedback.json'))) assert.ok(read(join(output, 'feedback.json')).cases.every(entry => ids.includes(entry.id)), 'Existing feedback would be omitted; use a new --out directory') + mkdirSync(output, { recursive: true }) + writeFileSync(join(output, 'cases.json'), JSON.stringify(data, null, 2) + '\n') + writeFileSync(join(output, 'review.html'), reviewHtml(data)) + console.log(JSON.stringify({ output, cases: ids })) +} + +if (process.argv[1] && resolve(process.argv[1]) === fileURLToPath(import.meta.url)) { + try { main() } catch (error) { console.error(error.message); process.exitCode = 1 } +} diff --git a/scripts/eval_commentary_pipeline.mjs b/scripts/eval_commentary_pipeline.mjs new file mode 100644 index 0000000..3a20c02 --- /dev/null +++ b/scripts/eval_commentary_pipeline.mjs @@ -0,0 +1,228 @@ +#!/usr/bin/env node +import assert from 'node:assert/strict' +import { mkdirSync, readFileSync, writeFileSync, existsSync } from 'node:fs' +import { resolve, join } from 'node:path' +import { createHash } from 'node:crypto' +import { createServer } from 'node:http' +import { readRecords, prepareCases, assertAnonymousSgf, assertSafeBoundary, teacherRequest, connectCdp, validatePipelineEvidence, validateMockCapture, canResume, validateImportedRecord, parseStoredResult, semanticProfile } from './lib/commentary_eval.mjs' + +const args = process.argv.slice(2) +function option(name, fallback) { + const at = args.indexOf(`--${name}`) + return at < 0 ? fallback : args[at + 1] +} +const mode = args.includes('--prepare') ? 'prepare' : args.includes('--real') ? 'real' : args.includes('--mock') ? 'mock' : 'help' +const directory = resolve(option('out', '.tmp/commentary-live')) +const fixturesDir = join(directory, 'fixtures') +const manifestPath = join(fixturesDir, 'manifest.json') +const variant = option('variant', 'candidate') +const phase = args.includes('--diagnostic') ? 'diagnostic' : 'formal' +const prefetchedMaxVisits = Number(option('visits', '96')) +assert.ok(Number.isInteger(prefetchedMaxVisits) && prefetchedMaxVisits >= 40 && prefetchedMaxVisits <= 3000, 'visits must be an integer between 40 and 3000') +assert.ok(['candidate', 'baseline'].includes(variant), 'variant must be candidate or baseline') +assert.ok(!(args.includes('--resume') && args.includes('--new-run')), 'Use --resume or --new-run, not both') +const casesPath = resolve(option('cases', '.tmp/concept-review-round2/cases.json')) +const dataPath = resolve(option('data', '.tmp/concept-data/comments.jsonl')) +const enginePath = option('engine', '') +const promptFile = option('prompt-file', '') +const promptOverride = promptFile ? readFileSync(resolve(promptFile), 'utf8').trim() : undefined +assert.ok(!promptFile || (promptOverride && promptOverride.length <= 4096), 'Prompt override must be nonempty and at most 4096 characters') +const digest = value => createHash('sha256').update(value).digest('hex') + +async function mockServer() { + const captured = [] + let current = null + const server = createServer((req, res) => { + let raw = '' + req.setEncoding('utf8') + req.on('data', chunk => { raw += chunk }) + req.on('end', () => { + try { + if (req.url?.endsWith('/models')) { res.writeHead(200, { 'Content-Type': 'application/json' }); res.end(JSON.stringify({ data: [{ id: 'commentary-eval-mock' }] })); return } + const body = JSON.parse(raw || '{}') + captured.push({ position: current?.id ?? 'probe', body }) + const messages = body.messages ?? [] + const names = new Set((body.tools ?? []).map(tool => tool.function.name)) + const tools = messages.filter(message => message.role === 'tool') + const needed = current ? [ + ['sgf_readGameRecord', { gameId: current.gameId, maxMoves: current.moveNumber }], + ['knowledge_matchPosition', { moveNumber: current.moveNumber, maxResults: 4 }], + ['katago_analyzePosition', { gameId: current.gameId, moveNumber: current.moveNumber, maxVisits: 96 }], + ['board_captureTeachingImage', { moveNumber: current.moveNumber }] + ].filter(([name]) => names.has(name) && !tools.some(tool => tool.name === name || tool.tool_call_id === `eval_${name}`)) : [] + const calls = needed.map(([name, input]) => ({ id: `eval_${name}`, type: 'function', function: { name, arguments: JSON.stringify(input) } })) + if (!current && names.has('readiness_check')) calls.push({ id: 'eval_readiness_check', type: 'function', function: { name: 'readiness_check', arguments: '{"value":"goagent"}' } }) + const probeText = JSON.stringify(messages).includes('BLUE') ? 'BLUE' : 'OK' + const message = calls.length ? { role: 'assistant', content: '', tool_calls: calls } : { role: 'assistant', content: current ? `当前第${current.moveNumber}手的讲解应以棋谱、棋盘截图与 KataGo 候选证据为依据。先核对落点与局部气数，再比较实战手和候选点；局部目的及后续仍需结合变化验证，不能仅凭术语断言。` : probeText } + if (body.stream) { + res.writeHead(200, { 'Content-Type': 'text/event-stream' }) + const delta = calls.length ? { tool_calls: calls.map((call, index) => ({ index, ...call })) } : { content: message.content } + res.end(`data: ${JSON.stringify({ choices: [{ index: 0, delta, finish_reason: calls.length ? 'tool_calls' : 'stop' }] })}\n\ndata: [DONE]\n\n`) + } else { + res.writeHead(200, { 'Content-Type': 'application/json' }) + res.end(JSON.stringify({ choices: [{ message, finish_reason: calls.length ? 'tool_calls' : 'stop' }] })) + } + } catch { res.writeHead(500); res.end('Mock request rejected') } + }) + }) + await new Promise(resolveListen => server.listen(0, '127.0.0.1', resolveListen)) + return { captured, endpoint: `http://127.0.0.1:${server.address().port}/v1`, setCase(value) { current = value }, close() { return new Promise(resolveClose => server.close(resolveClose)) } } +} + +async function main() { + if (mode === 'help') { + console.log('Usage: node scripts/eval_commentary_pipeline.mjs --prepare|--mock|--real [--cases PATH] [--data JSONL] [--engine JSON] [--out DIR] [--cdp PORT] [--bundle PATH] [--variant candidate|baseline] [--onlyids position-1,position-2] [--limit 6] [--visits 96] [--resume|--new-run] [--diagnostic] [--continue-on-error] [--model gpt-6-luna]') + return + } + if (mode === 'prepare') { + const fixtures = prepareCases(readRecords(casesPath), readRecords(dataPath), enginePath ? readRecords(resolve(enginePath)) : [], fixturesDir) + if (existsSync(manifestPath)) { + const previous = JSON.parse(readFileSync(manifestPath, 'utf8')) + const byLocalId = new Map(previous.cases.map(item => [item.localCaseId, item])) + for (const fixture of fixtures) { + const prior = byLocalId.get(fixture.localCaseId) + assert.ok(prior, 'Existing manifest differs from selected cases; use a separate --out directory') + assert.match(prior.id, /^position-\d+$/, 'Invalid anonymous fixture ID') + fixture.id = prior.id + fixture.filePath = join(fixturesDir, `${prior.id}.sgf`) + } + fixtures.sort((a, b) => Number(a.id.split('-').at(-1)) - Number(b.id.split('-').at(-1))) + } + mkdirSync(fixturesDir, { recursive: true }) + for (const fixture of fixtures) writeFileSync(fixture.filePath, fixture.sgf + '\n') + const manifest = { schemaVersion: 1, createdAt: new Date().toISOString(), files: fixtures.map(item => item.filePath), cases: fixtures.map(({ sgf, ...item }) => item) } + writeFileSync(manifestPath, JSON.stringify(manifest, null, 2) + '\n') + console.log(JSON.stringify({ mode, cases: fixtures.length, manifestPath, absentRules: fixtures.filter(item => !item.rules).length })) + return + } + assert.ok(existsSync(manifestPath), 'Run --prepare first') + const manifest = JSON.parse(readFileSync(manifestPath, 'utf8')) + const manifestIds = new Set(manifest.cases.map(item => item.localCaseId)) + const privateRecords = readRecords(dataPath).filter(item => manifestIds.has(item.sample_id)) + const onlyIds = option('onlyids', '').split(',').filter(Boolean) + const limit = Number(option('limit', '6')) + assert.ok(Number.isInteger(limit) && limit > 0, 'limit must be a positive integer') + let selected = manifest.cases.filter(item => !onlyIds.length || onlyIds.includes(item.id)).slice(0, limit) + assert.ok(selected.length, 'No cases selected') + for (const fixture of selected) { + const sgf = readFileSync(fixture.filePath, 'utf8').trim() + assertAnonymousSgf(sgf) + assert.equal(createHash('sha256').update(sgf).digest('hex'), fixture.sha256, 'Anonymous fixture differs from manifest') + } + if (mode === 'mock' && args.includes('--boundary-only')) { + const boundary = selected.map(fixture => ({ id: fixture.id, sgf: readFileSync(fixture.filePath, 'utf8'), request: teacherRequest(fixture, 'anonymous-game-hash', variant) })) + assertSafeBoundary(boundary, manifest.cases, privateRecords) + mkdirSync(join(directory, 'mock'), { recursive: true }) + writeFileSync(join(directory, 'mock', 'boundary.json'), JSON.stringify(boundary, null, 2)) + console.log(JSON.stringify({ mode, cases: selected.length, boundaryPassed: true, providerRequestCapture: false, note: 'Fixture/request boundary only. Use mock-provider mode for production provider capture.' })) + return + } + const outputDir = join(directory, mode === 'mock' ? 'mock' : variant) + mkdirSync(outputDir, { recursive: true }) + const bundlePath = resolve(option('bundle', 'out/main/index.js')) + assert.ok(variant !== 'baseline' || option('bundle', ''), 'Baseline evaluation requires --bundle pointing to the running baseline main bundle') + assert.ok(existsSync(bundlePath), 'Application bundle not found') + const bundleHash = digest(readFileSync(bundlePath)) + const client = await connectCdp(Number(option('cdp', mode === 'mock' ? '9225' : variant === 'baseline' ? '9224' : '9223'))) + let mock = null + try { + const runningBuild = await client.evaluate('({bundleHash:window.__goagentCommentaryEvalBuild?.bundleHash})') + assert.equal(runningBuild?.bundleHash, bundleHash, 'Running application bundle differs from --bundle; use the evaluation bootstrap for loaded-build verification') + if (mode === 'mock') { + // Refuse configuration writes unless the existing instance explicitly identifies itself as the isolated mock profile. + const profileId = await client.evaluate('(async()=>{const d=await window.goagent.getDashboard();return d.settings.activeLlmConnectionId;})()') + assert.equal(profileId, 'commentary-eval-mock', 'Mock requires an isolated apphome with commentary-eval-mock active profile') + mock = await mockServer() + await client.evaluate(`(async()=>{const d=await window.goagent.getDashboard();await window.goagent.updateSettings({activeLlmConnectionId:'commentary-eval-mock',llmBaseUrl:${JSON.stringify(mock.endpoint)},llmModel:'commentary-eval-mock',llmApiKey:'local-eval-dummy',llmConnections:d.settings.llmConnections.map(p=>p.id==='commentary-eval-mock'?{...p,provider:'openai-compatible',authMode:'api-key',endpoint:${JSON.stringify(mock.endpoint)},model:'commentary-eval-mock',enabled:true}:p)});const p=await window.goagent.testLlmSettings({llmBaseUrl:'',llmApiKey:'',llmModel:''});return {ok:p.ok};})()`) + } + const expectedModel = mode === 'mock' ? 'commentary-eval-mock' : option('model', 'gpt-6-luna') + assert.ok(mode === 'mock' || expectedModel === 'gpt-6-luna', 'Real commentary evaluation is fixed to the user-selected gpt-6-luna') + const state = await client.evaluate(`(async()=>{const d=await window.goagent.getDashboard(); const p=d.settings.llmConnections.find(x=>x.id===d.settings.activeLlmConnectionId); return {ready:d.systemProfile?.llmConnection?.ready,provider:p?.provider,model:p?.model,runtimePath:p?.executablePath,katagoConfig:d.settings.katagoConfig,connectionStatus:d.systemProfile?.llmConnection?.status,settings:{reviewLanguage:d.settings.reviewLanguage,teacherTerminologyDensity:d.settings.teacherTerminologyDensity,teacherExplanationPace:d.settings.teacherExplanationPace,teacherVariationDetail:d.settings.teacherVariationDetail,katagoEngineMode:d.settings.katagoEngineMode,katagoModelPreset:d.settings.katagoModelPreset,katagoAnalysisSpeedMode:d.settings.katagoAnalysisSpeedMode,katagoAnalysisThreads:d.settings.katagoAnalysisThreads,katagoSearchThreadsPerAnalysisThread:d.settings.katagoSearchThreadsPerAnalysisThread}};})()`) + assert.equal(state.ready, true, 'Application LLM connection is not ready') + assert.equal(state.provider, mode === 'mock' ? 'openai-compatible' : 'codex-app-server', 'Unexpected provider for evaluation mode') + assert.equal(state.model, expectedModel, 'Active model does not match requested model') + assert.ok(state.katagoConfig && existsSync(state.katagoConfig), 'KataGo generated config is missing') + const engineConfig = readFileSync(state.katagoConfig, 'utf8') + const configuredHome = [...engineConfig.matchAll(/^\s*homeDataDir\s*=\s*(.*?)\s*$/gm)].at(-1)?.[1] + const expectedHome = resolve(option('tuning-home', '.tmp/concept-runtime/KataGoData')) + assert.ok(configuredHome && resolve(configuredHome).toLowerCase() === expectedHome.toLowerCase(), 'KataGo config must explicitly use the evaluation tuning home before any analysis starts') + await client.evaluate('(async()=>{const r=await window.goagent.importLibrary(); return {imported:r.imported.length};})()') + const games = await client.evaluate('(async()=>{const d=await window.goagent.getDashboard();return d.games.map(g=>({id:g.id,filePath:g.filePath}));})()') + const runtimeHash = state.runtimePath && existsSync(state.runtimePath) ? digest(readFileSync(state.runtimePath)) : null + const engineConfigHash = digest(engineConfig) + const meta = { schemaVersion: 2, variant, phase, provider: state.provider, model: state.model, bundleHash, runtimeHash, engineConfigHash, evaluatorRetryPolicy: 'Single attempt per selected case; production adapter retries/vision repairs remain part of the normal pipeline.', inferenceSettings: { effort: 'provider-default', temperature: 'provider-default', note: 'Production Codex adapter sets no explicit effort/temperature; paired runs must share the same CLI binary and account config.' }, createdAt: new Date().toISOString(), cases: selected.map(item => item.id), applicationSettings: state.settings, requestSettings: { coachLevel: 'intermediate', studentAgeRange: 'adult', teacherStyle: 'rigorous', toolPolicy: 'auto', prefetchedMaxVisits }, baselineMethod: variant === 'baseline' ? 'External application bundle must be the baseline commit; this script uses identical request settings.' : undefined } + const runStamp = `${Date.now()}` + const runDir = join(outputDir, 'runs', runStamp) + const identities = new Map(selected.map(fixture => [fixture.id, { model: state.model, provider: state.provider, variant, phase, fixtureHash: fixture.sha256, bundleHash, runtimeHash, engineConfigHash, promptHash: digest(teacherRequest(fixture, '', variant, promptOverride).prompt), settingsHash: digest(JSON.stringify({ app: state.settings, request: meta.requestSettings })) }])) + // Validate existing results before writing any run metadata. Successful results from another setup never count as a resumed case. + for (const fixture of selected) { + const path = join(outputDir, `${fixture.id}.json`) + if (!existsSync(path)) continue + const previous = parseStoredResult(readFileSync(path, 'utf8')) + if (args.includes('--resume') && previous.success) assert.ok(canResume(previous, identities.get(fixture.id)), `Resume identity mismatch for ${fixture.id}; choose a separate output directory`) + else if (previous.success && !args.includes('--new-run')) throw new Error(`Successful result already exists for ${fixture.id}; use --resume, --new-run or a separate output directory`) + } + mkdirSync(runDir, { recursive: true }) + writeFileSync(join(runDir, 'run.json'), JSON.stringify(meta, null, 2)) + if (!existsSync(join(outputDir, 'run.json'))) writeFileSync(join(outputDir, 'run.json'), JSON.stringify(meta, null, 2)) + const outcomes = [] + for (const fixture of selected) { + const resultPath = join(outputDir, `${fixture.id}.json`) + const identity = identities.get(fixture.id) + if (args.includes('--resume') && existsSync(resultPath) && canResume(parseStoredResult(readFileSync(resultPath, 'utf8')), identity)) { outcomes.push({ id: fixture.id, success: true, resumed: true }); console.log(`${variant} ${fixture.id}: resumed`); continue } + if (existsSync(resultPath)) writeFileSync(join(runDir, `${fixture.id}.previous.json`), readFileSync(resultPath)) + const content = readFileSync(fixture.filePath, 'utf8') + const hash = createHash('sha1').update(content).digest('hex').slice(0, 12) + const matches = games.filter(game => game.id === hash) + assert.equal(matches.length, 1, 'Imported anonymous fixture hash must match exactly one game') + const gameId = matches[0].id + const imported = await client.evaluate(`window.goagent.getGameRecord(${JSON.stringify(gameId)}).then(r=>({boardSize:r.boardSize,moves:r.moves.map(m=>({color:m.color,gtp:m.gtp,row:m.row,col:m.col,pass:m.pass})),initialStones:r.initialStones}))`) + assert.equal(imported.boardSize, fixture.boardSize) + assert.equal(imported.moves.length, fixture.moveNumber) + const source = privateRecords.find(item => item.sample_id === fixture.localCaseId) + assert.ok(source, 'Local source record not found') + const boardVerification = validateImportedRecord(imported, source) + const request = teacherRequest(fixture, gameId, variant, promptOverride) + if (phase === 'diagnostic') request.runId += `-diagnostic-${runStamp}` + const start = Date.now() + mock?.setCase({ ...fixture, gameId }) + console.log(`${variant} ${fixture.id}: running`) + let payload + try { + payload = await client.evaluate(`(async()=>{const request=${JSON.stringify(request)};let analysis;const progress=[];const dispose=window.goagent.onTeacherRunProgress(p=>{if(p.runId===request.runId)progress.push(p)});try{analysis=await window.goagent.analyzePosition({gameId:request.gameId,moveNumber:request.moveNumber,maxVisits:${prefetchedMaxVisits}});const result=await window.goagent.runTeacherTask({...request,prefetchedAnalysis:analysis});return {analysis,result,progress};}catch(error){return {analysis,progress,pipelineError:String(error)};}finally{dispose();}})()`) + assert.ok(!payload.pipelineError, payload.pipelineError) + validatePipelineEvidence(payload, fixture) + if (mock) { + const requests = mock.captured.filter(item => item.position === fixture.id) + writeFileSync(join(runDir, `${fixture.id}.requests.json`), JSON.stringify(requests, null, 2)) + assertSafeBoundary(requests, manifest.cases, privateRecords) + validateMockCapture(requests, { gameId, moveNumber: fixture.moveNumber, requireConceptEvidence: variant === 'candidate' }) + writeFileSync(join(runDir, `${fixture.id}.boundary.json`), JSON.stringify({ passed: true, capturedRequests: requests.length, scope: 'Production OpenAI-compatible teacher messages and tool outputs; Codex stdio adapter is validated separately by real execution.' }, null, 2)) + } + const profileHash = digest(JSON.stringify(semanticProfile(payload.result.studentProfile))) + const artifact = JSON.stringify({ ...meta, id: fixture.id, identity, success: true, elapsedMs: Date.now() - start, request, boardVerification, profileHash, accountCondition: 'Same account and account config is an operational prerequisite; authentication data is not inspected.', ...payload }, null, 2) + writeFileSync(join(runDir, `${fixture.id}.json`), artifact) + writeFileSync(resultPath, artifact) + writeFileSync(join(runDir, `${fixture.id}.md`), payload.result.markdown) + writeFileSync(join(outputDir, `${fixture.id}.md`), payload.result.markdown) + outcomes.push({ id: fixture.id, success: true, elapsedMs: Date.now() - start }) + console.log(`${variant} ${fixture.id}: saved`) + } catch (error) { + const failed = JSON.stringify({ ...meta, id: fixture.id, identity, success: false, elapsedMs: Date.now() - start, request, ...payload, error: String(error) }, null, 2) + writeFileSync(join(runDir, `${fixture.id}.json`), failed) + writeFileSync(resultPath, failed) + if (mock) writeFileSync(join(runDir, `${fixture.id}.requests.json`), JSON.stringify(mock.captured.filter(item => item.position === fixture.id), null, 2)) + outcomes.push({ id: fixture.id, success: false, elapsedMs: Date.now() - start, error: String(error) }) + console.log(`${variant} ${fixture.id}: failed; artifact saved`) + if (!args.includes('--continue-on-error')) throw new Error(`${variant} ${fixture.id} failed; see local error artifact`) + } + } + const summary = { ...meta, denominator: selected.length, succeeded: outcomes.filter(item => item.success).length, failed: outcomes.filter(item => !item.success).length, resumed: outcomes.filter(item => item.resumed).length, completedAt: new Date().toISOString(), outcomes } + writeFileSync(join(runDir, 'summary.json'), JSON.stringify(summary, null, 2)) + if (existsSync(join(outputDir, 'summary.json'))) writeFileSync(join(runDir, 'previous-summary.json'), readFileSync(join(outputDir, 'summary.json'))) + writeFileSync(join(outputDir, 'summary.json'), JSON.stringify(summary, null, 2)) + console.log(JSON.stringify({ variant, phase, denominator: summary.denominator, succeeded: summary.succeeded, failed: summary.failed, summaryPath: join(runDir, 'summary.json') })) + if (summary.failed) process.exitCode = 1 + } finally { client.close(); if (mock) await mock.close() } +} +main().catch(error => { console.error(error.message); process.exitCode = 1 }) diff --git a/scripts/eval_move_concepts.mjs b/scripts/eval_move_concepts.mjs new file mode 100644 index 0000000..a71def1 --- /dev/null +++ b/scripts/eval_move_concepts.mjs @@ -0,0 +1,145 @@ +import { readFileSync, writeFileSync, mkdirSync, existsSync } from 'node:fs' +import { join } from 'node:path' +import { loadConceptModule } from './lib/concept_modules.mjs' + +const { recognizeMoveConcepts } = await loadConceptModule('src/main/services/knowledge/moveConcepts.ts') +const baselineCommit = 'a5c801c' +const { recognizeShapes } = await loadConceptModule('src/main/services/knowledge/shapeRecognitionEngine.ts', baselineCommit) +const dataPath = process.argv[2] ?? '.tmp/concept-data/comments.jsonl' +const output = process.argv[3] ?? '.tmp/concept-review' +// A later round can retain the original sample identities despite changed concepts. +const pinnedCases = process.argv[4] ? JSON.parse(readFileSync(process.argv[4], 'utf8')).cases : undefined +const pinnedIds = pinnedCases && new Set(pinnedCases.map((entry) => entry.id)) +const priorFeedback = process.argv[5] ? JSON.parse(readFileSync(process.argv[5], 'utf8')).cases : [] +const priorById = new Map(priorFeedback.map((entry) => [entry.id, entry])) +if (!existsSync(dataPath)) throw new Error('先运行 python scripts/experiments/prepare_comment_concepts.py --games 1000') +const rows = readFileSync(dataPath, 'utf8').trim().split('\n').map((line) => JSON.parse(line)) +const columns = 'ABCDEFGHJKLMNOPQRST' +const cases = [] +const counts = { positions: 0, abstained: 0, ambiguous: 0, observed: 0, extensionHypotheses: 0 } +const buckets = new Map() +const games = new Set() +for (const row of rows.filter((row) => row.split === 'test')) { + const boardSnapshot = row.board_before.flatMap((value, index) => value ? [{ color: value === 1 ? 'B' : 'W', point: `${columns[index % 19]}${Math.floor(index / 19) + 1}` }] : []) + const [playerColor, playedMove] = row.played_move + const moveHistory = row.moves_before.map(([color, point]) => ({ color, point })) + const concepts = recognizeMoveConcepts({ boardSize: 19, boardSnapshot, playedMove, playerColor, moveHistory }) + counts.positions++ + counts[concepts.status]++ + if (concepts.intents.some((intent) => intent.kind === 'side-extension')) counts.extensionHypotheses++ + const observed = concepts.geometry.filter((geometry) => geometry.status === 'observed') + const bucket = concepts.intents.some((intent) => intent.kind === 'side-extension') ? 'side-extension' : concepts.status === 'ambiguous' ? 'ambiguous' : observed[0]?.kind + if (pinnedIds) { + if (!pinnedIds.has(row.sample_id)) continue + } else { + if (!['side-extension', 'ambiguous', 'one-space-jump', 'small-knight', 'diagonal', 'two-space-jump', 'large-knight'].includes(bucket)) continue + if ((buckets.get(bucket) ?? 0) >= 3 || games.has(row.source_game_key ?? row.game_sha256)) continue + } + buckets.set(bucket, (buckets.get(bucket) ?? 0) + 1) + games.add(row.source_game_key ?? row.game_sha256) + const moveNumber = row.moves_after.length + const legacy = recognizeShapes({ boardSize: 19, boardSnapshot, playedMove, playerColor, moveNumber, totalMoves: row.moves_after.length, recentMoves: [], maxResults: 4 }) + cases.push({ id: row.sample_id, gameId: row.game_id, moveNumber, playerColor, playedMove, boardSnapshot, moveHistory, initialStones: row.initial_stones, bucket, + weakMentions: Object.keys(row.labels).filter((key) => row.labels[key]), concepts, + legacy: legacy.map((shape) => ({ shapeType: shape.shapeType, wording: shape.recognition, evidence: shape.evidence, safeWording: shape.safeWording })), + review: { geometryCorrect: null, intendedTerm: '', intentionCorrect: null, useful: null, notes: '' } }) +} +if (pinnedCases) { + const order = new Map(pinnedCases.map((entry, index) => [entry.id, index])) + cases.sort((a, b) => order.get(a.id) - order.get(b.id)) + if (cases.length !== pinnedCases.length) throw new Error('Pinned review cases missing from the data') +} +mkdirSync(output, { recursive: true }) +writeFileSync(join(output, 'cases.json'), JSON.stringify({ baselineCommit, warning: '自动词语标签不是围棋真值。以下是概念层候选措辞，不是最终LLM解说。', counts, cases }, null, 2)) +const escape = (value) => String(value).replace(/[&<>"']/g, (char) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' })[char]) +function boardSvg(entry) { + const step = 22, offset = 32, size = 19, end = offset + (size - 1) * step + let svg = `` + for (let i = 0; i < size; i++) { + const point = offset + i * step + svg += `${columns[i]}${size-i}` + } + const point = (gtp) => [offset + columns.indexOf(gtp[0]) * step, offset + (19 - Number(gtp.slice(1))) * step] + for (const stone of entry.boardSnapshot) { + const [x, y] = point(stone.point) + svg += `` + } + for (const anchor of entry.concepts.geometry.filter((g) => g.status === 'observed')) { + const [x, y] = point(anchor.anchor) + svg += `` + } + for (const target of new Set((entry.concepts.relationCandidates ?? []).flatMap((candidate) => candidate.enemyAnchor ? [candidate.enemyAnchor] : []))) { + const [x, y] = point(target) + svg += `` + } + const [x, y] = point(entry.playedMove) + svg += `` + return svg + '' +} +const cards = cases.map((entry, index) => { + const prior = priorById.get(entry.id) + const previous = pinnedCases?.find((old) => old.id === entry.id) + const candidates = entry.concepts.relationCandidates ?? [] + const history = entry.moveHistory.slice(-4).map((move) => `${move.color} ${move.point}`).join(' → ') + return `
    ${boardSvg(entry)}

    落子前局面 · 红圈：本手 ${entry.playerColor} ${escape(entry.playedMove)} · 蓝圈：己棋锚点 · 橙框：敌方目标

    最近实战：${escape(history || '无')}。初始摆子 ${entry.initialStones?.length ?? 0} 个；手数不含摆子。

    ${index+1}. ${escape(entry.id)} · 第 ${entry.moveNumber} 手

    ${escape(entry.bucket)} · 原棋谱

    + ${prior ? `
    你对第一版的反馈

    ${escape(prior.intendedTerm)}

    ${escape(prior.notes)}

    第一版评价：${escape(prior.geometryCorrect)}；${escape(prior.useful)}。这些反馈用于修改，不计入第二版独立准确率。

    ` : ''} + ${previous && prior ? `

    第一版：${escape(previous.concepts.wording)}

    ` : ''} +

    ${pinnedCases ? '第二版：关系与目的候选' : '分层概念候选'}

    ${escape(entry.concepts.wording)}

    + ${candidates.map((candidate) => `${candidate.purposeHypothesis ? `

    目的假设：${escape(candidate.purposeHypothesis.wording)}

    ` : ''}`).join('')} + ${entry.concepts.intents.map((intent) => `

    ${escape(intent.wording)}

    `).join('')} +
    关系证据和仍待验证的地方
      ${entry.concepts.relations.map((relation) => `
    • ${escape(relation.evidence)}
      • `).join('')}${candidates.flatMap((candidate) => [...candidate.evidence, ...candidate.counterEvidence]).map((text) => `
    • ${escape(text)}
      • `).join('')}${entry.concepts.geometry.map((g) => `
    • ${escape(g.wording)}
      • `).join('')}${entry.concepts.intents.flatMap((intent) => intent.counterEvidence).map((text) => `
    • ${escape(text)}
      • `).join('')}
    +
    原局部规则输出${entry.legacy.length ? entry.legacy.map((shape) => `

    ${escape(shape.safeWording)}：${escape(shape.wording)}

    `).join('') : '

    没有匹配。这里没有候选评价数据，不能据此比较完整解说。

    '}
    +

    原评论提及：${escape(entry.weakMentions.join(', ') || '没有实验标签')}。这不是本手正确术语。

    ` +}).join('') +const html = `围棋概念人工审阅

    围棋概念人工审阅

    共 ${cases.length} 个不同棋谱的测试局面。请判断：与具体敌棋或角部的关系术语是否正确、目的解释是否有帮助。前6例用于修正；后15例仍待独立审阅。

    这里展示真实概念函数的输出，尚未验证最终 LLM 解说质量。研究数据按整盘隔离；评论标签仅用于抽样。反馈自动保存到本浏览器，也可以下载后交给 Codex。

    ${cards}
    ` +const projectSaveScript = `` +writeFileSync(join(output, 'review.html'), html + .replace('', '<link rel="icon" href="data:,"><title>') + .replace('反馈自动保存到本浏览器，也可以下载后交给 Codex。', '填写时自动暂存在本浏览器；点“保存反馈到项目”后，Codex 可直接读取。下载 JSON 仅用于备份。') + .replace('</html>', projectSaveScript + '</html>')) +console.log(JSON.stringify({ output, counts, reviewCases: cases.length, note: 'Coverage and abstention only; no expert semantic accuracy claimed.' })) diff --git a/scripts/experiments/collect_katago_concepts.py b/scripts/experiments/collect_katago_concepts.py new file mode 100644 index 0000000..425f637 --- /dev/null +++ b/scripts/experiments/collect_katago_concepts.py @@ -0,0 +1,95 @@ +"""Collect official KataGo BLACK ownership/policy. Errors never become zero maps.""" +import argparse +import hashlib +import json +import math +from pathlib import Path +import queue +import subprocess +import threading +import time + + +def sha(path): return hashlib.sha256(Path(path).read_bytes()).hexdigest() + + +def normalized_rules(raw): + aliases = {'japanese':'japanese', 'japan':'japanese', 'jp':'japanese', + 'chinese':'chinese', 'china':'chinese', 'cn':'chinese', + 'aga':'aga', 'korean':'korean', 'korea':'korean', + 'new zealand':'new-zealand', 'new-zealand':'new-zealand', 'nz':'new-zealand', + 'tromp-taylor':'tromp-taylor', 'tromp taylor':'tromp-taylor', 'tt':'tromp-taylor', + 'stone-scoring':'stone-scoring'} + value = aliases.get(str(raw).strip().lower()) + return (value, None) if value else ('japanese', 'unknown or missing SGF rules; Japanese fallback') + + +def main(): + p=argparse.ArgumentParser() + for name in ('data','binary','model','config','out'): p.add_argument('--'+name,required=True) + p.add_argument('--max-samples',type=int,default=300) + p.add_argument('--seed',type=int,default=20261002) + p.add_argument('--timeout',type=float,default=120) + p.add_argument('--visits',type=int,default=16) + args=p.parse_args() + rows=[json.loads(s) for s in Path(args.data).read_text(encoding='utf-8').splitlines() if s.strip()] + rows=sorted(rows,key=lambda r:hashlib.sha256((str(args.seed)+r['sample_id']).encode()).hexdigest())[:args.max_samples] + out=Path(args.out); out.parent.mkdir(parents=True,exist_ok=True) + version=subprocess.run([args.binary,'version'],capture_output=True,text=True,timeout=30) + meta={'binary_sha256':sha(args.binary),'model_sha256':sha(args.model),'model_file':Path(args.model).name, + 'version':version.stdout.strip(),'config_sha256':sha(args.config),'perspective':'BLACK','visits':args.visits, + 'data_sha256':sha(args.data),'seed':args.seed} + meta['alignment'] = 'after played move: predicts comment mentions; does not explain intent from pre-move candidate comparison' + Path(str(out)+'.meta.json').write_text(json.dumps(meta,indent=2),encoding='utf-8') + log=open(str(out)+'.stderr.log','w',encoding='utf-8') + proc=subprocess.Popen([args.binary,'analysis','-model',args.model,'-config',args.config],stdin=subprocess.PIPE, + stdout=subprocess.PIPE,stderr=log,text=True,bufsize=1) + q=queue.Queue() + def reader(): + for line in proc.stdout: + try: q.put(json.loads(line)) + except json.JSONDecodeError: q.put({'invalid_output':line[:500]}) + q.put({'process_eof':True}) + threading.Thread(target=reader,daemon=True).start() + try: + with out.open('w',encoding='utf-8') as f: + for sample_index, row in enumerate(rows, 1): + rule, fallback = normalized_rules(row.get('rules')) + query={'id':row['sample_id'],'boardXSize':19,'boardYSize':19,'rules':rule, + 'komi':float(row.get('komi',6.5)), 'initialStones':row['initial_stones'], + 'moves':row['moves_after'],'maxVisits':args.visits,'includeOwnership':True,'includePolicy':True, + 'overrideSettings':{'reportAnalysisWinratesAs':'BLACK'}} + start=time.monotonic(); record={'sample_id':row['sample_id'], 'rules_source':row.get('rules'), + 'rules_used':rule, 'rules_fallback':fallback, 'komi_source':row.get('komi'), 'komi_used':query['komi']} + try: + proc.stdin.write(json.dumps(query)+'\n'); proc.stdin.flush() + while True: + remaining=args.timeout-(time.monotonic()-start) + if remaining<=0: raise TimeoutError('analysis timeout') + result=q.get(timeout=remaining) + if result.get('process_eof'): raise RuntimeError('KataGo exited') + if result.get('id') != row['sample_id']: continue + if result.get('error'): raise RuntimeError(result['error']) + if result.get('isDuringSearch'): continue + own=result.get('ownership'); policy=result.get('policy') + if own is None or policy is None or len(own)!=361 or len(policy)!=362: raise RuntimeError('missing or invalid feature map') + if not all(isinstance(v,(int,float)) and math.isfinite(v) and -1<=v<=1 for v in own+policy): + raise RuntimeError('nonfinite or out-of-range feature map') + moves = [{k:m.get(k) for k in ('move','prior','visits','pv','winrate','scoreLead')} + for m in result.get('moveInfos',[])[:3]] + record.update(status='ok',ownership=own,policy=policy,rootInfo=result.get('rootInfo',{}),moveInfos=moves); break + except (queue.Empty,TimeoutError,RuntimeError,BrokenPipeError) as exc: + record.update(status='error',error=str(exc) or 'analysis timeout; no features emitted') + record['elapsed_seconds']=time.monotonic()-start + f.write(json.dumps(record)+'\n'); f.flush() + if sample_index % 25 == 0 or sample_index == len(rows) or record['status'] != 'ok': + print(sample_index, '/', len(rows), record['sample_id'], record['status'],flush=True) + if proc.poll() is not None: break + finally: + proc.terminate() + try: proc.wait(timeout=10) + except subprocess.TimeoutExpired: proc.kill(); proc.wait() + log.close() + + +if __name__=='__main__': main() diff --git a/scripts/experiments/collect_review_analysis.py b/scripts/experiments/collect_review_analysis.py new file mode 100644 index 0000000..6e6ab92 --- /dev/null +++ b/scripts/experiments/collect_review_analysis.py @@ -0,0 +1,215 @@ +"""Collect ordinary KataGo before/after evidence without disclosing review text. + +The .anonymous.json sibling is the only payload intended for an LLM. The main +output contains local case mapping and reproducibility information. +""" +import argparse +import copy +import json +import math +from pathlib import Path +import queue +import subprocess +import threading +import time + +from collect_katago_concepts import normalized_rules, sha + + +LIMITATION = ('PV is a searched continuation, not proof of player intent. ' + 'Unknown or missing source rules use a Japanese fallback; ' + '128 visits and external model evidence do not establish concept truth.') + + +def build_requests(row, anonymous_id, visits=128, ownership=False): + """Whitelist engine fields and require the exact played-move history delta.""" + before, after = row['moves_before'], row['moves_after'] + played = row['played_move'] + if after != before + [played]: + raise ValueError('moves_after must equal moves_before plus played_move') + size = row['board_size'] + if not isinstance(size, int) or isinstance(size, bool) or not 2 <= size <= 25: + raise ValueError('invalid board_size') + komi = float(row['komi']) + if not math.isfinite(komi): + raise ValueError('invalid komi') + rules, fallback = normalized_rules(row.get('rules')) + base = {'boardXSize': size, 'boardYSize': size, 'rules': rules, 'komi': komi, + 'initialStones': copy.deepcopy(row['initial_stones']), + 'maxVisits': visits, 'includeOwnership': ownership, + 'overrideSettings': {'reportAnalysisWinratesAs': 'BLACK'}} + requests = {} + for phase, moves in (('pre', before), ('post', after)): + requests[phase] = dict(copy.deepcopy(base), id=f'{anonymous_id}-{phase}', + moves=copy.deepcopy(moves)) + return requests, fallback + + +def finite_number(value): + return isinstance(value, (int, float)) and not isinstance(value, bool) and math.isfinite(value) + + +def validate_result(result, request): + if result.get('id') != request['id']: + raise ValueError('unexpected response id') + if result.get('error'): + raise ValueError('KataGo: ' + str(result['error'])) + root = result.get('rootInfo') + moves = result.get('moveInfos') + if not isinstance(root, dict) or not isinstance(moves, list) or not moves: + raise ValueError('missing rootInfo or moveInfos') + for info in [root] + moves[:5]: + for field in ('visits', 'winrate', 'scoreLead'): + if not finite_number(info.get(field)): + raise ValueError('missing or nonfinite ' + field) + if not 0 <= info['winrate'] <= 1 or info['visits'] <= 0: + raise ValueError('invalid winrate or visits') + for move in moves[:5]: + if not isinstance(move.get('move'), str) or not isinstance(move.get('pv'), list) or not move['pv']: + raise ValueError('missing move or PV') + if move['pv'][0] != move['move'] or not all(isinstance(p, str) for p in move['pv']): + raise ValueError('PV does not begin at candidate move') + evidence = {'rootInfo': root, 'moveInfos': moves[:5], 'perspective': 'BLACK'} + if 'ownership' in result or request['includeOwnership']: + own = result.get('ownership') + expected = request['boardXSize'] * request['boardYSize'] + if not isinstance(own, list) or len(own) != expected or not all( + finite_number(v) and -1 <= v <= 1 for v in own): + raise ValueError('missing or invalid ownership') + evidence['ownership'] = own + return evidence + + +def anonymous_case(anonymous_id, requests, analyses, fallback): + positions = {} + for phase, request in requests.items(): + positions[phase] = {key: copy.deepcopy(request[key]) for key in + ('boardXSize', 'boardYSize', 'rules', 'komi', 'initialStones', 'moves')} + return {'id': anonymous_id, 'positions': positions, 'rules_fallback': fallback, + 'played_move': requests['post']['moves'][-1], 'analysis': analyses} + + +class Engine: + def __init__(self, binary, model, config, log_path): + self.log = open(log_path, 'w', encoding='utf-8') + try: + self.proc = subprocess.Popen([binary, 'analysis', '-model', model, '-config', config], + stdin=subprocess.PIPE, stdout=subprocess.PIPE, + stderr=self.log, text=True, bufsize=1) + except Exception: + self.log.close() + raise + self.responses = queue.Queue() + self.reader = threading.Thread(target=self._read, daemon=True) + self.reader.start() + + def _read(self): + for line in self.proc.stdout: + try: + self.responses.put(json.loads(line)) + except json.JSONDecodeError: + self.responses.put({'error': 'non-JSON engine output'}) + self.responses.put({'error': 'KataGo stdout closed'}) + + def query(self, request, timeout): + start = time.monotonic() + self.proc.stdin.write(json.dumps(request, allow_nan=False) + '\n') + self.proc.stdin.flush() + while True: + remaining = timeout - (time.monotonic() - start) + if remaining <= 0: + raise TimeoutError('analysis timeout') + try: + result = self.responses.get(timeout=remaining) + except queue.Empty as exc: + raise TimeoutError('analysis timeout') from exc + if result.get('error'): + raise RuntimeError(str(result['error'])) + if result.get('id') != request['id']: + raise RuntimeError('unexpected engine response id') + if result.get('isDuringSearch'): + continue + evidence = validate_result(result, request) + evidence['elapsed_seconds'] = time.monotonic() - start + return evidence + + def close(self): + self.proc.terminate() + try: + self.proc.wait(timeout=10) + except subprocess.TimeoutExpired: + self.proc.kill() + self.proc.wait() + self.reader.join(timeout=2) + self.proc.stdin.close() + self.proc.stdout.close() + self.log.close() + + +def write_json(path, value): + Path(path).write_text(json.dumps(value, indent=2, ensure_ascii=False, allow_nan=False), encoding='utf-8') + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + for name in ('data', 'cases', 'binary', 'model', 'config', 'out'): + parser.add_argument('--' + name, required=True) + parser.add_argument('--visits', type=int, default=128) + parser.add_argument('--timeout', type=float, default=120) + parser.add_argument('--ownership', action='store_true') + args = parser.parse_args() + if args.visits <= 0 or args.timeout <= 0: + parser.error('visits and timeout must be positive') + out = Path(args.out) + out.parent.mkdir(parents=True, exist_ok=True) + rows = {r['sample_id']: r for r in (json.loads(line) for line in + Path(args.data).read_text(encoding='utf-8').splitlines() if line.strip())} + cases = json.loads(Path(args.cases).read_text(encoding='utf-8'))['cases'] + version = subprocess.run([args.binary, 'version'], capture_output=True, text=True, timeout=30, check=True) + local = {'metadata': {'version': version.stdout.strip(), 'perspective': 'BLACK', + 'maxVisits': args.visits, 'binary_sha256': sha(args.binary), + 'model_sha256': sha(args.model), 'config_sha256': sha(args.config), + 'data_sha256': sha(args.data), 'cases_sha256': sha(args.cases), + 'case_order': 'input cases array order', 'limitation': LIMITATION}, 'cases': []} + anon = {'perspective': 'BLACK', 'maxVisits': args.visits, 'limitation': LIMITATION, 'cases': []} + anon_path = out.with_name(out.stem + '.anonymous.json') + start = time.monotonic() + engine = Engine(args.binary, args.model, args.config, str(out) + '.stderr.log') + failed = False + try: + for index, case in enumerate(cases, 1): + anonymous_id = f'position-{index:02d}' + record = {'local_case_id': case['id'], 'anonymous_id': anonymous_id} + local['cases'].append(record) + try: + row = rows[case['id']] + requests, fallback = build_requests(row, anonymous_id, args.visits, args.ownership) + if row['played_move'] != [case['playerColor'], case['playedMove']]: + raise ValueError('case played move disagrees with source history') + record.update(requests=requests, rules_source=row.get('rules'), rules_fallback=fallback) + record['analysis'] = {} + for phase, request in requests.items(): + record['analysis'][phase] = engine.query(request, args.timeout) + record['status'] = 'ok' + anon['cases'].append(anonymous_case(anonymous_id, requests, record['analysis'], fallback)) + except (KeyError, ValueError, RuntimeError, TimeoutError, OSError) as exc: + record.update(status='error', error=str(exc)) + failed = True + write_json(out, local) + write_json(anon_path, anon) + print(f'{index}/{len(cases)} {anonymous_id} {record["status"]}', flush=True) + if failed: + break + finally: + engine.close() + local['summary'] = {'requested': len(cases), 'succeeded': len(anon['cases']), + 'elapsed_seconds': time.monotonic() - start, + 'status': 'error' if failed else 'ok'} + write_json(out, local) + write_json(anon_path, anon) + print(json.dumps(local['summary']), flush=True) + return 1 if failed else 0 + + +if __name__ == '__main__': + raise SystemExit(main()) diff --git a/scripts/experiments/prepare_comment_concepts.py b/scripts/experiments/prepare_comment_concepts.py new file mode 100644 index 0000000..6d3c675 --- /dev/null +++ b/scripts/experiments/prepare_comment_concepts.py @@ -0,0 +1,178 @@ +"""Prepare noncommercial ACL2022 GTL comment-mention research data. + +No production weights/data are generated. SGF comments are weak labels, not +truth of concept presence. Only mainline move nodes are aligned after the move. +Run: python scripts/experiments/prepare_comment_concepts.py --games 1000 +Requires sgfmill (compatible with 1.1.0/1.1.1); caches original SGFs and provenance under .tmp/concept-data. +""" +import argparse +import concurrent.futures +import hashlib +import json +from pathlib import Path +import re +import urllib.request +from collections import Counter +from sgfmill import sgf, boards + +REPO = 'https://github.com/AndreHe02/go' +DEFAULT_REVISION = 'ad7c6bfab21fcd59650db28efe57df0d4531317d' +CONCEPTS = ['atari', 'ko', 'ladder', 'eye', 'cut', 'wall', 'sente', 'gote', + 'shape', 'influence', 'territory', 'joseki', 'tesuji', 'pincer', + 'attack', 'defend', 'invasion', 'life_death', 'connect', 'life', 'death', 'thick', 'weak', 'sacrifice'] +PATTERNS = { + 'attack': r'attack(?:s|ed|ing)?', 'defend': r'(?:defend(?:s|ed|ing)?|defen[cs]e)', + 'invasion': r'(?:invasions?|invad(?:e|es|ed|ing))', + 'life_death': r'(?:life|death|alive|dead|live|die|dying)', + 'eye': r'eyes?', 'cut': r'cut(?:s|ting)?', 'wall': r'walls?', + 'connect': r'connect(?:s|ed|ing|ion|ions)?', + 'thick': r'thick(?:ness)?', 'weak': r'weak(?:ness)?', +} + +def fetch(url): + with urllib.request.urlopen(url, timeout=45) as response: + return response.read() + +def gtp(point, size): + if point is None: + return 'pass' + row, col = point + return 'ABCDEFGHJKLMNOPQRSTUVXYZ'[col] + str(row + 1) + +def flat(board): + return [1 if board.get(row, col) == 'b' else -1 if board.get(row, col) == 'w' else 0 + for row in range(board.side) for col in range(board.side)] + +def canonical_game_key(game): + root = game.get_root() + black, white, empty = root.get_setup_stones() + initial = [['B', list(x)] for x in sorted(black)] + [['W', list(x)] for x in sorted(white)] + moves = [[color, point] for node in game.get_main_sequence() + for color, point in [node.get_move()] if color is not None] + # Excludes comments and metadata so reannotated copies of one game stay together. + payload = json.dumps([game.get_size(), initial, moves], separators=(',', ':')) + return hashlib.sha256(payload.encode()).hexdigest() + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument('--games', type=int, default=1000) + parser.add_argument('--revision', default=DEFAULT_REVISION, + help='Pinned full Git revision SHA for the annotation data') + parser.add_argument('--output', type=Path, default=Path('.tmp/concept-data')) + args = parser.parse_args() + out = args.output + out.mkdir(parents=True, exist_ok=True) + revision = args.revision.lower() + if not re.fullmatch(r'[0-9a-f]{40}', revision): + parser.error('--revision must be a full 40-character Git revision SHA') + tree_path = out / ('upstream-tree-' + revision + '.json') + if tree_path.exists(): + tree = json.loads(tree_path.read_text(encoding='utf-8')) + else: + # The old unversioned master cache is deliberately never reused. + tree = json.loads(fetch('https://api.github.com/repos/AndreHe02/go/git/trees/' + + revision + '?recursive=1')) + if tree.get('sha') != revision: + raise ValueError('Git tree SHA does not match requested revision: ' + + str(tree.get('sha')) + ' != ' + revision) + if tree.get('truncated'): + raise ValueError('Refusing a truncated Git tree listing') + if not tree_path.exists(): + tree_path.write_text(json.dumps(tree, indent=2), encoding='utf-8') + blob_hashes = {x['path']: x['sha'] for x in tree['tree'] if x['type'] == 'blob'} + paths = sorted((x['path'] for x in tree['tree'] if x['type'] == 'blob' and + x['path'].startswith('data/annotations/')), + key=lambda path: int(Path(path).stem))[:args.games] + base = 'https://raw.githubusercontent.com/AndreHe02/go/' + revision + '/' + def download(path): + dest = out / 'acl2022' / path + dest.parent.mkdir(parents=True, exist_ok=True) + def valid_blob(data): + header = ('blob ' + str(len(data)) + '\0').encode() + return hashlib.sha1(header + data).hexdigest() == blob_hashes[path] + if not dest.exists() or not valid_blob(dest.read_bytes()): + data = fetch(base + path) + if not valid_blob(data): + raise ValueError('Downloaded SGF blob does not match pinned revision: ' + path) + dest.write_bytes(data) + return dest + with concurrent.futures.ThreadPoolExecutor(max_workers=12) as executor: + files = list(executor.map(download, paths)) + stats = Counter() + positives = Counter() + split_games = {x: set() for x in ['train', 'val', 'test']} + errors = [] + seen = set() + with (out / 'comments.jsonl').open('w', encoding='utf-8') as writer: + for path in files: + digest = hashlib.sha256(path.read_bytes()).hexdigest() + if digest in seen: + stats['duplicate_games'] += 1 + continue + seen.add(digest) + try: + game = sgf.Sgf_game.from_bytes(path.read_bytes()) + source_game_key = canonical_game_key(game) + bucket = int(source_game_key[:8], 16) % 10 + split = 'test' if bucket == 0 else 'val' if bucket == 1 else 'train' + size = game.get_size() + if size != 19: + stats['non_19_games'] += 1 + continue + board = boards.Board(size) + root = game.get_root() + black, white, empty = root.get_setup_stones() + board.apply_setup(black, white, empty) + initial = [['B', gtp(x, size)] for x in sorted(black)] + [['W', gtp(x, size)] for x in sorted(white)] + moves = [] + for index, node in enumerate(game.get_main_sequence()): + if index and any(node.has_property(p) for p in ['AB', 'AW', 'AE']): + stats['midgame_setup_truncated'] += 1 + break + color, point = node.get_move() + if color is None: + if node.has_property('C'): + stats['nonmove_comments_excluded'] += 1 + continue + before = flat(board) + moves_before = list(moves) + if point is not None: + board.play(*point, color) + move = [color.upper(), gtp(point, size)] + moves.append(move) + if not node.has_property('C'): + continue + comment = node.get('C').strip() + if len(comment) < 15: + stats['short_comments_excluded'] += 1 + continue + labels = {word: int(bool(re.search(r'\b(?:' + PATTERNS.get(word, re.escape(word)) + r')\b', comment, re.I))) + for word in CONCEPTS} + positives.update({k: v for k, v in labels.items() if v}) + sample = dict(sample_id=path.stem + ':' + str(index), game_id=path.stem, + game_sha256=source_game_key, source_game_key=source_game_key, file_sha256=digest, split=split, board_size=size, + board_before=before, board_after=flat(board), initial_stones=initial, + moves_before=moves_before, moves_after=list(moves), played_move=move, + to_play_after='W' if color == 'b' else 'B', comment=comment, labels=labels, + alignment='comment_on_sgf_move_node_after_play', + komi=game.get_komi(), rules=root.get('RU') if root.has_property('RU') else None) + writer.write(json.dumps(sample, ensure_ascii=False) + '\n') + stats['samples'] += 1 + stats['samples_' + split] += 1 + split_games[split].add(source_game_key) + stats['parsed_19_games'] += 1 + except Exception as exc: + errors.append({'game': path.stem, 'error': type(exc).__name__ + ': ' + str(exc)}) + summary = dict(repo=REPO, commit=revision, requested_games=args.games, + selection='first numeric game ids', downloaded_games=len(files), counts=dict(stats), + split_games={k: len(v) for k, v in split_games.items()}, label_positives=dict(positives), + concepts=CONCEPTS, errors=errors, coordinate_system='row0 bottom, col0 left; index=row*19+col', + permission='GTL non-commercial digital redistribution; research only; do not redistribute production weights', + source_paper='https://aclanthology.org/2022.acl-short.90/', + label_warning='Mention labels only: hypotheticals, negation, alternative lines and omissions cause noise.') + (out / 'summary.json').write_text(json.dumps(summary, ensure_ascii=False, indent=2), encoding='utf-8') + print(json.dumps({k: summary[k] for k in ['downloaded_games', 'counts', 'split_games', 'label_positives']}, indent=2)) + +if __name__ == '__main__': + main() diff --git a/scripts/experiments/requirements-comment-concepts.txt b/scripts/experiments/requirements-comment-concepts.txt new file mode 100644 index 0000000..6d2ef26 --- /dev/null +++ b/scripts/experiments/requirements-comment-concepts.txt @@ -0,0 +1,7 @@ +# Python 3.11+; tested locally with CPU PyTorch, no CUDA requirement. +# Install torch with the platform-appropriate CPU wheel when no GPU training is desired. +# Standard-library-only KataGo collector requires no additional Python packages. +numpy==2.4.1 +torch==2.7.1 +scikit-learn==1.7.0 +sgfmill==1.1.0 diff --git a/scripts/experiments/train_comment_concepts.py b/scripts/experiments/train_comment_concepts.py new file mode 100644 index 0000000..a503d33 --- /dev/null +++ b/scripts/experiments/train_comment_concepts.py @@ -0,0 +1,171 @@ +"""Reproducible comment-keyword prediction; metrics are NOT Go semantic accuracy. + +Data and weights must stay local because upstream annotations have noncommercial +licensing. Never substitute rule-generated board labels for comment supervision. +""" +import argparse +import hashlib +import json +from pathlib import Path +import random +import numpy as np +import torch +from torch import nn +from sklearn.linear_model import LogisticRegression +from sklearn.metrics import f1_score, precision_score, recall_score, average_precision_score + + +def read_jsonl(path): + return [json.loads(s) for s in Path(path).read_text(encoding='utf-8').splitlines() if s.strip()] + + +def metrics(y, pred, labels, scores=None): + ap = [float(average_precision_score(y[:,i], scores[:,i])) if y[:,i].sum() else 0.0 + for i in range(len(labels))] if scores is not None else None + return {'macro_average_precision': float(np.mean(ap)) if ap is not None else None, + 'macro_f1': float(f1_score(y, pred, average='macro', zero_division=0)), + 'macro_precision': float(precision_score(y, pred, average='macro', zero_division=0)), + 'macro_recall': float(recall_score(y, pred, average='macro', zero_division=0)), + 'per_label': {label: {'prevalence': float(y[:, i].mean()), + 'f1': float(f1_score(y[:, i], pred[:, i], zero_division=0)), + 'precision': float(precision_score(y[:, i], pred[:, i], zero_division=0)), + 'recall': float(recall_score(y[:, i], pred[:, i], zero_division=0))} + for i, label in enumerate(labels)}} + + +def select_thresholds(y, probs, balanced): + if balanced: + grid = [0.01,0.03,0.05,0.1,0.2,0.3,0.5,0.7,0.9] + return np.array([max(grid,key=lambda t:f1_score(y[:,j],probs[:,j]>=t,zero_division=0)) + if y[:,j].sum() else 1.01 for j in range(y.shape[1])]) + grid = [0.1,0.2,0.3,0.5,0.7] + return max(grid,key=lambda t:f1_score(y,probs>=t,average='macro',zero_division=0)) + + +def threshold_json(t): return t.tolist() if isinstance(t,np.ndarray) else t + + +def planes(row, feature): + b = np.asarray(row['board_after']).reshape(19, 19) + last = np.zeros((19, 19), dtype=np.float32) + move = row['played_move'] + if isinstance(move, list): + color, move = move + else: + color = row['moves_after'][-1][0] + if move.lower() != 'pass': + last[int(move[1:])-1, 'ABCDEFGHJKLMNOPQRST'.index(move[0].upper())] = 1 + result = [(b == 1).astype('float32'), (b == -1).astype('float32'), last, + np.full((19,19), 1 if color == 'B' else -1, dtype='float32')] + if feature is not None: + result.extend([np.asarray(feature['ownership'], dtype='float32').reshape(19,19)[::-1].copy(), + np.asarray(feature['policy'][:361], dtype='float32').reshape(19,19)[::-1].copy()]) + return np.stack(result) + + +class TinyCNN(nn.Module): + def __init__(self, channels, labels): + super().__init__() + self.net = nn.Sequential(nn.Conv2d(channels, 8, 3, padding=1), nn.ReLU(), + nn.Conv2d(8, 8, 3, padding=1), nn.ReLU(), + nn.AdaptiveAvgPool2d((4,4)), nn.Flatten(), nn.Linear(128, labels)) + def forward(self, x): + return self.net(x) + + +def main(): + p = argparse.ArgumentParser() + p.add_argument('--data', required=True) + p.add_argument('--features') + p.add_argument('--out', required=True) + p.add_argument('--max-samples', type=int, default=300) + p.add_argument('--seed', type=int, default=20261002) + p.add_argument('--epochs', type=int, default=50) + p.add_argument('--balanced', action='store_true', help='Exploratory train-only weights and per-label validation thresholds') + args = p.parse_args() + random.seed(args.seed); np.random.seed(args.seed); torch.manual_seed(args.seed) + torch.set_num_threads(2); torch.use_deterministic_algorithms(True) + rows = read_jsonl(args.data) + rows = sorted(rows, key=lambda r: hashlib.sha256((str(args.seed)+r['sample_id']).encode()).hexdigest())[:args.max_samples] + feature_map = {r['sample_id']: r for r in read_jsonl(args.features) if r.get('status') == 'ok'} if args.features else {} + # Compare only identical samples when features are requested; failed queries are excluded and counted. + selected = rows + if args.features: + rows = [r for r in rows if r['sample_id'] in feature_map] + labels = sorted(rows[0]['labels']) + y = np.array([[int(r['labels'][k]) for k in labels] for r in rows], dtype='float32') + splits = {s: np.array([i for i,r in enumerate(rows) if r['split']==s], dtype=int) for s in ('train','val','test')} + groups = {s: {rows[i]['game_sha256'] for i in idx} for s,idx in splits.items()} + assert not (groups['train'] & groups['val'] or groups['train'] & groups['test'] or groups['val'] & groups['test']), 'game leakage' + assert all(len(v) for v in splits.values()), 'each split must have samples' + report = {'seed': args.seed, 'target': 'human comment keyword weak labels; NOT Go concept correctness', + 'balanced': args.balanced, 'evaluation_status': 'exploratory: test already observed in earlier protocol' if args.balanced else 'initial fixed protocol', + 'data_sha256': hashlib.sha256(Path(args.data).read_bytes()).hexdigest(), + 'labels': labels, 'selected_samples': len(selected), 'feature_failures_excluded': len(selected)-len(rows), + 'split_counts': {s: len(idx) for s,idx in splits.items()}, + 'game_counts': {s: len(g) for s,g in groups.items()}, + 'split_label_positives': {s: dict(zip(labels, y[idx].sum(0).astype(int).tolist())) for s,idx in splits.items()}, + 'validation_labels_without_positives': [labels[j] for j in range(len(labels)) if y[splits['val'],j].sum()==0], + 'runtime': {'torch': torch.__version__, 'numpy': np.__version__}, + 'sample_ids': {s: [rows[i]['sample_id'] for i in idx] for s,idx in splits.items()}, + 'train_prevalence': dict(zip(labels, y[splits['train']].mean(0).tolist())), 'models': {}} + if args.features: + metadata_path = Path(args.features+'.meta.json') + feature_metadata = json.loads(metadata_path.read_text(encoding='utf-8')) + assert feature_metadata['data_sha256'] == report['data_sha256'], 'features generated from different dataset' + report['feature_metadata'] = feature_metadata + tr, va, te = (splits[s] for s in ('train','val','test')) + majority = (y[tr].mean(0)>=0.5).astype(int) + freq = np.tile(y[tr].mean(0),(len(te),1)) + report['models']['majority'] = {'test': metrics(y[te], np.tile(majority,(len(te),1)), labels,freq)} + report['models']['frequency_dummy'] = {'test':metrics(y[te],freq>=0.5,labels,freq)} + report['models']['always_positive'] = {'test': metrics(y[te], np.ones_like(y[te]), labels,np.ones_like(y[te]))} + out = Path(args.out); out.mkdir(parents=True, exist_ok=True) + for mode in (['board', 'board_katago'] if args.features else ['board']): + x = np.stack([planes(r, feature_map[r['sample_id']] if mode=='board_katago' else None) for r in rows]) + flat = x.reshape(len(x), -1) + linear = [] + for j in range(len(labels)): + if len(np.unique(y[tr,j])) < 2: + linear.append(np.full(len(x), y[tr,j][0])) + else: + clf = LogisticRegression(C=0.01, max_iter=500, random_state=args.seed,class_weight='balanced' if args.balanced else None) + clf.fit(flat[tr], y[tr,j]); linear.append(clf.predict_proba(flat)[:,1]) + probs = np.stack(linear,1) + threshold = select_thresholds(y[va],probs[va],args.balanced) + report['models'][mode+'_linear'] = {'threshold_selected_on_val': threshold_json(threshold), + 'val': metrics(y[va],probs[va]>=threshold,labels,probs[va]), 'test': metrics(y[te],probs[te]>=threshold,labels,probs[te])} + np.savez_compressed(out/(mode+'_linear.test.local.npz'), truth=y[te], scores=probs[te], + prediction=probs[te]>=threshold, sample_ids=[rows[i]['sample_id'] for i in te], labels=labels) + torch.manual_seed(args.seed) + model = TinyCNN(x.shape[1],len(labels)); opt = torch.optim.Adam(model.parameters(),lr=0.003) + xt = torch.from_numpy(x); yt = torch.from_numpy(y) + positives = y[tr].sum(0) + weights = np.minimum((len(tr)-positives)/np.maximum(positives,1),20).astype('float32') + loss_fn = nn.BCEWithLogitsLoss(pos_weight=torch.from_numpy(weights) if args.balanced else None) + best_score=-1; best_state=None; best_threshold=None; best_epoch=None + for epoch in range(args.epochs): + model.train() + for batch in torch.randperm(len(tr)).split(32): + ix = tr[batch.numpy()]; opt.zero_grad(); loss=loss_fn(model(xt[ix]),yt[ix]); loss.backward(); opt.step() + model.eval() + with torch.no_grad(): vp = model(xt[va]).sigmoid().numpy() + t = select_thresholds(y[va],vp,args.balanced) + score = f1_score(y[va],vp>=t,average='macro',zero_division=0) + if score>best_score: + best_score=score; best_state={k:v.detach().clone() for k,v in model.state_dict().items()}; best_threshold=t; best_epoch=epoch+1 + model.load_state_dict(best_state); model.eval() + # Test is evaluated exactly once after all model/threshold selection on validation. + with torch.no_grad(): vp=model(xt[va]).sigmoid().numpy(); tp=model(xt[te]).sigmoid().numpy() + report['models'][mode+'_cnn'] = {'epoch_selected_on_val':best_epoch,'threshold_selected_on_val':threshold_json(best_threshold), + 'train_pos_weights':dict(zip(labels,weights.tolist())) if args.balanced else None, + 'parameters':sum(v.numel() for v in model.parameters()), 'val':metrics(y[va],vp>=best_threshold,labels,vp), + 'test':metrics(y[te],tp>=best_threshold,labels,tp)} + torch.save(best_state,out/(mode+'_cnn.local.pt')) + np.savez_compressed(out/(mode+'_cnn.test.local.npz'), truth=y[te], scores=tp, + prediction=tp>=best_threshold, sample_ids=[rows[i]['sample_id'] for i in te], labels=labels) + (out/'report.json').write_text(json.dumps(report,ensure_ascii=False,indent=2),encoding='utf-8') + print(json.dumps({k: v['test']['macro_f1'] for k,v in report['models'].items()})) + + +if __name__ == '__main__': main() diff --git a/scripts/lib/commentary_blind_review_checks.mjs b/scripts/lib/commentary_blind_review_checks.mjs new file mode 100644 index 0000000..5b07d72 --- /dev/null +++ b/scripts/lib/commentary_blind_review_checks.mjs @@ -0,0 +1,63 @@ +// Synthetic in-memory checks only: this does not create commentary results or review output. +import assert from 'node:assert/strict' +import { createHash } from 'node:crypto' +import { fixtureBoards, validatePair, reviewHtml, runStatus } from '../build_commentary_blind_review.mjs' + +const hash = value => createHash('sha256').update(value).digest('hex') +const sgf = '(;SZ[5]KM[0]AB[ab];B[cc];W[bc];B[ae];W[cd];B[ee];W[dc];B[aa];W[cb])' +const fixture = { id: 'position-1', sha256: hash(sgf), boardSize: 5, moveNumber: 8 } +const source = { moveNumber: 8, playerColor: 'W', playedMove: 'C4', boardSnapshot: [ + ['A4', 'B'], ['C3', 'B'], ['B3', 'W'], ['A1', 'B'], ['C2', 'W'], ['E1', 'B'], ['D3', 'W'], ['A5', 'B'] +].map(([point, color]) => ({ point, color })) } +const boards = fixtureBoards(fixture, source, sgf + '\n') +assert.ok(boards.before.some(stone => stone.point === 'C3' && stone.color === 'B')) +assert.ok(!boards.after.some(stone => stone.point === 'C3')) +assert.ok(boards.after.some(stone => stone.point === 'C4' && stone.color === 'W')) +assert.ok(boards.after.some(stone => stone.point === 'A4' && stone.color === 'B')) +assert.deepEqual(boards.recentMoves.at(-1), { moveNumber: 7, color: 'B', point: 'A5' }) +assert.deepEqual(boards.recentMoves[0], { moveNumber: 1, color: 'B', point: 'C3' }) +assert.throws(() => fixtureBoards(fixture, { ...source, boardSnapshot: [] }, sgf), /snapshot/) +const passSgf = sgf.slice(0, -1) + ';B[])' +assert.deepEqual(fixtureBoards({ ...fixture, moveNumber: 9, sha256: hash(passSgf) }, { ...source, moveNumber: 9, playerColor: 'B', playedMove: 'pass', boardSnapshot: boards.after }, passSgf).after, boards.after) + +const request = { mode: 'current-move', gameId: 'anonymous-hash', moveNumber: 8, prompt: '同一个完整提示', coachLevel: 'intermediate', studentAgeRange: 'adult', teacherStyle: 'rigorous', toolPolicy: 'auto' } +const applicationSettings = { reviewLanguage: 'zh-CN' }; const requestSettings = { coachLevel: 'intermediate', prefetchedMaxVisits: 96 } +function result(variant) { + const identity = { model: 'gpt-6-luna', provider: 'codex-app-server', variant, phase: 'formal', fixtureHash: fixture.sha256, bundleHash: hash(variant), runtimeHash: hash('same-runtime'), engineConfigHash: hash('same-engine'), promptHash: hash(request.prompt), settingsHash: hash(JSON.stringify({ app: applicationSettings, request: requestSettings })) } + return { id: fixture.id, success: true, ...identity, identity, profileHash: hash('same-semantic-profile'), analysis: { gameId: request.gameId, boardSize: 5, moveNumber: 8 }, boardVerification: { verified: true, before: Array(25).fill(0), after: Array(25).fill(0) }, applicationSettings, requestSettings, inferenceSettings: { effort: 'provider-default' }, request: { ...request, runId: variant }, result: { markdown: '完整原文\n<script>alert("escape")</script>\n末尾仍保留' } } +} +const candidate = result('candidate'); const baseline = result('baseline') +validatePair(fixture, candidate, baseline) +for (const change of [ + result => { result.success = false }, + result => { result.model = 'commentary-eval-mock'; result.identity.model = result.model }, + result => { result.provider = 'openai-compatible'; result.identity.provider = result.provider }, + result => { result.runtimeHash = hash('other'); result.identity.runtimeHash = result.runtimeHash }, + result => { result.identity.fixtureHash = hash('other') }, + result => { result.request.prompt = 'changed' }, + result => { result.requestSettings.coachLevel = 'beginner' }, + result => { result.identity.settingsHash = undefined } + ,result => { result.phase = 'diagnostic'; result.identity.phase = 'diagnostic' } + ,result => { delete result.phase } + ,result => { result.profileHash = hash('changed-profile') } + ,result => { delete result.profileHash } + ,result => { result.analysis.gameId = 'another-game' } + ,result => { result.boardVerification.before[0] = 1 } +]) { const changed = structuredClone(baseline); change(changed); assert.throws(() => validatePair(fixture, candidate, changed)) } +const cases = ['A', 'B'].map(letter => ({ id: `position-1:${letter}`, ...boards, markdown: candidate.result.markdown })) +const html = reviewHtml({ model: 'gpt-6-luna', reviewSetId: 'synthetic-check', cases }) +assert.ok(html.includes('<script>alert("escape")</script>')) +assert.ok(html.includes('末尾仍保留')) +assert.ok(!html.includes('<script>alert(')) +assert.ok(html.includes('data-board="before"') && html.includes('data-board="after"')) +assert.ok(html.includes('if(Object.hasOwn(saved,id)||!byId.has(id))return')) +assert.ok(html.includes('/api/feedback') && html.includes('localStorage.setItem') && html.includes('application/json')) +assert.ok(!/localCaseId|candidate|baseline|weakMentions/.test(html)) +assert.ok(html.includes('预定 1 局') && html.includes('最近手顺') && html.includes('第 7 手 黑 A5')) +const interruption = { ...baseline, success: false, error: 'Recovery incomplete: evaluator interruption' } +assert.match(runStatus(interruption).text, /讲解已返回.*评测中断/) +assert.equal(runStatus({ ...baseline, phase: 'diagnostic' }).attempted, false) +const statusOnly = reviewHtml({ model: 'gpt-6-luna', reviewSetId: 'synthetic-check', cases: [], positions: [{ id: 'position-11', state: 'unpaired', statuses: ['整链验收已通过', runStatus(interruption).text] }], summary: { plannedPositions: 10, completePairs: 0, unpairedPositions: 10, expectedRuns: 20, passedRuns: 9, unpassedRuns: 1, pendingRuns: 10 } }) +assert.ok(statusOnly.includes('预定 10 局') && statusOnly.includes('流水线尚未完成') && statusOnly.includes('position-11')) +assert.ok(!/localCaseId|candidate|baseline|weakMentions/.test(statusOnly)) +console.log('Blind review checks passed: real-pair constraints, mismatch rejection, SGF coordinates/setup/capture/pass, full escaped text, local feedback recovery.') diff --git a/scripts/lib/commentary_eval.mjs b/scripts/lib/commentary_eval.mjs new file mode 100644 index 0000000..38a236a --- /dev/null +++ b/scripts/lib/commentary_eval.mjs @@ -0,0 +1,297 @@ +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import { createHash, randomInt } from 'node:crypto' +import { resolve } from 'node:path' + +export function readRecords(path) { + const text = readFileSync(path, 'utf8').trim() + if (!text) return [] + if (path.endsWith('.jsonl')) return text.split(/\r?\n/).filter(Boolean).map(line => JSON.parse(line)) + const value = JSON.parse(text) + return Array.isArray(value) ? value : value.cases ?? value.positions ?? value.evidence ?? value.records ?? [] +} + +export function gtpToSgf(point, size) { + if (/^pass$/i.test(String(point)) || point === '') return '' + const match = /^([A-HJ-Z])(\d+)$/i.exec(String(point)) + assert.ok(match, `Invalid GTP point: ${point}`) + const col = 'ABCDEFGHJKLMNOPQRSTUVWXYZ'.indexOf(match[1].toUpperCase()) + const row = size - Number(match[2]) + assert.ok(col >= 0 && col < size && row >= 0 && row < size, 'Point outside board') + return String.fromCharCode(97 + col, 97 + row) +} + +export function anonymousSgf(record, engine = {}) { + const size = Number(record.board_size) + assert.ok(Number.isInteger(size) && size >= 2 && size <= 25, 'Invalid board size') + const komi = Number(engine.komi_used ?? record.komi ?? 7.5) + assert.ok(Number.isFinite(komi) && Math.abs(komi) <= 100, 'Invalid komi') + const rawRules = engine.rules_used ?? record.rules + const rules = rawRules == null ? null : String(rawRules).trim() + assert.ok(!rules || /^[a-zA-Z0-9 _-]{1,60}$/.test(rules), 'Unsupported rules string') + const moves = [...record.moves_before, record.played_move] + const render = ([color, point]) => { + assert.ok(color === 'B' || color === 'W', 'Invalid stone color') + return `${color}[${gtpToSgf(point, size)}]` + } + const setups = ['B', 'W'].map(color => { + const points = (record.initial_stones ?? []).filter(stone => stone[0] === color).map(stone => gtpToSgf(stone[1], size)) + assert.ok(points.every(Boolean), 'Initial stones cannot pass') + return points.length ? `A${color}${points.map(point => `[${point}]`).join('')}` : '' + }).join('') + const sgf = `(;SZ[${size}]KM[${komi}]${rules ? `RU[${rules}]` : ''}${setups}${moves.map(move => `;${render(move)}`).join('')})` + assertAnonymousSgf(sgf) + return { sgf, moveNumber: moves.length, boardSize: size, komi, rules, rulesOrigin: engine.rules_used != null ? 'engine' : record.rules != null ? 'record' : 'absent' } +} + +export function assertAnonymousSgf(sgf) { + const properties = [...sgf.matchAll(/([A-Z]+)\[/g)].map(match => match[1]) + assert.ok(properties.every(key => ['SZ', 'KM', 'RU', 'AB', 'AW', 'B', 'W'].includes(key)), 'SGF contains prohibited metadata') + assert.ok(!/\b(?:GTL|feedback|weak.?labels?|source|comment)\b/i.test(sgf), 'SGF contains prohibited text') +} + +export function prepareCases(cases, records, engines, directory) { + const dataById = new Map(records.map(item => [item.sample_id ?? item.id, item])) + const engineById = new Map(engines.map(item => [item.sample_id ?? item.local_case_id ?? item.localCaseId ?? item.id, item])) + const shuffled = [...cases] + for (let index = shuffled.length - 1; index > 0; index--) { + const other = randomInt(index + 1) + ;[shuffled[index], shuffled[other]] = [shuffled[other], shuffled[index]] + } + return shuffled.map((item, index) => { + const localCaseId = item.id ?? item.sample_id + const record = dataById.get(localCaseId) + assert.ok(record, `Missing local record: ${localCaseId}`) + // Anonymous engine evidence has an independent shuffled ID namespace. Match full history, never array order or position-N. + const matchingEngine = engineById.get(localCaseId) ?? engines.find(engine => { + const pre = engine.positions?.pre + return pre && pre.boardXSize === record.board_size && JSON.stringify(pre.moves) === JSON.stringify(record.moves_before) && JSON.stringify(pre.initialStones ?? []) === JSON.stringify(record.initial_stones ?? []) + }) + assert.ok(!engines.length || matchingEngine, 'No engine evidence matches this complete source history') + const pre = matchingEngine?.positions?.pre ?? matchingEngine?.requests?.pre + if (pre) { + assert.equal(pre.boardXSize, record.board_size, 'Engine evidence board size mismatch') + assert.deepEqual(pre.moves, record.moves_before, 'Engine evidence before history mismatch') + assert.deepEqual(pre.initialStones ?? [], record.initial_stones ?? [], 'Engine evidence setup mismatch') + const post = matchingEngine?.positions?.post ?? matchingEngine?.requests?.post + if (post) assert.deepEqual(post.moves, [...record.moves_before, record.played_move], 'Engine evidence played move mismatch') + } + const fixture = anonymousSgf(record, pre ? { rules_used: pre.rules, komi_used: pre.komi } : matchingEngine) + assert.equal(fixture.moveNumber, item.moveNumber ?? fixture.moveNumber, 'Case/history move number mismatch') + const id = `position-${index + 1}` + return { id, localCaseId, filePath: resolve(directory, `${id}.sgf`), ...fixture, sha256: createHash('sha256').update(fixture.sgf).digest('hex') } + }) +} + +export const teacherPrompt = '请讲解当前这一手：先说明落点和局部棋形事实，再解释它与周围棋子的关系、目的、风险和可验证的后续。结合实际棋盘截图和 KataGo 数据；证据不足时明确保留判断。用自然中文讲给中级成年棋友，避免堆术语。本轮只分析当前手，请不要更新学生档案或训练配置。' + +export function teacherRequest(fixture, gameId, variant, promptOverride = teacherPrompt) { + return { runId: `commentary-eval-${variant}-${fixture.id}`, mode: 'current-move', toolPolicy: 'auto', prompt: promptOverride, gameId, moveNumber: fixture.moveNumber, coachLevel: 'intermediate', studentAgeRange: 'adult', teacherStyle: 'rigorous' } +} + +export function assertSafeBoundary(value, localCases, records) { + const strings = [] + function visit(item, depth = 0) { + assert.ok(depth < 40, 'Boundary nesting is too deep') + if (typeof item === 'string') { + strings.push(item) + try { const parsed = JSON.parse(item); if (typeof parsed === 'object' && parsed !== null) visit(parsed, depth + 1) } catch (error) { if (error?.code === 'ERR_ASSERTION') throw error } + } else if (Array.isArray(item)) item.forEach(child => visit(child, depth + 1)) + else if (item && typeof item === 'object') for (const [key, child] of Object.entries(item)) { + if (['localCaseId', 'sample_id', 'comment', 'labels', 'feedback', 'source_game_key', 'weakLabels'].includes(key)) assert.ok(child == null || child === '' || (Array.isArray(child) && !child.length), 'Prohibited populated field at external boundary') + visit(child, depth + 1) + } + } + visit(value) + for (const item of localCases) assert.ok(!strings.some(text => text.includes(item.localCaseId)), 'Original case ID leaked') + for (const record of records) { + if (typeof record.comment === 'string' && record.comment.trim()) { + const comment = record.comment.trim() + const escaped = JSON.stringify(comment).slice(1, -1) + assert.ok(!strings.some(text => text === comment || (comment.length > 12 && (text.includes(comment) || text.includes(escaped)))), 'Original commentary leaked') + } + for (const key of ['source_game_key', 'game_sha256', 'file_sha256']) { + if (typeof record[key] === 'string' && record[key].length > 6) assert.ok(!strings.some(text => text.includes(record[key])), 'Source identifier leaked') + } + } +} + +export function validateFreshPrefetch(analysis) { + assert.equal(analysis?.runtimeEvidence?.cacheStatus, 'written', 'Prefetched analysis must be freshly computed') + assert.match(analysis.runtimeEvidence.cacheReason ?? '', /cache lookup bypassed/, 'Prefetched analysis must explicitly bypass derived-analysis cache lookup') +} + +export function validatePipelineEvidence(payload, fixture) { + const analysis = payload.analysis + assert.equal(analysis?.moveNumber, fixture.moveNumber, 'Analysis move number mismatch') + assert.equal(analysis?.boardSize, fixture.boardSize, 'Analysis board size mismatch') + assert.ok(Number.isFinite(analysis?.before?.winrate) && Number.isFinite(analysis?.before?.scoreLead), 'Invalid KataGo root evidence') + assert.ok(analysis.before.topMoves?.length > 0, 'Missing KataGo candidates') + assert.ok(payload.result?.markdown?.trim(), 'Teacher returned no markdown') + const vision = payload.result.visionEvidence + assert.ok(vision?.attached && vision.imageCount > 0 && vision.images?.some(image => image.valid), 'Missing valid teacher image evidence') + assert.ok(vision.images.some(image => image.valid && image.moveNumber === fixture.moveNumber && image.role === 'current-board'), 'Teacher image does not represent the requested move') + const logs = payload.result.toolLogs ?? [] + assert.ok(!logs.some(log => log.status === 'done' && ['studentProfile.write', 'settings.update', 'settings.write', 'configuration.write'].includes(log.name)), 'Training/profile configuration write invalidates paired evaluation') + for (const name of ['katago.analyzePosition', 'board.captureTeachingImage']) assert.ok(logs.some(log => log.name === name && log.status === 'done'), `Missing successful ${name}`) + assert.ok(logs.some(log => ['knowledge.matchPosition', 'knowledge.searchLocal'].includes(log.name) && log.status === 'done'), 'Missing successful knowledge evidence') +} + +export function validateMockCapture(requests, expected) { + const messages = requests.flatMap(item => item.body.messages ?? []) + for (const name of ['sgf_readGameRecord', 'knowledge_matchPosition', 'katago_analyzePosition', 'board_captureTeachingImage']) { + const matching = messages.filter(message => message.role === 'tool' && message.tool_call_id === `eval_${name}`) + assert.ok(matching.some(message => { + try { return JSON.parse(message.content).ok === true } catch { + // Production compactToolResult retains the envelope prefix and truncates large payload tails. + return /^\s*\{\s*"ok"\s*:\s*true\s*[,}]/.test(message.content) && /\[tool result truncated: \d+ chars omitted\]$/.test(message.content.trim()) + } + }), `No successful production ${name} result captured`) + } + if (expected) { + const board = messages.find(message => message.role === 'tool' && message.tool_call_id === 'eval_board_captureTeachingImage') + const images = JSON.parse(board.content).result.images + assert.ok(images.some(image => image.gameId === expected.gameId && image.moveNumber === expected.moveNumber), 'Captured board tool image does not match the requested game and move') + if (expected.requireConceptEvidence) { + const contents = messages.filter(message => message.role === 'tool' && message.tool_call_id === 'eval_knowledge_matchPosition').map(message => message.content) + const evidence = contents.map(content => extractCompactJsonField(content, 'moveConceptEvidence')).find(Boolean) + assert.equal(evidence?.moveNumber, expected.moveNumber, 'Complete concept evidence did not enter the provider for the requested move') + assert.equal(evidence.boardTiming, 'before-move') + assert.ok(evidence.prompt?.trim() && evidence.entries?.some(entry => entry.role === 'played'), 'Missing complete played-move concept evidence') + } + } + assert.ok(messages.some(message => Array.isArray(message.content) && message.content.some(part => part.type === 'image_url' && /^data:image\//.test(part.image_url?.url ?? ''))), 'No actual image_url entered the provider messages') +} + +export function extractCompactJsonField(content, name) { + const match = new RegExp(`"${name}"\\s*:\\s*\\{`).exec(content) + if (!match) return null + const start = content.indexOf('{', match.index) + let depth = 0, quoted = false, escaped = false + for (let index = start; index < content.length; index++) { + const char = content[index] + if (quoted) { + if (escaped) escaped = false + else if (char === '\\') escaped = true + else if (char === '"') quoted = false + } else if (char === '"') quoted = true + else if (char === '{') depth++ + else if (char === '}' && --depth === 0) { + try { return JSON.parse(content.slice(start, index + 1)) } catch { return null } + } + } + return null +} + +export function parseStoredResult(text) { + try { return JSON.parse(text) } catch { return { success: false, invalidArtifact: true } } +} + +export function semanticProfile(profile) { + if (!profile) return null + const { createdAt, updatedAt, ...stable } = profile + return stable +} + +export function canResume(previous, identity) { + return previous?.success === true && JSON.stringify(previous.identity) === JSON.stringify(identity) +} + +export function replayResearchBoard(size, initialStones, moves) { + // Research snapshots are flattened bottom row first. GTP row 1 maps to offset 0. + const board = Array(size * size).fill(0) + function indexFor(point) { + if (point === '' || /^pass$/i.test(point)) return null + gtpToSgf(point, size) + return (Number(point.slice(1)) - 1) * size + 'ABCDEFGHJKLMNOPQRSTUVWXYZ'.indexOf(point[0].toUpperCase()) + } + const adjacent = index => { + const x = index % size, y = Math.floor(index / size) + return [x > 0 ? index - 1 : -1, x + 1 < size ? index + 1 : -1, y > 0 ? index - size : -1, y + 1 < size ? index + size : -1].filter(index => index >= 0) + } + function group(start) { + const stones = new Set([start]), liberties = new Set(), todo = [start] + while (todo.length) for (const next of adjacent(todo.pop())) { + if (board[next] === 0) liberties.add(next) + else if (board[next] === board[start] && !stones.has(next)) { stones.add(next); todo.push(next) } + } + return { stones, liberties } + } + for (const [color, point] of initialStones) { + const index = indexFor(point) + assert.notEqual(index, null, 'Setup stone cannot pass') + assert.equal(board[index], 0, 'Duplicate setup point') + board[index] = color === 'B' ? 1 : -1 + } + for (const [color, point] of moves) { + const index = indexFor(point) + if (index === null) continue + const stone = color === 'B' ? 1 : -1 + assert.equal(board[index], 0, 'History plays on an occupied point') + board[index] = stone + for (const next of adjacent(index)) if (board[next] === -stone) { + const opponent = group(next) + if (!opponent.liberties.size) for (const captured of opponent.stones) board[captured] = 0 + } + assert.ok(group(index).liberties.size > 0, 'Suicide history is outside this Japanese-rule evaluation') + } + return board +} + +export function validateImportedRecord(imported, source) { + assert.equal(imported.boardSize, source.board_size) + const expected = [...source.moves_before, source.played_move] + assert.deepEqual(imported.moves.map(move => [move.color, move.gtp]), expected, 'Imported full move history differs from source') + for (const move of imported.moves) { + const coordinate = gtpToSgf(move.gtp, imported.boardSize) + assert.equal(move.row, coordinate ? coordinate.charCodeAt(1) - 97 : null, 'Imported move row mismatch') + assert.equal(move.col, coordinate ? coordinate.charCodeAt(0) - 97 : null, 'Imported move column mismatch') + } + const setup = (imported.initialStones ?? []).map(stone => [stone.color, stone.point]) + assert.deepEqual([...setup].sort(), [...source.initial_stones].sort(), 'Imported AB/AW differ from source') + for (const stone of imported.initialStones ?? []) { + const coordinate = gtpToSgf(stone.point, imported.boardSize) + assert.equal(stone.row, coordinate.charCodeAt(1) - 97) + assert.equal(stone.col, coordinate.charCodeAt(0) - 97) + } + const before = replayResearchBoard(imported.boardSize, setup, expected.slice(0, -1)) + const after = replayResearchBoard(imported.boardSize, setup, expected) + assert.deepEqual(before, source.board_before, 'Replayed before board differs from research snapshot') + assert.deepEqual(after, source.board_after, 'Replayed after board differs from research snapshot') + return { before, after, verified: true } +} + +export async function connectCdp(port, timeoutMs = 600_000) { + const response = await fetch(`http://127.0.0.1:${port}/json/list`) + assert.ok(response.ok, 'Cannot list CDP pages') + const pages = await response.json() + const page = pages.find(item => item.type === 'page' && item.webSocketDebuggerUrl && !/devtools/i.test(item.url ?? '')) + assert.ok(page, 'No application renderer CDP page') + const socket = new WebSocket(page.webSocketDebuggerUrl) + const pending = new Map() + let id = 0 + await new Promise((resolveOpen, reject) => { socket.onopen = resolveOpen; socket.onerror = () => reject(new Error('CDP connection failed')) }) + socket.onmessage = event => { + const message = JSON.parse(event.data) + const item = pending.get(message.id) + if (!item) return + pending.delete(message.id) + clearTimeout(item.timer) + if (message.error) item.reject(new Error(message.error.message)) + else item.resolve(message.result) + } + socket.onclose = () => { for (const item of pending.values()) { clearTimeout(item.timer); item.reject(new Error('CDP closed')) }; pending.clear() } + return { + async evaluate(expression) { + const result = await new Promise((resolveValue, reject) => { + const requestId = ++id + const timer = setTimeout(() => { pending.delete(requestId); reject(new Error('CDP evaluation timed out')) }, timeoutMs) + pending.set(requestId, { resolve: resolveValue, reject, timer }) + socket.send(JSON.stringify({ id: requestId, method: 'Runtime.evaluate', params: { expression, awaitPromise: true, returnByValue: true } })) + }) + if (result.exceptionDetails) throw new Error(result.exceptionDetails.exception?.description ?? result.exceptionDetails.text) + return result.result?.value + }, + close() { socket.close() } + } +} diff --git a/scripts/lib/concept_modules.mjs b/scripts/lib/concept_modules.mjs new file mode 100644 index 0000000..e878734 --- /dev/null +++ b/scripts/lib/concept_modules.mjs @@ -0,0 +1,31 @@ +import { readFileSync } from 'node:fs' +import { fileURLToPath } from 'node:url' +import { execFileSync } from 'node:child_process' +import ts from 'typescript' + +// Execute the real recognizers in Node, with only Electron's app path lookup stubbed. +const urls = new Map() +export function conceptModuleUrl(relative, revision) { + if (revision && !/^[a-f0-9]{7,40}$/.test(revision)) throw new Error('Expected a pinned commit hash') + const key = `${revision ?? 'working'}:${relative}` + if (urls.has(key)) return urls.get(key) + const path = new URL(`../../${relative}`, import.meta.url) + const input = revision ? execFileSync('git', ['show', `${revision}:${relative}`], { cwd: fileURLToPath(new URL('../../', import.meta.url)), encoding: 'utf8' }) : readFileSync(path, 'utf8') + let source = ts.transpileModule(input, { + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ES2022 } + }).outputText + source = source.replace(/from ['"]([^'"]+)['"]/g, (match, spec) => { + if (spec === 'electron') return `from 'data:text/javascript,export const app={isPackaged:false}'` + if (spec.startsWith('node:')) return match + if (!spec.startsWith('.')) throw new Error(`Unexpected runtime dependency: ${spec}`) + const dependency = new URL(`${spec}.ts`, path) + const root = new URL('../../', import.meta.url) + const relativeDependency = fileURLToPath(dependency).slice(fileURLToPath(root).length).replaceAll('\\', '/') + return `from '${conceptModuleUrl(relativeDependency, revision)}'` + }) + const url = `data:text/javascript;base64,${Buffer.from(source).toString('base64')}` + urls.set(key, url) + return url +} + +export async function loadConceptModule(relative, revision) { return import(conceptModuleUrl(relative, revision)) } diff --git a/scripts/serve_concept_review.py b/scripts/serve_concept_review.py new file mode 100644 index 0000000..9fc6ea5 --- /dev/null +++ b/scripts/serve_concept_review.py @@ -0,0 +1,176 @@ +#!/usr/bin/env python3 +"""Serve the local concept review and save validated feedback beside it.""" +import argparse +import json +import os +from pathlib import Path +import tempfile +import threading +from http.server import SimpleHTTPRequestHandler, ThreadingHTTPServer +from urllib.parse import urlsplit + +MAX_BODY = 256 * 1024 +FIELDS = {"id": 128, "geometryCorrect": 256, "intendedTerm": 1024, + "useful": 256, "notes": 8192} + + +def validate_feedback(value, allowed_ids): + if not isinstance(value, dict) or set(value) != {"schemaVersion", "cases"}: + raise ValueError("Expected schemaVersion and cases") + if type(value["schemaVersion"]) is not int or value["schemaVersion"] != 1: + raise ValueError("Unsupported schemaVersion") + cases = value["cases"] + if not isinstance(cases, list) or len(cases) > len(allowed_ids): + raise ValueError("Invalid cases list") + seen = set() + for case in cases: + if not isinstance(case, dict) or "id" not in case or set(case) - FIELDS.keys(): + raise ValueError("Invalid case fields") + for field, text in case.items(): + if not isinstance(text, str) or len(text) > FIELDS[field]: + raise ValueError("Invalid field type or length") + if case["id"] not in allowed_ids or case["id"] in seen: + raise ValueError("Unknown or duplicate case id") + seen.add(case["id"]) + return value + + +def make_server(directory, port=8847): + root = Path(directory).resolve() + source = json.loads((root / "cases.json").read_text(encoding="utf-8")) + allowed_ids = {case["id"] for case in source["cases"]} + feedback_path = root / "feedback.json" + save_lock = threading.Lock() + + class Handler(SimpleHTTPRequestHandler): + def __init__(self, *args, **kwargs): + super().__init__(*args, directory=str(root), **kwargs) + + def log_message(self, format, *args): + # Request paths and user feedback are deliberately not logged. + pass + + def json_response(self, status, value): + body = json.dumps(value, ensure_ascii=False).encode("utf-8") + self.send_response(status) + self.send_header("Content-Type", "application/json; charset=utf-8") + self.send_header("Content-Length", str(len(body))) + self.send_header("Cache-Control", "no-store") + self.send_header("X-Content-Type-Options", "nosniff") + self.end_headers() + self.wfile.write(body) + + def local_request(self): + hosts = {f"127.0.0.1:{self.server.server_port}", + f"localhost:{self.server.server_port}"} + if self.headers.get("Host") not in hosts: + self.json_response(403, {"ok": False, "error": "Local host required"}) + return False + origins = self.headers.get_all("Origin", []) + if origins and (len(origins) != 1 or origins[0] not in {"http://" + h for h in hosts}): + self.json_response(403, {"ok": False, "error": "Origin rejected"}) + return False + return True + + def send_head(self): + # SimpleHTTPRequestHandler normalizes traversal; this additionally + # rejects symlinks/junctions pointing outside the document root. + candidate = Path(self.translate_path(self.path)).resolve() + if not candidate.is_relative_to(root): + self.send_error(403, "Outside document root") + return None + if candidate.is_dir(): + for name in ("index.html", "index.htm"): + index = candidate / name + if index.exists(): + if not index.resolve().is_relative_to(root): + self.send_error(403, "Outside document root") + return None + break + return super().send_head() + + def do_GET(self): + if not self.local_request(): + return + if urlsplit(self.path).path == "/api/feedback": + try: + if not feedback_path.resolve().is_relative_to(root): + self.json_response(403, {"ok": False, "error": "Outside document root"}) + return + value = (json.loads(feedback_path.read_text(encoding="utf-8")) + if feedback_path.exists() else {"schemaVersion": 1, "cases": []}) + self.json_response(200, value) + except (OSError, ValueError): + self.json_response(500, {"ok": False, "error": "Cannot read saved feedback"}) + return + super().do_GET() + + def do_HEAD(self): + if self.local_request(): + super().do_HEAD() + + def do_POST(self): + if not self.local_request(): + return + if self.path != "/api/feedback": + self.json_response(404, {"ok": False, "error": "Unknown endpoint"}) + return + if self.headers.get_content_type() != "application/json": + self.json_response(415, {"ok": False, "error": "JSON content type required"}) + return + try: + if self.headers.get("Transfer-Encoding"): + raise ValueError("Transfer encoding unsupported") + size = int(self.headers.get("Content-Length", "-1")) + if size < 0: + raise ValueError("Content length required") + if size > MAX_BODY: + self.json_response(413, {"ok": False, "error": "Feedback too large"}) + return + self.connection.settimeout(10) + raw = self.rfile.read(size) + if len(raw) != size: + raise ValueError("Incomplete body") + value = validate_feedback(json.loads(raw), allowed_ids) + except (ValueError, UnicodeError, OSError): + self.json_response(400, {"ok": False, "error": "Invalid feedback"}) + return + temporary = None + try: + with save_lock: + with tempfile.NamedTemporaryFile(mode="w", encoding="utf-8", dir=root, + prefix=".feedback-", suffix=".tmp", delete=False) as out: + temporary = out.name + json.dump(value, out, ensure_ascii=False, indent=2) + out.write("\n") + out.flush() + os.fsync(out.fileno()) + os.replace(temporary, feedback_path) + self.json_response(200, {"ok": True, "savedCases": len(value["cases"]), + "path": str(feedback_path)}) + except OSError: + self.json_response(500, {"ok": False, "error": "Cannot save feedback"}) + finally: + if temporary and os.path.exists(temporary): + os.unlink(temporary) + + return ThreadingHTTPServer(("127.0.0.1", port), Handler) + + +def main(): + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--port", type=int, default=8847) + parser.add_argument("--directory", type=Path, + default=Path(__file__).resolve().parent.parent / ".tmp" / "concept-review") + args = parser.parse_args() + with make_server(args.directory, args.port) as server: + print(f"Review: http://127.0.0.1:{server.server_port}/review.html", flush=True) + print(f"Feedback: {args.directory.resolve() / 'feedback.json'}", flush=True) + try: + server.serve_forever() + except KeyboardInterrupt: + pass + + +if __name__ == "__main__": + main() diff --git a/scripts/tests/test_review_analysis.py b/scripts/tests/test_review_analysis.py new file mode 100644 index 0000000..2b88d8a --- /dev/null +++ b/scripts/tests/test_review_analysis.py @@ -0,0 +1,109 @@ +import json +from pathlib import Path +import sys +import unittest +from unittest.mock import Mock, patch +import queue + +EXPERIMENTS = Path(__file__).resolve().parents[1] / 'experiments' +sys.path.insert(0, str(EXPERIMENTS)) +import collect_review_analysis as collector + + +class ReviewAnalysisTests(unittest.TestCase): + def setUp(self): + self.row = {'sample_id': 'SECRET-ID', 'comment': 'SECRET-COMMENT', + 'metadata': {'secret': True}, 'labels': {'attack': 1}, + 'feedback': 'SECRET-FEEDBACK', 'source': 'SECRET-SOURCE', + 'board_size': 19, 'komi': 6, 'rules': None, + 'initial_stones': [['B', 'D4']], 'moves_before': [['W', 'Q16']], + 'moves_after': [['W', 'Q16'], ['B', 'D16']], 'played_move': ['B', 'D16']} + + def test_requests_are_whitelisted_and_anonymous(self): + requests, fallback = collector.build_requests(self.row, 'position-01') + self.assertEqual(requests['pre']['id'], 'position-01-pre') + self.assertEqual(requests['post']['id'], 'position-01-post') + self.assertNotIn('SECRET', json.dumps(requests)) + self.assertEqual(set(requests['pre']), {'id', 'boardXSize', 'boardYSize', 'rules', + 'komi', 'initialStones', 'maxVisits', 'includeOwnership', 'overrideSettings', 'moves'}) + self.assertEqual(requests['pre']['maxVisits'], 128) + self.assertEqual(requests['pre']['overrideSettings'], {'reportAnalysisWinratesAs': 'BLACK'}) + self.assertIn('Japanese fallback', fallback) + + def test_before_after_preserve_real_history_and_setup(self): + requests, _ = collector.build_requests(self.row, 'position-01') + self.assertEqual(requests['pre']['moves'], self.row['moves_before']) + self.assertEqual(requests['post']['moves'], self.row['moves_after']) + self.assertEqual(requests['pre']['initialStones'], self.row['initial_stones']) + self.assertEqual(requests['pre']['komi'], 6) + requests['pre']['moves'].append(['W', 'A1']) + self.assertEqual(len(self.row['moves_before']), 1) + self.assertEqual(len(requests['post']['moves']), 2) + + def test_rejects_wrong_history_timepoint(self): + self.row['moves_after'] = self.row['moves_before'] + with self.assertRaisesRegex(ValueError, 'plus played_move'): + collector.build_requests(self.row, 'position-01') + + def test_known_rules_preserved(self): + self.row['rules'] = 'Chinese' + requests, fallback = collector.build_requests(self.row, 'position-01') + self.assertEqual(requests['pre']['rules'], 'chinese') + self.assertIsNone(fallback) + + def test_anonymous_export_does_not_copy_source_fields(self): + requests, fallback = collector.build_requests(self.row, 'position-01') + payload = collector.anonymous_case('position-01', requests, {}, fallback) + text = json.dumps(payload) + self.assertNotIn('SECRET', text) + for field in ('local_case_id', 'sample_id', 'source', 'labels', 'comment', 'metadata', 'feedback'): + self.assertNotIn(field, payload) + self.assertEqual(payload['positions']['pre']['moves'], self.row['moves_before']) + self.assertEqual(payload['positions']['post']['moves'], self.row['moves_after']) + + def test_result_validation_and_optional_ownership(self): + requests, _ = collector.build_requests(self.row, 'position-01', ownership=True) + result = {'id': requests['pre']['id'], + 'rootInfo': {'visits': 128, 'winrate': .5, 'scoreLead': -1}, + 'moveInfos': [{'move': 'D16', 'pv': ['D16', 'Q4'], 'visits': 80, + 'winrate': .51, 'scoreLead': -.2}], 'ownership': [0.] * 361} + self.assertEqual(collector.validate_result(result, requests['pre'])['rootInfo']['visits'], 128) + for bad in (None, [0.] * 360, [float('nan')] * 361, [2.] * 361, [False] * 361): + result['ownership'] = bad + with self.assertRaises(ValueError): + collector.validate_result(result, requests['pre']) + result['ownership'] = [0.] * 361 + del result['rootInfo']['scoreLead'] + with self.assertRaises(ValueError): + collector.validate_result(result, requests['pre']) + + def test_query_sends_anonymous_request_and_measures_response_time(self): + requests, _ = collector.build_requests(self.row, 'position-01') + request = requests['pre'] + result = {'id': request['id'], 'rootInfo': {'visits': 128, 'winrate': .5, 'scoreLead': 1}, + 'moveInfos': [{'move': 'D16', 'pv': ['D16'], 'visits': 128, + 'winrate': .5, 'scoreLead': 1}]} + engine = collector.Engine.__new__(collector.Engine) + engine.proc = Mock() + engine.responses = queue.Queue() + engine.responses.put(dict(result, isDuringSearch=True)) + engine.responses.put(result) + with patch.object(collector.time, 'monotonic', side_effect=[10., 10.1, 10.2, 10.7]): + evidence = engine.query(request, 120) + sent = json.loads(engine.proc.stdin.write.call_args.args[0]) + self.assertEqual(sent, request) + self.assertNotIn('SECRET', json.dumps(sent)) + self.assertAlmostEqual(evidence['elapsed_seconds'], .7) + + def test_engine_error_raises_instead_of_fabricating_evidence(self): + requests, _ = collector.build_requests(self.row, 'position-01') + engine = collector.Engine.__new__(collector.Engine) + engine.proc = Mock() + engine.responses = queue.Queue() + engine.responses.put({'id': requests['pre']['id'], 'error': 'illegal move'}) + with self.assertRaisesRegex(RuntimeError, 'illegal move'): + engine.query(requests['pre'], 120) + + +if __name__ == '__main__': + unittest.main() diff --git a/scripts/tests/test_review_server.py b/scripts/tests/test_review_server.py new file mode 100644 index 0000000..6949c27 --- /dev/null +++ b/scripts/tests/test_review_server.py @@ -0,0 +1,96 @@ +"""Real HTTP behavior checks; all feedback is written to disposable fixtures.""" +import http.client +import importlib.util +import json +from pathlib import Path +import tempfile +import threading +import unittest + +spec = importlib.util.spec_from_file_location("review_server", Path(__file__).resolve().parents[1] / "serve_concept_review.py") +module = importlib.util.module_from_spec(spec) +spec.loader.exec_module(module) + + +class ReviewServerTest(unittest.TestCase): + def setUp(self): + self.temp = tempfile.TemporaryDirectory() + self.root = Path(self.temp.name) + (self.root / "cases.json").write_text(json.dumps({"cases": [{"id": "004:30"}]})) + (self.root / "review.html").write_text("review fixture") + self.server = module.make_server(self.root, 0) + self.thread = threading.Thread(target=self.server.serve_forever, daemon=True) + self.thread.start() + + def tearDown(self): + self.server.shutdown() + self.server.server_close() + self.thread.join() + self.temp.cleanup() + + def request(self, method="GET", body=None, headers=None, path="/api/feedback"): + connection = http.client.HTTPConnection("127.0.0.1", self.server.server_port, timeout=5) + connection.request(method, path, body, headers or {}) + response = connection.getresponse() + result = response.status, response.read() + connection.close() + return result + + def post(self, value, **headers): + return self.request("POST", json.dumps(value, ensure_ascii=False).encode("utf-8"), + {"Content-Type": "application/json", **headers}) + + def test_persist_and_reload(self): + self.assertEqual(json.loads(self.request()[1]), {"schemaVersion": 1, "cases": []}) + value = {"schemaVersion": 1, "cases": [{"id": "004:30", "geometryCorrect": "正确", + "intendedTerm": "拆二", "useful": "有帮助", "notes": "中文反馈"}]} + status, raw = self.post(value, Origin=f"http://127.0.0.1:{self.server.server_port}") + self.assertEqual(status, 200) + self.assertEqual(json.loads(raw), {"ok": True, "savedCases": 1, "path": str(self.root / "feedback.json")}) + self.assertEqual(json.loads((self.root / "feedback.json").read_text(encoding="utf-8")), value) + self.assertEqual(json.loads(self.request()[1]), value) + self.server.shutdown() + self.server.server_close() + self.thread.join() + self.server = module.make_server(self.root, 0) + self.thread = threading.Thread(target=self.server.serve_forever, daemon=True) + self.thread.start() + self.assertEqual(json.loads(self.request()[1]), value) + self.assertEqual(self.post({"schemaVersion": 1, "cases": [{"id": "unknown"}]})[0], 400) + self.assertEqual(json.loads(self.request()[1]), value) + self.assertEqual(list(self.root.glob(".feedback-*.tmp")), []) + self.assertEqual(self.request(path="/review.html"), (200, b"review fixture")) + + def test_invalid_requests_do_not_write(self): + invalid = [{"schemaVersion": 2, "cases": []}, + {"schemaVersion": True, "cases": []}, + {"schemaVersion": 1, "cases": [{"id": "unknown"}]}, + {"schemaVersion": 1, "cases": [{"id": "004:30", "notes": 1}]}, + {"schemaVersion": 1, "cases": [{"id": "004:30", "notes": "x" * 8193}]}, + {"schemaVersion": 1, "cases": [{"id": "004:30", "path": "outside"}]}] + for value in invalid: + with self.subTest(value=str(value)[:80]): + self.assertEqual(self.post(value)[0], 400) + self.assertEqual(self.request("POST", b"{", {"Content-Type": "application/json"})[0], 400) + self.assertEqual(self.post({"schemaVersion": 1, "cases": []}, Origin="https://example.com")[0], 403) + self.assertEqual(self.request(headers={"Origin": "null"})[0], 403) + self.assertEqual(self.request(headers={"Host": "example.com"})[0], 403) + self.assertEqual(self.request("POST", b"{}", {"Content-Type": "text/plain"})[0], 415) + self.assertEqual(self.request("POST", b"", {"Content-Type": "application/json", + "Content-Length": str(module.MAX_BODY + 1)})[0], 413) + self.assertFalse((self.root / "feedback.json").exists()) + + def test_static_stays_within_root(self): + with tempfile.TemporaryDirectory() as outside: + secret = Path(outside) / "secret.txt" + secret.write_text("outside secret") + self.assertNotIn(b"outside secret", self.request(path="/../secret.txt")[1]) + try: + (self.root / "linked.txt").symlink_to(secret) + except OSError: + return # Windows may require symlink privileges. + self.assertEqual(self.request(path="/linked.txt")[0], 403) + + +if __name__ == "__main__": + unittest.main() diff --git a/src/main/services/analysis/ownership.ts b/src/main/services/analysis/ownership.ts new file mode 100644 index 0000000..22d34e2 --- /dev/null +++ b/src/main/services/analysis/ownership.ts @@ -0,0 +1,36 @@ +interface RootOwnershipResponse { + ownership?: unknown + ownershipStdev?: unknown + rootInfo?: { + ownership?: unknown + ownershipStdev?: unknown + } +} + +function validOwnershipArray(value: unknown, boardSize: number, minimum: number): number[] | undefined { + if (!Number.isInteger(boardSize) || boardSize <= 0 || !Array.isArray(value) || value.length !== boardSize * boardSize) { + return undefined + } + // Array.from also exposes sparse entries so a hole cannot pass validation. + const values = Array.from(value) + if (!values.every((entry) => typeof entry === 'number' && Number.isFinite(entry) && entry >= minimum && entry <= 1)) { + return undefined + } + return values +} + +/** + * Official Analysis Engine root ownership lives at the response top level. + * Legacy remote adapters may nest it in rootInfo. Preserve the reported + * SIDETOMOVE perspective, just as candidate ownership does: an after response + * belongs to the after position's player, not the before position's player. + */ +export function readRootOwnership(response: RootOwnershipResponse, boardSize: number): { + ownership?: number[] + ownershipStdev?: number[] +} { + return { + ownership: validOwnershipArray(response.ownership === undefined ? response.rootInfo?.ownership : response.ownership, boardSize, -1), + ownershipStdev: validOwnershipArray(response.ownershipStdev === undefined ? response.rootInfo?.ownershipStdev : response.ownershipStdev, boardSize, 0) + } +} diff --git a/src/main/services/katago.ts b/src/main/services/katago.ts index 2f5448d..c7f8a49 100644 --- a/src/main/services/katago.ts +++ b/src/main/services/katago.ts @@ -27,11 +27,14 @@ import { buildKataGoTracePacket } from './teacher/katagoTraceTranslator' import { classifyMoveAnalysis } from './analysis/classifier' import { buildPvConfidenceReport } from './analysis/pvConfidence' import { createKataGoSearchProgressTracker, onlyKataGoSearchProgressFor } from './analysis/searchTelemetry' +import { readRootOwnership } from './analysis/ownership' interface KataGoResponse { id?: string error?: string isDuringSearch?: boolean + ownership?: number[] + ownershipStdev?: number[] rootInfo?: { currentPlayer?: GameMove['color'] visits?: number @@ -268,7 +271,7 @@ function responseSideToMove(response: KataGoResponse, fallback: GameMove['color' : fallback } -function root(response: KataGoResponse, sideToMove: GameMove['color']): { winrate: number; scoreLead: number; ownership?: number[]; ownershipStdev?: number[] } { +function root(response: KataGoResponse, sideToMove: GameMove['color'], boardSize: number): { winrate: number; scoreLead: number; ownership?: number[]; ownershipStdev?: number[] } { if (!response.rootInfo) { throw new Error(`KataGo 没有返回 rootInfo${response.error ? `: ${response.error}` : ''}`) } @@ -278,8 +281,7 @@ function root(response: KataGoResponse, sideToMove: GameMove['color']): { winrat return { winrate: blackWinrateFromSideToMove(rawWinrate, actualSideToMove), scoreLead: blackScoreLeadFromSideToMove(rawScoreLead, actualSideToMove), - ownership: Array.isArray(response.rootInfo.ownership) ? response.rootInfo.ownership.map(Number) : undefined, - ownershipStdev: Array.isArray(response.rootInfo.ownershipStdev) ? response.rootInfo.ownershipStdev.map(Number) : undefined + ...readRootOwnership(response, boardSize) } } @@ -971,8 +973,8 @@ function buildMoveAnalysis( actualResponse?: KataGoResponse, trialContext?: TrialBranchSummary ): KataGoMoveAnalysis { - const beforeRoot = root(beforeResponse, beforeSideToMove) - const afterRoot = root(afterResponse, afterSideToMove) + const beforeRoot = root(beforeResponse, beforeSideToMove, boardSize) + const afterRoot = root(afterResponse, afterSideToMove, boardSize) const searchMoves = candidates(beforeResponse, beforeSideToMove) const forcedActual = forcedPlayedCandidate(actualResponse, currentMove) const topMoves = displayCandidates(beforeResponse, beforeSideToMove, currentMove, forcedActual) @@ -1496,7 +1498,7 @@ export async function analyzeGameQuick( } try { const sideToMove = sideToMoveAt(moves, position) - roots.set(position, root(response, sideToMove)) + roots.set(position, root(response, sideToMove, record.boardSize)) topMovesByPosition.set(position, candidates(response, sideToMove).slice(0, 8)) emitIfReady(position) emitIfReady(position + 1) @@ -1525,7 +1527,7 @@ export async function analyzeGameQuick( } if (response && !roots.has(moveNumber)) { try { - roots.set(moveNumber, root(response, sideToMoveAt(moves, moveNumber))) + roots.set(moveNumber, root(response, sideToMoveAt(moves, moveNumber), record.boardSize)) } catch { // Keep the quick graph resilient: one invalid branch point should not block the rest. } diff --git a/src/main/services/knowledge/katagoShapeFeatures.ts b/src/main/services/knowledge/katagoShapeFeatures.ts index c16ac69..dc3870f 100644 --- a/src/main/services/knowledge/katagoShapeFeatures.ts +++ b/src/main/services/knowledge/katagoShapeFeatures.ts @@ -70,7 +70,7 @@ export function extractKataGoShapeFeatures(input: KataGoShapeFeatureInput): Kata score: 18 + Math.min(8, scoreLoss * 2), evidence: [`actual/best distance=${localDistance}`, `scoreLoss=${scoreLoss.toFixed(1)}`, `phase=${gamePhase}`], counterEvidence: [], - recognition: '实战手和首选手在同一局部，KataGo 认为差别主要来自棋形、气数、先后手或次序。', + recognition: '实战手和首选手在同一局部且评价不同。需要比较变化，才能判断差别来自气数、连接、先后手还是次序；距离和目差本身不能确定原因。', wrongThinking: '只看这里都能下，没有比较哪一手更补形、更先手或更限制对方。', correctThinking: '同一区域的两个候选先比较气数、连接、断点、眼形和对方最强应手。', drillPrompt: '遮住 AI 首选，只在这个局部列两个候选，判断哪手更先手、形更完整。', @@ -78,7 +78,7 @@ export function extractKataGoShapeFeatures(input: KataGoShapeFeatureInput): Kata }) } - if (scoreLoss >= 1.5 && localDistance >= 6) { + if (actualPoint && bestPoint && scoreLoss >= 1.5 && localDistance >= 6) { features.push({ id: 'katago-global-vs-local-shape-choice', shapeType: 'local_vs_global_shape', @@ -102,7 +102,7 @@ export function extractKataGoShapeFeatures(input: KataGoShapeFeatureInput): Kata score: 12 + Math.min(6, input.principalVariation?.length ?? 0), evidence: [`pvLength=${input.principalVariation?.length ?? 0}`, `scoreLoss=${scoreLoss.toFixed(1)}`], counterEvidence: [], - recognition: 'KataGo 给出了较长 PV，说明这个棋形判断要结合后续应手，不宜只讲第一感。', + recognition: '已提供后续候选变化，可摆出对应分支比较。PV 的长度本身不能证明棋形、强制性或搜索可靠程度。', wrongThinking: '只看推荐点，不摆对方最强应手，容易把手筋或补形讲错。', correctThinking: '讲棋形时至少沿 PV 摆到双方各 2-3 手，确认收益来自哪里。', drillPrompt: '复盘时先摆首选 PV 前 6 手，再用一句话说出这条线的收益。', diff --git a/src/main/services/knowledge/localPatternMatcher.ts b/src/main/services/knowledge/localPatternMatcher.ts index f2b5a74..3a42f9a 100644 --- a/src/main/services/knowledge/localPatternMatcher.ts +++ b/src/main/services/knowledge/localPatternMatcher.ts @@ -20,7 +20,7 @@ export interface ShapePatternCard { title: string shapeType: string category: string - anchorRole: 'actual' | 'candidate' | 'either' + anchorRole: 'actual' | 'candidate' | 'either' | 'friendly-stone' | 'enemy-stone' | 'empty' phase: Array<'opening' | 'middlegame' | 'endgame' | 'any'> regions: Array<'corner' | 'side' | 'center' | 'any'> tags: string[] @@ -43,6 +43,8 @@ export interface LocalPatternMatcherInput { boardSnapshot?: BoardSnapshotStone[] localWindows?: LocalWindow[] anchors: Array<string | undefined> + playedMove?: string + candidateMoves?: string[] playerColor?: 'B' | 'W' phase?: 'opening' | 'middlegame' | 'endgame' } @@ -232,7 +234,7 @@ function matchPatternAt( const ratio = required > 0 ? matched / required : 0 const rawScore = Math.round(ratio * 20 + Math.min(8, matched) + (constraintsOk ? 4 : 0) - (antiHit ? 8 : 0)) const minScore = card.minScore ?? 16 - if (rawScore < minScore || counterEvidence.length > Math.max(1, required - matched + 1)) continue + if (rawScore < minScore || counterEvidence.length > 0) continue const confidence: LocalPatternConfidence = rawScore >= 26 ? 'strong' : rawScore >= 20 ? 'medium' : 'weak' const candidate: LocalPatternMatch = { card, @@ -252,18 +254,35 @@ function matchPatternAt( } export function findLocalPatternMatches(cards: ShapePatternCard[], input: LocalPatternMatcherInput): LocalPatternMatch[] { + if (!input.boardSnapshot) return [] const board = buildBoard(input.boardSnapshot, input.boardSize) - const windowAnchors = (input.localWindows ?? []).map((window) => window.anchor) - const anchors = Array.from(new Set([...input.anchors, ...windowAnchors].filter(Boolean) as string[])) - const perspectives = Array.from(new Set([input.playerColor, 'B', 'W'].filter(Boolean) as Array<'B' | 'W'>)) + // A window can be centered on a future PV point; its center alone is not + // evidence of an actual move or a current candidate. + const anchors = Array.from(new Set([...input.anchors, input.playedMove, ...(input.candidateMoves ?? [])].filter(Boolean) as string[])) + const perspectives: Array<'B' | 'W'> = input.playerColor ? [input.playerColor] : ['B', 'W'] const matches: LocalPatternMatch[] = [] for (const card of cards) { if (!phaseMatches(card, input.phase)) continue for (const anchor of anchors) { for (const perspective of perspectives) { + const coord = gtpToCoord(anchor, input.boardSize) + if (!coord) continue + const stone = board.get(key(coord.row, coord.col)) + if (card.anchorRole === 'actual' && anchor !== input.playedMove) continue + if (card.anchorRole === 'candidate' && !(input.candidateMoves ?? []).includes(anchor)) continue + if (card.anchorRole === 'friendly-stone' && stone !== perspective) continue + if (card.anchorRole === 'enemy-stone' && stone !== opposite(perspective)) continue + if (card.anchorRole === 'empty' && stone !== undefined) continue const match = matchPatternAt(card, board, anchor, input.boardSize, perspective) - if (match) matches.push(match) + if (match) { + match.evidence.unshift(`anchor-role=${card.anchorRole}`) + if (!input.playerColor) { + match.confidence = 'weak' + match.evidence.unshift(`perspective-unverified=${perspective}; current-player unknown`) + } + matches.push(match) + } } } } diff --git a/src/main/services/knowledge/matchEngine.ts b/src/main/services/knowledge/matchEngine.ts index ca1e47e..fe3553c 100644 --- a/src/main/services/knowledge/matchEngine.ts +++ b/src/main/services/knowledge/matchEngine.ts @@ -61,8 +61,6 @@ interface QueryFeatures { explicitIntentTypes: Set<KnowledgeMatchType> moveFeatures: Set<string> candidateFeatures: Set<string> - pvFeatures: Set<string> - allPoints: Set<string> } type ProblemEntry = LifeDeathProblem | TesujiProblem @@ -293,7 +291,6 @@ function detectRegion(query: KnowledgeMatchQuery): TrainingRegion { const points = [ query.playedMove, ...(query.candidateMoves ?? []).slice(0, 3), - ...(query.principalVariation ?? []).slice(0, 5), ...query.recentMoves.map((move) => move.gtp) ].filter(Boolean) as string[] let corner = 0 @@ -325,15 +322,6 @@ function buildFeatures(query: KnowledgeMatchQuery): QueryFeatures { ...addFeaturesFromGtp(query.playedMove ? [query.playedMove] : [], query.boardSize) ]) const candidateFeatures = addFeaturesFromGtp(query.candidateMoves, query.boardSize) - const pvFeatures = addFeaturesFromGtp(query.principalVariation, query.boardSize) - const allPoints = new Set([ - query.playedMove, - ...(query.candidateMoves ?? []), - ...(query.principalVariation ?? []), - ...query.recentMoves.map((move) => move.gtp), - ...(query.boardSnapshot ?? []).map((stone) => stone.point), - ...(query.localWindows ?? []).flatMap((window) => [window.anchor, ...window.stones.map((stone) => stone.point)]) - ].filter(Boolean) as string[]) return { phase: phaseFromMove(query.moveNumber, query.totalMoves), @@ -343,9 +331,7 @@ function buildFeatures(query: KnowledgeMatchQuery): QueryFeatures { intentTypes: detectIntentTypes(tokens), explicitIntentTypes: detectIntentTypes(explicitTokens), moveFeatures, - candidateFeatures, - pvFeatures, - allPoints + candidateFeatures } } @@ -394,19 +380,6 @@ function featureOverlap(values: string[], features: Set<string>, weight: number, return score } -function sequenceOverlap(sequence: string[], points: Set<string>, reasons: string[]): number { - let overlap = 0 - for (const point of sequence) { - if (points.has(point)) { - overlap += 1 - } - } - if (overlap > 0) { - reasons.push(`sequence-overlap:${overlap}`) - } - return overlap -} - function uniqueValidPoints(points: Array<string | undefined>, boardSize: number): string[] { const seen = new Set<string>() const values: string[] = [] @@ -446,6 +419,7 @@ function stoneKey(stone: RelativeStone, transform: RelativeTransform, swapColors } function queryStonesForAnchor(query: KnowledgeMatchQuery, anchor: string): BoardSnapshotStone[] { + if (query.boardSnapshot) return query.boardSnapshot const localWindow = (query.localWindows ?? []).find((window) => window.anchor === anchor) if (localWindow && localWindow.stones.length > 0) { return localWindow.stones @@ -586,8 +560,6 @@ function localShapeGeometryMatch(problem: ProblemEntry, query: KnowledgeMatchQue const queryAnchors = uniqueValidPoints([ query.playedMove, ...(query.candidateMoves ?? []).slice(0, 6), - ...(query.principalVariation ?? []).slice(0, 4), - ...(query.localWindows ?? []).map((window) => window.anchor) ], query.boardSize) const problemAnchors = uniqueValidPoints(problem.correctMoves.slice(0, 2).map((move) => move.move), query.boardSize) if (queryAnchors.length === 0 || problemAnchors.length === 0) return null @@ -609,6 +581,10 @@ function localShapeGeometryMatch(problem: ProblemEntry, query: KnowledgeMatchQue for (const transform of LOCAL_SHAPE_TRANSFORMS) { for (const swapColors of [false, true]) { + // A color-swapped shape is relevant only when the problem's player + // maps to the real current player. Unknown roles remain analogies. + const mappedPlayer = swapColors ? (problem.toPlay === 'B' ? 'W' : 'B') : problem.toPlay + if (query.playerColor && mappedPlayer !== query.playerColor) continue let matched = 0 const transformedKeys = new Set(problemRelatives.map((stone) => stoneKey(stone, transform.apply, swapColors))) for (const key of transformedKeys) { @@ -659,15 +635,14 @@ function capConfidence(value: KnowledgeMatchConfidence, max: KnowledgeMatchConfi function sortMatchScore(match: KnowledgeMatch): number { const intentBonus = match.reason.some((reason) => reason.startsWith('explicit-intent:')) ? 8 : 0 - const exactEvidenceBonus = match.reason.some((reason) => reason.startsWith('answer-overlap') || reason.startsWith('sequence-overlap')) ? 4 : 0 + const topicBonus = match.reason.some((reason) => reason.startsWith('explicit-title-topic:')) ? 30 : 0 const geometryBonus = match.reason.some((reason) => reason.startsWith('geometry:')) ? 6 : 0 - return CONFIDENCE_RANK[match.confidence] * 1000 + match.score + intentBonus + exactEvidenceBonus + geometryBonus + return CONFIDENCE_RANK[match.confidence] * 1000 + match.score + intentBonus + topicBonus + geometryBonus } function applicabilityFor(confidenceValue: KnowledgeMatchConfidence, type: KnowledgeMatchType): string { - if (confidenceValue === 'exact') return '本局局部手顺、候选点和区域都高度一致，可以作为同型讲解。' - if (confidenceValue === 'strong') return '本局棋形和 KataGo 候选点相近，可以作为强相关型讲解，但仍要看全局厚薄。' - if (confidenceValue === 'partial') return `本局只是像这个${type === 'joseki' ? '定式' : '棋形'}，老师应说“像这个型”，不能硬套结论。` + if (confidenceValue === 'exact' || confidenceValue === 'strong') return '当前局部与训练题几何相似，仅表示检索相关；未验证题库答案在本局是否合法，不能据此确认死活、手筋或定式结论。' + if (confidenceValue === 'partial') return `仅供${type === 'joseki' ? '定式' : '棋形'}训练类比，尚未确认本局同型；题库答案坐标不对应本局，不能硬套结论。` return '弱相关，只适合作为备用训练建议，不应进入主讲。' } @@ -733,13 +708,7 @@ function josekiMatch(line: JosekiLine, query: KnowledgeMatchQuery, features: Que reasons.push('context-intent:joseki') } score += addOverlapScore([...line.tags, line.title, line.family], features.tokens, 4, reasons, 'text') - score += featureOverlap(line.normalizedFeatures, new Set([...features.moveFeatures, ...features.candidateFeatures, ...features.pvFeatures]), 4, reasons, 'shape') - const overlap = sequenceOverlap(line.relativeSequence, features.allPoints, reasons) - score += overlap * (features.phase === 'opening' || explicitJosekiIntent ? 5 : 2) - if ((query.candidateMoves ?? []).includes(line.relativeSequence[1])) { - score += 5 - reasons.push('katago-candidate-prefix') - } + score += featureOverlap(line.normalizedFeatures, new Set([...features.moveFeatures, ...features.candidateFeatures]), 4, reasons, 'shape') if (query.moveNumber <= 70) { score += 2 reasons.push('opening-timing') @@ -749,11 +718,10 @@ function josekiMatch(line: JosekiLine, query: KnowledgeMatchQuery, features: Que reasons.push(explicitTacticalIntent ? 'penalty:tactical-query' : 'penalty:non-opening-joseki-context') } if (score < 8) return null - const exactish = overlap >= 3 && (features.phase === 'opening' || explicitJosekiIntent) && !explicitTacticalIntent - const rawConfidence = confidence(score, exactish) - const confidenceValue = features.phase !== 'opening' && !explicitJosekiIntent - ? capConfidence(rawConfidence, 'partial') - : rawConfidence + // Text, region and move-coordinate features rank training analogies; they + // do not establish an ordered, color-aware joseki in the current position. + const confidenceValue = capConfidence(confidence(score), 'partial') + reasons.unshift('training-analogy:no-verified-joseki-geometry') return { id: line.id, matchType: 'joseki', @@ -764,7 +732,7 @@ function josekiMatch(line: JosekiLine, query: KnowledgeMatchQuery, features: Que applicability: applicabilityFor(confidenceValue, 'joseki'), teachingPayload: { summary: line.katagoEraJudgement, - recognition: `识别为${line.title}相关局部：看角部手顺、挂角/点三三位置和外势方向。`, + recognition: `${line.title}训练参考：看角部手顺、挂角/点三三位置和外势方向，尚未确认本局符合该型。`, correctIdea: line.decisionRules.join(' '), keyVariations: line.branches.slice(0, 3).map((branch) => `${branch.name}: ${branch.whenToChoose}`), memoryCue: '定式先问方向，再问先手，最后才背手顺。', @@ -814,32 +782,16 @@ function problemMatch( reasons.push(`specific-text:${type}`) } score += addOverlapScore(textValues, features.tokens, 4, reasons, 'text') - score += featureOverlap(problem.tags, new Set([...features.moveFeatures, ...features.candidateFeatures, ...features.pvFeatures]), 3, reasons, 'shape') - const answerOverlap = sequenceOverlap(problem.correctMoves.map((move) => move.move), features.allPoints, reasons) - score += answerOverlap * 12 + score += featureOverlap(problem.tags, new Set([...features.moveFeatures, ...features.candidateFeatures]), 3, reasons, 'shape') const geometry = localShapeGeometryMatch(problem, query) if (geometry) { score += geometry.score - reasons.push(`geometry:${geometry.transform}:${geometry.colorMode}:${geometry.matched}/${geometry.expected}`) - reasons.push(`liberties:${geometry.libertyScore}`) + reasons.unshift(`geometry:${geometry.transform}:${geometry.colorMode}:${geometry.matched}/${geometry.expected}`, `liberties:${geometry.libertyScore}`) if (geometry.ratio >= 0.72) { score += 6 reasons.push('geometry-strong-local-shape') } } - const answerPoints = new Set(problem.correctMoves.map((move) => move.move)) - if (query.playedMove && answerPoints.has(query.playedMove)) { - score += 5 - reasons.push('answer-played') - } - if ((query.candidateMoves ?? []).some((move) => answerPoints.has(move))) { - score += 7 - reasons.push('answer-candidate') - } - if ((query.principalVariation ?? []).some((move) => answerPoints.has(move))) { - score += 5 - reasons.push('answer-pv') - } if (type === 'life_death' && features.moveFeatures.has('eye-shape')) { score += 5 reasons.push('eye-shape') @@ -849,8 +801,14 @@ function problemMatch( reasons.push('local-tesuji-relation') } if (score < 8) return null - const exactish = (answerOverlap >= 1 && specificTextHit && (explicitTypeIntent || score >= 24)) || Boolean(geometry && geometry.ratio >= 0.72 && geometry.matched >= 3) - const confidenceValue = confidence(score, exactish) + const grounded = Boolean(geometry && geometry.ratio >= 0.72 && query.playerColor && query.boardSnapshot) + // Even matching local stones and liberties is a shape analogy, not proof + // that the training answer is legal or solves the live position. + const confidenceValue = capConfidence(confidence(score), grounded ? 'strong' : 'partial') + reasons.unshift(grounded ? 'grounding:relative-geometry-and-player-role' : 'training-analogy:no-verified-geometry-and-role') + const titleTopic = problem.title.split(/[：:]/)[0].trim().toLowerCase() + if (titleTopic.length >= 2 && query.text?.toLowerCase().includes(titleTopic)) reasons.unshift(`explicit-title-topic:${titleTopic}`) + if (!query.boardSnapshot && geometry) reasons.unshift('incomplete-local-window:empty-points-and-liberties-unverified') const lifeTeaching = type === 'life_death' ? (problem as LifeDeathProblem).teaching : undefined const tesujiTeaching = type === 'tesuji' ? (problem as TesujiProblem).teaching : undefined return { @@ -863,7 +821,7 @@ function problemMatch( applicability: applicabilityFor(confidenceValue, type), teachingPayload: { summary: problem.objective, - recognition: lifeTeaching?.recognition ?? tesujiTeaching?.recognition ?? '先识别局部形状和双方气数。', + recognition: `训练类比，尚未确认本局同型或战术成立：${lifeTeaching?.recognition ?? tesujiTeaching?.recognition ?? '先识别局部形状和双方气数。'}`, correctIdea: lifeTeaching?.explanation ?? tesujiTeaching?.tesujiIdea ?? '先找急所，再读失败手。', keyVariations: problem.correctMoves.slice(0, 2).map((move) => `${move.move}: ${move.explanation ?? '正确第一手'}`), memoryCue: lifeTeaching?.memoryCue ?? tesujiTeaching?.memoryCue ?? '记住急所和次序。', @@ -905,14 +863,13 @@ export function searchKnowledgeMatchEngine(dataRoot: string, query: KnowledgeMat text: query.text, playedMove: query.playedMove, candidateMoves: query.candidateMoves, - principalVariation: query.principalVariation, lossScore: query.lossScore, judgement: query.judgement }).slice(0, 4) for (const pattern of patternMatches) { const matchType: KnowledgeMatchType = pattern.card.category === 'shape' ? 'shape' : pattern.card.category - const confidenceValue: KnowledgeMatchConfidence = pattern.confidence === 'high' ? 'strong' : pattern.confidence === 'medium' ? 'partial' : 'weak' + const confidenceValue: KnowledgeMatchConfidence = pattern.confidence === 'low' ? 'weak' : 'partial' matches.push({ id: pattern.card.id, matchType, @@ -923,7 +880,7 @@ export function searchKnowledgeMatchEngine(dataRoot: string, query: KnowledgeMat applicability: applicabilityFor(confidenceValue, matchType), teachingPayload: { summary: pattern.card.teaching.correctIdea, - recognition: pattern.card.teaching.recognition, + recognition: `训练类比，尚未确认本局同型或战术成立：${pattern.card.teaching.recognition}`, correctIdea: pattern.card.teaching.correctIdea, keyVariations: pattern.card.variations.slice(0, 3).map((variation) => `${variation.name}: ${variation.whenToChoose}`), memoryCue: pattern.card.teaching.memoryCue, @@ -1004,15 +961,15 @@ export function formatKnowledgeMatchForPrompt(match: KnowledgeMatch): string { return [ `匹配类型: ${match.matchType}`, `名称: ${match.title}`, - `置信度: ${match.confidence}`, + `检索相关置信度: ${match.confidence}（不代表本局概念或战术成立）`, `匹配依据: ${match.reason.join(', ')}`, `适用边界: ${match.applicability}`, `识别特征: ${match.teachingPayload.recognition}`, - `正确思路: ${match.teachingPayload.correctIdea}`, - `常见变化: ${match.teachingPayload.keyVariations.join('；')}`, + `训练题思路: ${match.teachingPayload.correctIdea}`, + `训练题变化（坐标不对应本局）: ${match.teachingPayload.keyVariations.join('；')}`, `记忆法: ${match.teachingPayload.memoryCue}`, `常见误区: ${match.teachingPayload.commonMistakes.join('；')}`, `训练建议: ${match.relatedProblems.map((problem) => `${problem.title}(${problem.difficulty})`).join('、') || match.teachingPayload.drills.join('；')}`, - '老师使用边界: exact/strong 可以说“这是某某型”；partial 只能说“像某某型”；weak 不进入主讲。' + '老师使用边界: 所有置信度仅表示检索相关；本局概念命名必须有真实棋子关系证据，死活、手筋或定式判断必须另有合法手顺和读棋证据，不能由题库命中确认。weak 不进入主讲。' ].join('\n') } diff --git a/src/main/services/knowledge/moveConcepts.ts b/src/main/services/knowledge/moveConcepts.ts new file mode 100644 index 0000000..31c459b --- /dev/null +++ b/src/main/services/knowledge/moveConcepts.ts @@ -0,0 +1,346 @@ +import { buildOwnDevelopment, type OwnDevelopmentCandidate } from '../teacher/ownDevelopment' +/** A deliberately conservative, board-only concept experiment. + * Geometry is observed; purpose is a hypothesis. No score here is calibrated confidence. + */ +export type ConceptColor = 'B' | 'W' +export type MoveGeometryKind = 'connected' | 'diagonal' | 'one-space-jump' | 'two-space-jump' | 'three-space-jump' | 'small-knight' | 'large-knight' +export interface MoveConceptInput { + boardSize: number + /** Complete position BEFORE the move; absence means unknown, not an empty board. */ + boardSnapshot?: Array<{ point: string; color: ConceptColor }> + playedMove: string + playerColor: ConceptColor + /** Already played moves, ending BEFORE this move. Pass is allowed. */ + moveHistory?: Array<{ point: string; color: ConceptColor }> + /** Optional external assessments. These must not be generated from liberty counts alone. */ + groupAssessments?: Array<{ stones: string[]; color: ConceptColor; status: 'unsettled' | 'settled' | 'unknown'; source: string; evidence: string[] }> + /** KataGo ownership uses row-major order, top row first. Explicit perspective is required. */ + engineFeatures?: { ownership?: number[]; ownershipPerspective?: 'black' | 'white'; source?: string } +} +export interface MoveGeometryEvidence { + kind: MoveGeometryKind + label: string + anchor: string + move: string + offset: { dx: number; dy: number } + status: 'observed' | 'blocked' + interveningPoints: string[] + blockers: string[] + wording: string +} +export interface MoveIntentHypothesis { + kind: 'side-extension' | 'escape' | 'attack' | 'press' + status: 'hypothesis' + evidence: string[] + counterEvidence: string[] + wording: string +} +export interface MoveConceptResult { + status: 'observed' | 'ambiguous' | 'abstained' + geometry: MoveGeometryEvidence[] + ownDevelopment: OwnDevelopmentCandidate[] + primaryFriendlyGeometry?: MoveGeometryEvidence + relationCandidates: MoveRelationCandidate[] + relations: Array<{ kind: 'enemy-contact' | 'enemy-position' | 'between-enemies' | 'ownership-at-move'; points: string[]; evidence: string; offset?: { dx: number; dy: number }; value?: number }> + intents: MoveIntentHypothesis[] + wording: string + limitations: string[] +} +export interface MoveRelationCandidate { + kind: 'attachment-under' | 'contact-press' | 'cap' | 'diagonal-contact' | 'corner-entry' | 'knight-pressure' | 'corner-enclosure' | 'shoulder-probe' + status: 'candidate' + geometryStatus: 'observed' + enemyAnchor?: string + friendlyAnchor?: string + supportingFriendPoints: string[] + emptyPoints: string[] + side?: 'left' | 'right' | 'bottom' | 'top' + offset: { dx: number; dy: number } + evidence: string[] + counterEvidence: string[] + wording: string + purposeHypothesis: { status: 'hypothesis'; wording: string } +} + +const COLUMNS = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +const SHAPES: Record<string, [MoveGeometryKind, string]> = { + '0,1': ['connected', '相邻连接'], '1,1': ['diagonal', '尖'], + '0,2': ['one-space-jump', '一间跳'], '0,3': ['two-space-jump', '二间跳'], + '0,4': ['three-space-jump', '三间跳'], '1,2': ['small-knight', '小飞'], '1,3': ['large-knight', '大飞'] +} +type Point = { x: number; y: number; point: string } +function parsePoint(raw: string, size: number): Point | undefined { + const point = raw.trim().toUpperCase() + const match = /^([A-HJ-Z])([1-9]\d?)$/.exec(point) + if (!match) return undefined + const x = COLUMNS.indexOf(match[1]); const y = Number(match[2]) - 1 + return x >= 0 && x < size && y >= 0 && y < size ? { x, y, point } : undefined +} +function name(x: number, y: number): string { return `${COLUMNS[x]}${y + 1}` } +function distance(a: Point, b: Point): number { return Math.max(Math.abs(a.x - b.x), Math.abs(a.y - b.y)) } +function edge(p: Point, size: number): number { return Math.min(p.x, p.y, size - 1 - p.x, size - 1 - p.y) } +function fail(reason: string): MoveConceptResult { + return { status: 'abstained', geometry: [], ownDevelopment: [], relationCandidates: [], relations: [], intents: [], wording: '现有证据不足以命名这手棋的棋形或意图。', limitations: [reason] } +} + +/** A local edge frame prevents corner edge-switches from changing inward/outward. */ +function nearbySides(p: Point, size: number) { + const sides = [ + { side: 'left' as const, depth: p.x, nx: 1, ny: 0 }, + { side: 'right' as const, depth: size - 1 - p.x, nx: -1, ny: 0 }, + { side: 'bottom' as const, depth: p.y, nx: 0, ny: 1 }, + { side: 'top' as const, depth: size - 1 - p.y, nx: 0, ny: -1 } + ] + const minimum = Math.min(...sides.map((s) => s.depth)) + // Ties lack a unique side direction. Targets must be within the fourth line. + return minimum <= 3 ? sides.filter((s) => s.depth === minimum) : [] +} + +function recognizeRelations(input: MoveConceptInput, move: Point, enemies: Point[], board: Map<string, ConceptColor>, geometry: MoveGeometryEvidence[]): MoveRelationCandidate[] { + const candidates: MoveRelationCandidate[] = [] + const observed = geometry.filter((g) => g.status === 'observed') + const add = (kind: MoveRelationCandidate['kind'], enemy: Point, support: string[], emptyPoints: string[], wording: string, purpose: string, evidence: string[], side?: MoveRelationCandidate['side'], friendlyFocus = false) => { + candidates.push({ kind, status: 'candidate', geometryStatus: 'observed', ...(friendlyFocus ? { friendlyAnchor: enemy.point } : { enemyAnchor: enemy.point }), + supportingFriendPoints: support, emptyPoints, side, offset: { dx: move.x - enemy.x, dy: move.y - enemy.y }, + evidence, counterEvidence: ['形状关系不证明限制、封锁或防守已经成功；仍需验证敌方应对与后续变化。'], wording, + purposeHypothesis: { status: 'hypothesis', wording: purpose } }) + } + for (const enemy of enemies) { + const dx = move.x - enemy.x; const dy = move.y - enemy.y + const contact = Math.abs(dx) + Math.abs(dy) === 1 + const diagonalSupports = observed.filter((g) => g.kind === 'diagonal').filter((g) => { + const anchor = parsePoint(g.anchor, input.boardSize)! + // The contact is ahead of the diagonal, not on its back side. + return edge(anchor, input.boardSize) <= 3 && (move.x - anchor.x) * (enemy.x - move.x) + (move.y - anchor.y) * (enemy.y - move.y) > 0 + }) + if (contact && diagonalSupports.length) { + const cornerStar = input.boardSize === 19 && diagonalSupports.some((g) => { + const anchor = parsePoint(g.anchor, 19)! + if (![3, 15].includes(anchor.x) || ![3, 15].includes(anchor.y)) return false + const ex = anchor.x === 3 ? enemy.x : 18 - enemy.x + const ey = anchor.y === 3 ? enemy.y : 18 - enemy.y + return (ex === 2 && ey === 5) || (ex === 5 && ey === 2) + }) + add('diagonal-contact', enemy, diagonalSupports.map((g) => g.anchor), [], + `${move.point}相对己棋${diagonalSupports.map((g) => g.anchor).join('、')}成尖，并正交接触前方敌棋${enemy.point}，可检查是否应称尖顶。`, + cornerStar ? `可能借尖顶限制挂角子${enemy.point}向角部发展；不证明守角已经成功，具体应对仍需验证。` : `可能借尖顶向敌棋${enemy.point}施压；是否有效及是否兼有守角作用仍需读后续。`, ['己方尖的伸出方向与敌方正交接触方向一致。']) + // Same-side spacing is context, never an invented continuation or forced reply. + const frames = nearbySides(enemy, input.boardSize) + if (frames.length === 1) { + const frame = frames[0] + for (const other of enemies) { + const ddx = other.x - enemy.x; const ddy = other.y - enemy.y + if (ddx * frame.nx + ddy * frame.ny !== 0 || Math.abs(ddx * frame.ny - ddy * frame.nx) !== 3) continue + const outward = board.get(name(other.x + Math.sign(ddx) * 2, other.y + Math.sign(ddy) * 2)) === input.playerColor + const corridor = [1, 2].map((i) => name(enemy.x + Math.sign(ddx) * i, enemy.y + Math.sign(ddy) * i)) + if (!outward || corridor.some((p) => board.has(p))) continue + const candidate = candidates[candidates.length - 1] + candidate.evidence.push(`敌棋${enemy.point}与${other.point}沿同侧相隔三格，中间${corridor.join('、')}为空；${other.point}沿该方向再两格有己棋${name(other.x + Math.sign(ddx) * 2, other.y + Math.sign(ddy) * 2)}。`) + candidate.purposeHypothesis.wording += ` 结合${other.point}及其外侧己棋配置，对方沿边展开空间可能受限；未指定应手，不判断其只能如何拆边。` + } + } + } + const sides = nearbySides(enemy, input.boardSize) + if (sides.length !== 1) continue + const frame = sides[0] + const inward = dx * frame.nx + dy * frame.ny + const tangent = dx * frame.ny - dy * frame.nx + const supports = observed.map((g) => g.anchor) + if (contact && tangent === 0 && inward === -1 && supports.length) add('attachment-under', enemy, supports, [], + `${move.point}从${frame.side === 'right' ? '右' : frame.side === 'left' ? '左' : frame.side === 'top' ? '上' : '下'}边线一侧接触敌棋${enemy.point}，可作为托的关系候选。`, + `可能通过托${enemy.point}争取角边空间、限制对方进入；不能据此断定已经防住进角。`, ['敌棋有唯一邻近边线；落点沿同一边线法向低一路接触。'], frame.side) + const closeSupports = observed.filter((g) => ['connected', 'one-space-jump'].includes(g.kind) && + g.offset.dx * frame.nx + g.offset.dy * frame.ny === 0) + if (contact && tangent === 0 && inward === 1 && closeSupports.length) add('contact-press', enemy, closeSupports.map((g) => g.anchor), closeSupports.flatMap((g) => g.interveningPoints), + `${move.point}在敌棋${enemy.point}朝中腹的一侧直接接触，与己棋${closeSupports.map((g) => g.anchor).join('、')}有沿边近距离关系，可检查是否应称靠压。`, + `可能压低${enemy.point}、限制其向中腹发展；实际效果需验证应对。`, ['落点沿敌棋邻近边线的法向高一路，形成正交接触。', '己方支持沿同一深度相邻或空一间；中间走廊没有棋子阻挡。'], frame.side) + if (inward === 1 && Math.abs(tangent) === 1) { + const shoulderSupports = observed.filter((g) => ['one-space-jump', 'two-space-jump'].includes(g.kind) && + g.offset.dx * frame.nx + g.offset.dy * frame.ny === 0 && + (g.offset.dx * frame.ny - g.offset.dy * frame.nx) * tangent > 0) + if (shoulderSupports.length) { + const partners = [...board].filter(([, color]) => color === input.playerColor).map(([p]) => parsePoint(p, input.boardSize)!).filter((p) => + distance(p, move) <= 5 && Math.abs((p.x - move.x) * frame.nx + (p.y - move.y) * frame.ny) <= 1 && + ((p.x - move.x) * frame.ny - (p.y - move.y) * frame.nx) * tangent > 0) + add('shoulder-probe', enemy, shoulderSupports.map((g) => g.anchor), shoulderSupports.flatMap((g) => g.interveningPoints), + `${move.point}斜邻敌棋${enemy.point}，沿其邻近边线高一路并向另一侧展开，与己棋${shoulderSupports.map((g) => g.anchor).join('、')}有开放沿边关系，可检查是否应称尖冲。`, + `可能借此向另一侧扩张空间${partners.length ? `，与该侧己棋${partners.map((p) => p.point).join('、')}配合` : ''}；扩张或配合是否有效仍需验证后续。`, + ['落点对敌斜邻，沿同一边线法向高一路；己方沿边支持位于展开的来向，走廊为空。', ...(partners.length ? [`展开侧五格内、相近深度有己棋${partners.map((p) => p.point).join('、')}；这只是配置事实。`] : [])], frame.side) + } + } + if (tangent === 0 && inward === 2 && supports.length) { + const middle = name(enemy.x + frame.nx, enemy.y + frame.ny) + if (!board.has(middle)) add('cap', enemy, supports, [middle], + `${move.point}在敌棋${enemy.point}朝中腹的正前方空一格，形成镇的关系候选。`, + `可能限制${enemy.point}向中腹跳起，并为己棋争取出头空间；不能断言对方无法跳出。`, [`${enemy.point}到${move.point}沿同一边线法向，中间${middle}为空。`], frame.side) + } + // Enemy and friendly endpoints straddle the knight's tangential level. + if (inward === 2 && Math.abs(tangent) === 1) { + const supports = observed.filter((g) => g.kind === 'small-knight').filter((g) => { + const anchor = parsePoint(g.anchor, input.boardSize)! + return (anchor.x - enemy.x) * frame.nx + (anchor.y - enemy.y) * frame.ny === 0 && + anchor.x === enemy.x + 2 * (dx - 2 * frame.nx) && anchor.y === enemy.y + 2 * (dy - 2 * frame.ny) + }) + const middle = name(enemy.x + dx - 2 * frame.nx, enemy.y + dy - 2 * frame.ny) + const front = name(move.x - frame.nx, move.y - frame.ny) + if (supports.length && !board.has(middle) && !board.has(front)) add('knight-pressure', enemy, supports.map((g) => g.anchor), [middle, front], + `${move.point}与己棋${supports.map((g) => g.anchor).join('、')}成小飞，位于敌棋${enemy.point}朝中腹的一侧，可作为小飞限制的候选。`, + `可能限制敌棋${enemy.point}向外发展；封锁能否成立仍需验证后续应对。`, ['敌棋与己方小飞锚点沿边错开两格；落点处于两者之间的切向高度并向中腹展开。', `${middle}、${front}为空，是局部空间证据，并非已验证的逃出走廊。`], frame.side) + } + } + // Small-knight enclosure of a friendly 3-4 corner, with no enemy in the local corner box. + if (input.boardSize === 19) for (const sx of [1, -1]) for (const sy of [1, -1]) { + const cx = sx === 1 ? 0 : 18; const cy = sy === 1 ? 0 : 18 + for (const [ax, ay, mx, my] of [[3, 2, 2, 4], [2, 3, 4, 2]]) { + if (move.x !== cx + mx * sx || move.y !== cy + my * sy) continue + const anchor = parsePoint(name(cx + ax * sx, cy + ay * sy), 19)! + if (board.get(anchor.point) !== input.playerColor || enemies.some((p) => (p.x - cx) * sx <= 5 && (p.y - cy) * sy <= 5)) continue + const emptyPoints: string[] = [] + for (let x = Math.min(anchor.x, move.x); x <= Math.max(anchor.x, move.x); x++) for (let y = Math.min(anchor.y, move.y); y <= Math.max(anchor.y, move.y); y++) { + const point = name(x, y) + if (point !== move.point && !board.has(point)) emptyPoints.push(point) + } + add('corner-enclosure', anchor, [anchor.point], emptyPoints, + `${move.point}与同角己方小目${anchor.point}成小飞，构成小飞守角的配置候选。`, + '可能加固己方角部、争取角地；不证明角地已确定，也不判断本手是否比其他大场更急。', + [`己方${anchor.point}位于同角第三、第四路；落点为第三、第五路的小飞配置。`, '同角六路范围内没有敌棋；这是局部配置事实，不是守角成功的验证。'], undefined, true) + } + } + // Exact three-three below an enemy star. Smaller boards need separate corner rules. + if (input.boardSize === 19) for (const sx of [1, -1]) for (const sy of [1, -1]) { + const cx = sx === 1 ? 0 : 18; const cy = sy === 1 ? 0 : 18 + if (move.x !== cx + 2 * sx || move.y !== cy + 2 * sy) continue + const star = enemies.find((p) => p.x === cx + 3 * sx && p.y === cy + 3 * sy) + if (!star) continue + const history = input.moveHistory ?? [] + const [approach, pincer] = history.slice(-2) + const a = approach && parsePoint(approach.point, 19); const p = pincer && parsePoint(pincer.point, 19) + const approachShape = a && [[5, 2], [2, 5]].some(([x, y]) => a.x === cx + x * sx && a.y === cy + y * sy) + const pincerShape = a && p && ((a.x === cx + 5 * sx && p.x === cx + 7 * sx && a.y === cy + 2 * sy && p.y === a.y) || + (a.y === cy + 5 * sy && p.y === cy + 7 * sy && a.x === cx + 2 * sx && p.x === a.x)) + const afterPincer = approachShape && pincerShape && approach.color === input.playerColor && pincer.color !== input.playerColor && board.get(a!.point) === approach.color && board.get(p!.point) === pincer.color + add('corner-entry', star, afterPincer ? [a!.point] : [], [], + `${move.point}是敌方星位${star.point}下的三三点，${afterPincer ? '最近两手为己方挂角、敌方夹击，可按被夹后点三三理解' : '可按点三三进入角部理解'}。`, + '可能在角部争取根据地或转换；是否活棋及转换是否有利需验证后续。', [`落点在该角两条边各第三路；同角第四路星位${star.point}属于敌方。`, ...(afterPincer ? [`历史对应己方${a!.point}挂角、敌方${p!.point}夹击。`] : ['缺少匹配的最近挂角、夹击历史，不断言被夹后点角。'])]) + } + const priority: MoveRelationCandidate['kind'][] = ['corner-entry', 'diagonal-contact', 'attachment-under', 'contact-press', 'cap', 'knight-pressure', 'shoulder-probe', 'corner-enclosure'] + return candidates.sort((a, b) => priority.indexOf(a.kind) - priority.indexOf(b.kind) || (a.enemyAnchor ?? a.friendlyAnchor ?? '').localeCompare(b.enemyAnchor ?? b.friendlyAnchor ?? '')) +} + +export function recognizeMoveConcepts(input: MoveConceptInput): MoveConceptResult { + if (!Number.isInteger(input.boardSize) || input.boardSize < 2 || input.boardSize > COLUMNS.length) return fail('棋盘尺寸无效。') + const move = parsePoint(input.playedMove, input.boardSize) + if (!move) return fail('落点无效或为停着。') + if (!input.boardSnapshot) return fail('缺少完整落子前棋盘。') + const board = new Map<string, ConceptColor>() + for (const stone of input.boardSnapshot) { + const point = parsePoint(stone.point, input.boardSize) + if (!point || (stone.color !== 'B' && stone.color !== 'W')) return fail('棋盘包含无效棋子。') + if (board.has(point.point)) return fail('棋盘包含重复落点，无法确定局面。') + board.set(point.point, stone.color) + } + if (input.playerColor !== 'B' && input.playerColor !== 'W') return fail('落子颜色无效。') + if (board.has(move.point)) return fail('落点已被占用；必须提供落子前棋盘。') + const friends: Point[] = []; const enemies: Point[] = [] + for (const [point, color] of board) (color === input.playerColor ? friends : enemies).push(parsePoint(point, input.boardSize)!) + const geometry: MoveGeometryEvidence[] = [] + for (const anchor of friends) { + const dx = move.x - anchor.x; const dy = move.y - anchor.y + const axes = [Math.abs(dx), Math.abs(dy)].sort((a, b) => a - b) + const shape = SHAPES[axes.join(',')] + if (!shape) continue + const interveningPoints: string[] = [] + // Only straight jumps have a unique intervening corridor. Knight paths are not invented. + if (axes[0] === 0) { + for (let i = 1; i < axes[1]; i++) interveningPoints.push(name(anchor.x + Math.sign(dx) * i, anchor.y + Math.sign(dy) * i)) + } + const blockers = interveningPoints.filter((point) => board.has(point)) + const status = blockers.length ? 'blocked' : 'observed' + geometry.push({ kind: shape[0], label: shape[1], anchor: anchor.point, move: move.point, offset: { dx, dy }, status, + interveningPoints, blockers, + wording: status === 'blocked' ? `${move.point}与己棋${anchor.point}的坐标间隔对应${shape[1]}，但中间已有${blockers.join('、')}，不能按完整${shape[1]}讲解。` : `相对己棋${anchor.point}，${move.point}具有${shape[1]}的几何关系。` }) + } + geometry.sort((a, b) => a.anchor.localeCompare(b.anchor)) + const ownDevelopment = buildOwnDevelopment(input) + const developed = ownDevelopment.find(d => d.kind === 'own-development') + // Prefer a clear frontier jump only when all competing anchors belong to one real string. + const sameString = developed && geometry.filter(g => g.status === 'observed').every(g => developed.groupStones.includes(g.anchor)) + const primaryFriendlyGeometry = sameString ? geometry.find(g => g.anchor === developed.anchor && g.status === 'observed') : undefined + const relationCandidates = recognizeRelations(input, move, enemies, board, geometry) + if (developed) for (const candidate of relationCandidates) { + candidate.counterEvidence.push(`自身发展竞争解释：${developed.wording}；对敌形状不能单独确定整手主要目的。`) + candidate.purposeHypothesis.wording += ' 同时须比较附近己串的发展需求，不能仅凭该形状优先认定攻击或封锁对方。' + } + const relations: MoveConceptResult['relations'] = [] + const contacts = enemies.filter((p) => Math.abs(p.x - move.x) + Math.abs(p.y - move.y) === 1) + if (contacts.length) relations.push({ kind: 'enemy-contact', points: contacts.map((p) => p.point), evidence: `落点与敌棋${contacts.map((p) => p.point).join('、')}直接相邻；相邻本身不证明攻击或压迫。` }) + for (const enemy of enemies.filter((p) => distance(p, move) <= 2).sort((a, b) => a.point.localeCompare(b.point))) { + const offset = { dx: move.x - enemy.x, dy: move.y - enemy.y } + relations.push({ kind: 'enemy-position', points: [enemy.point], offset, + evidence: `${move.point}相对敌棋${enemy.point}的坐标差为dx=${offset.dx}、dy=${offset.dy}；这是一项位置事实，不能单凭间隔确定战术效果。` }) + } + const oppositeEnemies = enemies.filter((p) => distance(p, move) <= 2).filter((p) => enemies.some((other) => other.x === 2 * move.x - p.x && other.y === 2 * move.y - p.y)) + if (oppositeEnemies.length) relations.push({ kind: 'between-enemies', points: oppositeEnemies.map((p) => p.point), evidence: '落点位于对称的两侧敌棋之间；这不足以证明分断、夹击或可逃出。' }) + const limitations = ['几何关系不等于战术目的；气数和ownership均不能单独证明棋块死活。', '未验证本手合法性、自杀、劫争或后续变化。'] + const ownership = input.engineFeatures?.ownership + if (ownership) { + if (ownership.length === input.boardSize * input.boardSize && ownership.every((value) => Number.isFinite(value) && value >= -1 && value <= 1) && ['black', 'white'].includes(input.engineFeatures?.ownershipPerspective ?? '')) { + const value = ownership[(input.boardSize - 1 - move.y) * input.boardSize + move.x] + relations.push({ kind: 'ownership-at-move', points: [move.point], value, evidence: `${input.engineFeatures?.source || '外部引擎'}预测落点ownership=${value}（正值视角=${input.engineFeatures?.ownershipPerspective}）；只表示区域归属预测，不作为死活或意图判据。` }) + } else limitations.push('ownership尺寸、取值或视角不完整，已忽略。') + } + const observed = geometry.filter((g) => g.status === 'observed') + const intents: MoveIntentHypothesis[] = [] + // A side extension needs an open straight relation parallel to a nearby edge. + if (observed.length === 1 && observed[0].interveningPoints.length > 0) { + const anchor = parsePoint(observed[0].anchor, input.boardSize)! + const sideDepth = Math.min(3, Math.floor(input.boardSize / 4)) + const horizontalDepth = Math.min(move.y, input.boardSize - 1 - move.y) + const verticalDepth = Math.min(move.x, input.boardSize - 1 - move.x) + const parallelSide = (anchor.y === move.y && horizontalDepth <= sideDepth && edge(anchor, input.boardSize) === horizontalDepth && edge(move, input.boardSize) === horizontalDepth) || + (anchor.x === move.x && verticalDepth <= sideDepth && edge(anchor, input.boardSize) === verticalDepth && edge(move, input.boardSize) === verticalDepth) + const ux = Math.sign(move.x - anchor.x); const uy = Math.sign(move.y - anchor.y) + const length = distance(anchor, move) + const projection = (p: Point) => (p.x - anchor.x) * ux + (p.y - anchor.y) * uy + const inExpansionBand = (p: Point) => projection(p) >= 0 && projection(p) <= length + 1 && Math.abs((p.x - anchor.x) * uy - (p.y - anchor.y) * ux) <= 3 + const endpointContact = (p: Point) => Math.abs(p.x - anchor.x) + Math.abs(p.y - anchor.y) === 1 || Math.abs(p.x - move.x) + Math.abs(p.y - move.y) === 1 + const behind = enemies.filter((p) => projection(p) < 0 && distance(p, anchor) <= 3) + if (parallelSide && !enemies.some((p) => inExpansionBand(p) || endpointContact(p))) intents.push({ kind: 'side-extension', status: 'hypothesis', evidence: [`与唯一己棋锚点${anchor.point}沿边平行展开，中间${observed[0].interveningPoints.join('、')}未被占用。`, '展开区间及相邻三格带内没有敌棋，两个端点没有敌方直接接触。'], counterEvidence: ['尚未确认该棋块的根据地、势力方向或最佳拆边距离。', ...(behind.length ? [`锚点背侧仍有敌棋${behind.map((p) => p.point).join('、')}；其压力和后续侵消尚未验证。`] : [])], wording: '这手有沿边展开、争取区域的可能，可作为拆边候选理解；不能仅凭间隔确定是拆或已扩张成空。' }) + } + for (const assessment of input.groupAssessments ?? []) { + if (assessment.status !== 'unsettled' || !assessment.source.trim() || !assessment.evidence.length || !assessment.stones.length) continue + const targets = assessment.stones.map((raw) => parsePoint(raw, input.boardSize)) + if (targets.some((p) => !p || board.get(p.point) !== assessment.color)) { limitations.push('外部棋块评估与当前棋盘不一致，已忽略。'); continue } + const nearby = (targets as Point[]).filter((p) => distance(p, move) <= 2) + if (!nearby.length) continue + const provenance = `外部评估${assessment.source}标记${assessment.stones.join('、')}尚未安定：${assessment.evidence.join('；')}` + if (assessment.color === input.playerColor && nearby.some((p) => edge(move, input.boardSize) > edge(p, input.boardSize))) { + intents.push({ kind: 'escape', status: 'hypothesis', evidence: [provenance, '落点朝更远离边线的方向发展。'], counterEvidence: ['尚未验证逃出路线、连通性和敌方封锁。'], wording: '结合外部棋块评估，这手可能有向中腹出头的意图；是否属于有效逃出还需读后续。' }) + } else if (assessment.color !== input.playerColor) { + intents.push({ kind: 'attack', status: 'hypothesis', evidence: [provenance, '落点位于该敌棋附近。'], counterEvidence: ['靠近敌棋也可能是防守、连接或官子；尚未验证威胁。'], wording: '这手靠近外部评估尚未安定的敌棋，可能形成压力；攻击目的仍待后续验证。' }) + if (contacts.some((p) => assessment.stones.map((s) => s.trim().toUpperCase()).includes(p.point) && edge(move, input.boardSize) > edge(p, input.boardSize))) { + intents.push({ kind: 'press', status: 'hypothesis', evidence: [provenance, '与敌棋直接相邻，落点比敌棋更靠近中腹。'], counterEvidence: ['尚未验证能否限制敌棋向中腹发展，不能仅凭接触命名为压。'], wording: '这手在敌棋朝中腹的一侧接触，可能限制其出头；是否称为压仍需验证后续。' }) + } + } + } + const status = observed.length > 1 ? 'ambiguous' : geometry.length || relationCandidates.length || relations.length || intents.length ? 'observed' : 'abstained' + if (observed.length > 1) limitations.push('存在多个己棋锚点，保留全部相对几何关系，不按最近锚点给整手命名。') + const enemyFacts = relations.filter((r) => r.kind === 'enemy-contact' || r.kind === 'enemy-position') + const wording = developed ? `${primaryFriendlyGeometry?.wording ?? ''} ${developed.wording} ${relationCandidates.map(r => r.wording).join(' ')}`.trim() : relationCandidates.length ? relationCandidates.map((r) => r.wording).join(' ') : enemyFacts.length ? enemyFacts.map((r) => r.evidence).join(' ') : observed.length > 1 ? `这手与多个己棋存在不同或重复的几何关系（${observed.map((g) => `${g.anchor}：${g.label}`).join('；')}）；尚不能据此给整手确定术语。` : observed[0]?.wording || geometry[0]?.wording || relations[0]?.evidence || '现有证据不足以命名这手棋的棋形或意图。' + return { status, ownDevelopment, primaryFriendlyGeometry, geometry, relationCandidates, relations, intents, wording, limitations } +} + +/** Preserve competing anchors and counter-evidence when supplying this experiment to an LLM. */ +export function formatMoveConceptsForPrompt(result: MoveConceptResult): string { + return [ + '保守落子概念证据：几何只是相对关系；hypothesis不是确定术语，不可升级为定论。', + '先核对本手发展的己方真实棋串，再比较自身发展与对敌关系；形状事实不能独自确定主体目的。多锚点事实全部保留，独立棋串不能混为同串。candidate是术语候选，形状事实不证明目的或效果已经实现。', + `识别状态：${result.status}。${result.wording}`, + ...result.ownDevelopment.map(i => `自身发展[${i.kind}/${i.status}] 锚点=${i.anchor}，己串={${i.groupStones.join('、')}}：${i.wording}\n支持：${i.evidence.join('；')}\n反证/未验证：${i.counterEvidence.join('；')}`), + ...result.relationCandidates.map((r) => `术语[${r.kind}/${r.status}] ${r.enemyAnchor ? `敌棋=${r.enemyAnchor}` : `己方角部锚点=${r.friendlyAnchor}`}，己方支持=${r.supportingFriendPoints.join('、') || '无'}：${r.wording}\n形状[${r.geometryStatus}]：${r.evidence.join('；')}\n目的[${r.purposeHypothesis.status}]：${r.purposeHypothesis.wording}\n反证/未验证：${r.counterEvidence.join('；')}`), + ...result.geometry.map((g) => `几何[${g.status}] 锚点=${g.anchor}，落点=${g.move}，dx=${g.offset.dx}，dy=${g.offset.dy}：${g.wording}${g.blockers.length ? ` 阻挡=${g.blockers.join('、')}` : ''}`), + ...result.relations.map((r) => `关系[${r.kind}] ${r.evidence}`), + ...result.intents.map((i) => `意图[${i.kind}/${i.status}] ${i.wording}\n支持：${i.evidence.join('；')}\n反证/未验证：${i.counterEvidence.join('；')}`), + `限制：${result.limitations.join('；')}` + ].join('\n') +} diff --git a/src/main/services/knowledge/shapeRecognitionEngine.ts b/src/main/services/knowledge/shapeRecognitionEngine.ts index d501329..d8b83e7 100644 --- a/src/main/services/knowledge/shapeRecognitionEngine.ts +++ b/src/main/services/knowledge/shapeRecognitionEngine.ts @@ -87,20 +87,33 @@ function safeWording(confidence: ShapeRecognitionConfidence, counterEvidence: st } function fromLocalMatch(match: ReturnType<typeof findLocalPatternMatches>[number]): RecognizedShape { + const observations: Record<string, string> = { + cut_point: '局部存在需要检查连接与切断的几何关系；是否真的可断，需验证双方应手。', + empty_triangle: '局部命中空三角的几何模式；形状本身不能证明本手低效或错误。', + bamboo_joint: '局部命中竹节的几何模式；连接是否稳固仍需检查断点、气数及对方应手。', + false_eye_risk: '局部具备检查眼位的几何条件；需要核对对角、断点及周边棋块，尚不能确定是假眼。', + net: '局部具备检查枷的包围模式；尚未验证敌棋逃路，不能确定已经封住。', + throw_in_snapback: '局部具备检查扑入或倒扑的几何条件；必须验证扑、提、吃回的合法变化。', + semeai_liberty_race: '敌我棋块邻近且有局部空点，需要分别数气；邻近本身不能确定双方正在对杀。', + ko_shape: '局部具备检查劫形的模式；必须验证提子、提回条件和劫禁，尚不能确定形成劫。' + } + // Pattern scores are retrieval similarity, not tactical proof or calibrated confidence. + const confidence = match.confidence === 'strong' ? 'medium' : match.confidence + const counterEvidence = [...match.counterEvidence, '局部模式未验证合法后续，不能单独证明死活、手筋或本手目的。'] return { id: `local-pattern:${match.card.id}:${match.anchor}`, title: match.card.title, shapeType: match.card.shapeType, category: match.card.category, - confidence: match.confidence, + confidence, score: match.score, evidence: match.evidence, - counterEvidence: match.counterEvidence, - safeWording: safeWording(match.confidence, match.counterEvidence), + counterEvidence, + safeWording: '只作为训练类比', relatedMoves: [match.anchor], - recognition: match.card.teaching.recognition, - wrongThinking: match.card.teaching.wrongThinking, - correctThinking: match.card.teaching.correctThinking, + recognition: observations[match.card.shapeType] ?? `局部命中${match.card.title}的相似模式；这不是已验证的战术结论。`, + wrongThinking: '把局部模式相似度当成已证明的手筋、死活或这手棋的目的。', + correctThinking: `检查提示：${match.card.teaching.correctThinking} 本局面仍需独立验证，不能把题库答案直接套用。`, drillPrompt: match.card.teaching.drillPrompt, sourceRefs: match.card.sourceRefs, sourceQuality: match.card.sourceQuality @@ -126,13 +139,14 @@ function uniqueShapes(shapes: RecognizedShape[]): RecognizedShape[] { export function recognizeShapes(input: ShapeRecognitionInput): RecognizedShape[] { const anchors = [ input.playedMove, - ...(input.candidateMoves ?? []).slice(0, 6), - ...(input.principalVariation ?? []).slice(0, 6) + ...(input.candidateMoves ?? []).slice(0, 6) ] const localMatches = findLocalPatternMatches(loadShapePatternCards(), { boardSize: input.boardSize, boardSnapshot: input.boardSnapshot, localWindows: input.localWindows, + playedMove: input.playedMove, + candidateMoves: input.candidateMoves, anchors, playerColor: input.playerColor, phase: phase(input) @@ -174,6 +188,7 @@ export function recognizedShapesToKnowledgePackets(shapes: RecognizedShape[]): K selectedBody: [ `棋形识别: ${shape.recognition}`, `安全措辞: ${shape.safeWording}`, + '模式分数是检索排序，不是围棋语义准确率；未验证的战术模式只能作检查提示。', `识别依据: ${shape.evidence.join('；') || '无'}`, shape.counterEvidence.length ? `反证/降置信: ${shape.counterEvidence.join('；')}` : '', `常见误区: ${shape.wrongThinking}`, diff --git a/src/main/services/llm/codexAppServerClient.ts b/src/main/services/llm/codexAppServerClient.ts index 85e423e..f1683fb 100644 --- a/src/main/services/llm/codexAppServerClient.ts +++ b/src/main/services/llm/codexAppServerClient.ts @@ -14,6 +14,7 @@ import type { } from '@main/lib/types' import type { ChatMessage, ChatTool, ChatToolCall, ChatTurnResult } from './provider' import type { AgentRuntimeTurnInput, AgentToolExecutor } from './agentRuntime' +import { toolExecutionHistory } from './toolExecutionHistory' interface RpcResponse { id?: number | string @@ -428,23 +429,22 @@ export class CodexAppServerClient { arguments: JSON.stringify(params.arguments ?? {}) } } + let result: Awaited<ReturnType<AgentToolExecutor>> try { - const result = await context.execute(toolCall) - if (result.ok) context.executedTools.push(toolName) - context.followupMessages.push(...result.followupMessages) - await this.write({ - id: message.id, - result: { contentItems: contentItemsFromToolResult(result), success: result.ok } - }) + result = await context.execute(toolCall) } catch (error) { - await this.write({ - id: message.id, - result: { - contentItems: [{ type: 'inputText', text: `工具执行失败：${String(error)}` }], - success: false - } - }).catch(() => undefined) + result = { + ok: false, + toolResult: `工具执行失败：${String(error)}`, + followupMessages: [] + } } + if (result.ok) context.executedTools.push(toolName) + context.followupMessages.push(...toolExecutionHistory(toolCall, result)) + await this.write({ + id: message.id, + result: { contentItems: contentItemsFromToolResult(result), success: result.ok } + }).catch(() => undefined) } private write(message: unknown): Promise<void> { diff --git a/src/main/services/llm/toolExecutionHistory.ts b/src/main/services/llm/toolExecutionHistory.ts new file mode 100644 index 0000000..3d2d61d --- /dev/null +++ b/src/main/services/llm/toolExecutionHistory.ts @@ -0,0 +1,19 @@ +import type { AgentToolExecutionResult } from './agentRuntime' +import type { ChatMessage, ChatToolCall } from './provider' + +/** Preserve the evidence from an executed tool for subsequent repair turns. */ +export function toolExecutionHistory( + toolCall: ChatToolCall, + result: AgentToolExecutionResult +): ChatMessage[] { + return [ + { role: 'assistant', content: '', tool_calls: [toolCall] }, + { + role: 'tool', + name: toolCall.function.name, + tool_call_id: toolCall.id, + content: result.toolResult + }, + ...result.followupMessages + ] +} diff --git a/src/main/services/teacher/analysisReuse.ts b/src/main/services/teacher/analysisReuse.ts new file mode 100644 index 0000000..65ef0cc --- /dev/null +++ b/src/main/services/teacher/analysisReuse.ts @@ -0,0 +1,17 @@ +import type { KataGoMoveAnalysis } from '@main/lib/types' + +/** Forced-played searches do not establish the depth of the unrestricted root search. */ +export function canReuseTeacherAnalysis(analysis: KataGoMoveAnalysis | undefined, request: { + gameId: string + moveNumber: number + trialBranchHash?: string + requestedMaxVisits?: number +}): boolean { + if (!analysis || analysis.gameId !== request.gameId || analysis.moveNumber !== request.moveNumber || + (analysis.trialContext?.active ? analysis.trialContext.branchHash : undefined) !== request.trialBranchHash) return false + if (request.requestedMaxVisits === undefined) return true + if (!Number.isFinite(request.requestedMaxVisits) || request.requestedMaxVisits < 1) return false + const rootVisits = analysis.analysisQuality?.totalVisits + // KataGo may report one less completed root visit than its configured budget. + return typeof rootVisits === 'number' && Number.isFinite(rootVisits) && rootVisits >= 0 && rootVisits + 1 >= request.requestedMaxVisits +} diff --git a/src/main/services/teacher/boardGroupRelations.ts b/src/main/services/teacher/boardGroupRelations.ts new file mode 100644 index 0000000..d608b66 --- /dev/null +++ b/src/main/services/teacher/boardGroupRelations.ts @@ -0,0 +1,235 @@ +import { buildOwnDevelopment, type OwnDevelopmentCandidate } from './ownDevelopment' +/** Board facts and explainable regional intentions, never tactical or life/death verdicts. */ +type Color = 'B' | 'W' +type Stone = { point: string; color: Color } +type Point = { point: string; x: number; y: number } +export interface ConnectedBoardGroup { + id: string + color: Color + stones: string[] + bounds: { minX: number; maxX: number; minY: number; maxY: number } + emptyAdjacentPoints: string[] +} +export interface PotentialBoardLink { + color: Color + groups: [string, string] + endpoints: [string, string] + kind: 'diagonal' | 'jump' | 'knight' + emptyPath: string[] + status: 'potential-not-connected' +} +export interface RegionalIntent { + kind: 'contest-head' | 'separate-contact' + status: 'hypothesis' + targetGroups: string[] + targetStones: string[] + friendlySupport: string[] + evidence: string[] + limitations: string[] + wording: string +} +export interface BoardGroupRelations { + status: 'observed' | 'abstained' + positionFacts: { + timing: 'before-move' + stonesByColor: { B: string[]; W: string[] } + connectedGroups: ConnectedBoardGroup[] + potentialLinks: PotentialBoardLink[] + } + ownDevelopment: OwnDevelopmentCandidate[] + regionIntents: RegionalIntent[] + prompt: string + limitations: string[] +} +const COLUMNS = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +const LIMITATIONS = ['正交连通串不等于战略整龙；潜在联系不等于已经连接。', '意图候选只描述可核对的方向和配置，不证明分割、封锁已经成功，不判断死活或唯一逃路。'] +function parse(raw: unknown, size: number): Point | undefined { + if (typeof raw !== 'string') return undefined + const point = raw.trim().toUpperCase(); const match = /^([A-HJ-Z])([1-9]\d?)$/.exec(point) + if (!match) return undefined + const x = COLUMNS.indexOf(match[1]); const y = Number(match[2]) - 1 + return x >= 0 && x < size && y >= 0 && y < size ? { point, x, y } : undefined +} +const name = (x: number, y: number) => `${COLUMNS[x]}${y + 1}` +const distance = (a: Point, b: Point) => Math.max(Math.abs(a.x - b.x), Math.abs(a.y - b.y)) +const compare = (a: Point, b: Point) => a.y - b.y || a.x - b.x +function neighbors(p: Point, size: number): Point[] { + return [[1, 0], [-1, 0], [0, 1], [0, -1]].map(([dx, dy]) => ({ x: p.x + dx, y: p.y + dy, point: name(p.x + dx, p.y + dy) })) + .filter((q) => q.x >= 0 && q.x < size && q.y >= 0 && q.y < size) +} +function abstain(reason: string): BoardGroupRelations { + return { status: 'abstained', positionFacts: { timing: 'before-move', stonesByColor: { B: [], W: [] }, connectedGroups: [], potentialLinks: [] }, ownDevelopment: [], regionIntents: [], limitations: [...LIMITATIONS, reason], prompt: `棋块/区域证据弃权：${reason}` } +} +/** A single monotone shortest empty path is an observed route, not a guaranteed connection. */ +function emptyPath(a: Point, b: Point, board: Map<string, Color>): string[] | undefined { + const visit = (p: Point, path: string[]): string[] | undefined => { + if (p.point === b.point) return path + for (const [dx, dy] of [[Math.sign(b.x - p.x), 0], [0, Math.sign(b.y - p.y)]]) { + if (!dx && !dy) continue + const q = { x: p.x + dx, y: p.y + dy, point: name(p.x + dx, p.y + dy) } + if (q.point !== b.point && board.has(q.point)) continue + const result = visit(q, q.point === b.point ? path : [...path, q.point]) + if (result) return result + } + return undefined + } + return visit(a, []) +} +function linkKind(a: Point, b: Point): PotentialBoardLink['kind'] | undefined { + const [low, high] = [Math.abs(a.x - b.x), Math.abs(a.y - b.y)].sort((x, y) => x - y) + if (low === 1 && high === 1) return 'diagonal' + if (low === 0 && (high === 2 || high === 3)) return 'jump' + if (low === 1 && (high === 2 || high === 3)) return 'knight' + return undefined +} + +export function buildBoardGroupRelations(input: { boardSize: number; boardSnapshot?: Stone[]; playerColor: Color; playedMove: string }): BoardGroupRelations { + const size = input.boardSize + if (!Number.isInteger(size) || size < 2 || size > COLUMNS.length) return abstain('棋盘尺寸无效。') + if (!Array.isArray(input.boardSnapshot)) return abstain('缺少完整落子前棋盘。') + if (input.playerColor !== 'B' && input.playerColor !== 'W') return abstain('实际落子颜色无效。') + const move = parse(input.playedMove, size) + if (!move) return abstain('落点无效或为停着。') + const board = new Map<string, Color>(); const points = new Map<string, Point>() + for (const stone of input.boardSnapshot) { + const p = parse(stone?.point, size) + if (!p || (stone.color !== 'B' && stone.color !== 'W')) return abstain('棋盘包含无效棋子。') + if (board.has(p.point)) return abstain('棋盘包含重复或冲突落点。') + board.set(p.point, stone.color); points.set(p.point, p) + } + if (board.has(move.point)) return abstain('落点已被占用；需要落子前棋盘。') + const ordered = [...points.values()].sort(compare) + const stonesByColor: { B: string[]; W: string[] } = { B: [], W: [] } + for (const p of ordered) stonesByColor[board.get(p.point)!].push(p.point) + const connectedGroups: ConnectedBoardGroup[] = []; const visited = new Set<string>() + for (const first of ordered) { + if (visited.has(first.point)) continue + const color = board.get(first.point)!; const pending = [first]; const group: Point[] = []; const empty = new Set<string>() + visited.add(first.point) + while (pending.length) { + const p = pending.pop()!; group.push(p) + for (const q of neighbors(p, size)) { + if (!board.has(q.point)) empty.add(q.point) + else if (board.get(q.point) === color && !visited.has(q.point)) { visited.add(q.point); pending.push(q) } + } + } + group.sort(compare) + connectedGroups.push({ id: `${color}:${group[0].point}`, color, stones: group.map((p) => p.point), + bounds: { minX: Math.min(...group.map((p) => p.x)), maxX: Math.max(...group.map((p) => p.x)), minY: Math.min(...group.map((p) => p.y)), maxY: Math.max(...group.map((p) => p.y)) }, + emptyAdjacentPoints: [...empty].sort((a, b) => compare(points.get(a) ?? parse(a, size)!, points.get(b) ?? parse(b, size)!)) }) + } + const potentialLinks: PotentialBoardLink[] = [] + for (let i = 0; i < connectedGroups.length; i++) for (let j = i + 1; j < connectedGroups.length; j++) { + const a = connectedGroups[i]; const b = connectedGroups[j] + if (a.color !== b.color) continue + // Preserve one shortest representative per pair; never merge the real groups. + const candidates = a.stones.flatMap((x) => b.stones.map((y) => ({ a: points.get(x)!, b: points.get(y)! }))) + .filter(({ a, b }) => linkKind(a, b)).sort((x, y) => distance(x.a, x.b) - distance(y.a, y.b) || compare(x.a, y.a) || compare(x.b, y.b)) + for (const pair of candidates) { + const path = emptyPath(pair.a, pair.b, board) + if (!path) continue + potentialLinks.push({ color: a.color, groups: [a.id, b.id], endpoints: [pair.a.point, pair.b.point], kind: linkKind(pair.a, pair.b)!, emptyPath: path, status: 'potential-not-connected' }); break + } + } + const ownDevelopment = buildOwnDevelopment(input) + const enemy: Color = input.playerColor === 'B' ? 'W' : 'B' + const enemyGroups = connectedGroups.filter((g) => g.color === enemy) + const friendlyPoints = stonesByColor[input.playerColor].map((p) => points.get(p)!) + const support = friendlyPoints.filter((p) => distance(p, move) <= 4 && + (Math.abs(p.x - move.x) + Math.abs(p.y - move.y) === 1 || linkKind(p, move)) && emptyPath(p, move, board)) + const regionIntents: RegionalIntent[] = [] + const groupById = new Map(connectedGroups.map((g) => [g.id, g])) + // A regional index is deliberately narrower than the full analogy graph. + // Only close diagonal / one-space jumps participate; weak knights and wider + // jumps remain context and cannot transitively turn half a board into a target. + const strongRegionalLinks = potentialLinks.filter((link) => { + const a = points.get(link.endpoints[0])!; const b = points.get(link.endpoints[1])! + return link.kind === 'diagonal' || (link.kind === 'jump' && distance(a, b) === 2) + }) + const localEnemyGroups = enemyGroups.filter((group) => group.stones.some((stone) => distance(points.get(stone)!, move) <= 6)) + const localGroupIds = new Set(localEnemyGroups.map((group) => group.id)) + const clusterVisited = new Set<string>() + for (const seed of localEnemyGroups) { + if (clusterVisited.has(seed.id)) continue + const ids = new Set([seed.id]); const pending = [seed.id]; clusterVisited.add(seed.id) + while (pending.length) { + const id = pending.pop()! + for (const link of strongRegionalLinks.filter((l) => l.color === enemy && l.groups.includes(id))) { + const other = link.groups.find((x) => x !== id)! + if (localGroupIds.has(other) && !ids.has(other)) { ids.add(other); pending.push(other); clusterVisited.add(other) } + } + } + const targets = [...ids].flatMap((id) => groupById.get(id)!.stones).map((p) => points.get(p)!) + if (targets.length < 2) continue + const edges = [ + { label: '左边', depth: Math.min(...targets.map((p) => p.x)), nx: 1, ny: 0 }, + { label: '右边', depth: Math.min(...targets.map((p) => size - 1 - p.x)), nx: -1, ny: 0 }, + { label: '下边', depth: Math.min(...targets.map((p) => p.y)), nx: 0, ny: 1 }, + { label: '上边', depth: Math.min(...targets.map((p) => size - 1 - p.y)), nx: 0, ny: -1 } + ].sort((a, b) => a.depth - b.depth) + if (edges[0].depth > 3 || edges[0].depth === edges[1].depth) continue + const frame = edges[0]; const normal = (p: Point) => p.x * frame.nx + p.y * frame.ny + const tangent = (p: Point) => p.x * frame.ny - p.y * frame.nx + const front = Math.max(...targets.map(normal)); const advance = normal(move) - front + const frontTargets = targets.filter((p) => normal(p) === front) + const nearbyFront = frontTargets.find((p) => distance(p, move) <= 3 && Math.abs(tangent(p) - tangent(move)) <= 2 && emptyPath(p, move, board)) + if (!nearbyFront || advance < 1 || advance > 3) continue + const frontSupports = support.filter((p) => normal(p) >= front - 1) + const left = frontSupports.filter((p) => tangent(p) < tangent(nearbyFront)); const right = frontSupports.filter((p) => tangent(p) > tangent(nearbyFront)) + if (!left.length || !right.length) continue + const supports = [...left, ...right].map((p) => p.point) + regionIntents.push({ kind: 'contest-head', status: 'hypothesis', targetGroups: [...ids], targetStones: targets.sort(compare).map((p) => p.point), friendlySupport: supports, + evidence: [`敌方这一片棋最靠近${frame.label}，其向中腹前沿包括${frontTargets.map((p) => p.point).join('、')}。`, `${move.point}位于该前沿朝中腹方向${advance}格，近前沿${nearbyFront.point}；两侧己棋${supports.join('、')}有近距离开放几何支持。`, `前沿到落点存在空路径${emptyPath(nearbyFront, move, board)!.join('、')}，仅为一条可见路线。`], + limitations: [...LIMITATIONS, '区域索引只纳入有棋子距落点六格内的真实串，以尖/空一间跳作强邻接；其它弱联系仅作上下文。这不是已确认整条龙，对手其它出头和联络路线仍需验证。'], + wording: `${move.point}结合两侧己棋配置，可能争夺敌方${targets.map((p) => p.point).join('、')}这一片棋向中腹的头部位置，带有限制出头/封锁的意图；实际效果待验证。` }) + } + // Wider regional contact, not restricted to an exact symmetric pair of stones. + const separations: Array<RegionalIntent & { rank: number; directionKey: string }> = [] + if (support.length) for (let i = 0; i < enemyGroups.length; i++) for (let j = i + 1; j < enemyGroups.length; j++) { + const a = enemyGroups[i]; const b = enemyGroups[j] + if (potentialLinks.some((l) => l.groups.includes(a.id) && l.groups.includes(b.id))) continue + const pairs = a.stones.flatMap((x) => b.stones.map((y) => ({ a: points.get(x)!, b: points.get(y)! }))) + for (const pair of pairs) { + const da = distance(pair.a, move); const db = distance(pair.b, move); const span = distance(pair.a, pair.b) + if (Math.min(da, db) > 2 || Math.max(da, db) > 7 || span < 3 || span > 8) continue + const dx = pair.b.x - pair.a.x; const dy = pair.b.y - pair.a.y; const length2 = dx * dx + dy * dy + const t = ((move.x - pair.a.x) * dx + (move.y - pair.a.y) * dy) / length2 + const off = Math.abs((move.x - pair.a.x) * dy - (move.y - pair.a.y) * dx) / Math.sqrt(length2) + if (t < 0.12 || t > 0.88 || off > 1.2) continue + if (!emptyPath(pair.a, move, board) || !emptyPath(move, pair.b, board)) continue + const near = da <= db ? pair.a : pair.b; const far = da <= db ? pair.b : pair.a + const nearGroup = da <= db ? a.id : b.id + separations.push({ kind: 'separate-contact', status: 'hypothesis', targetGroups: [a.id, b.id], targetStones: [...a.stones, ...b.stones], friendlySupport: support.map((p) => p.point), rank: da + db + off, + directionKey: `${nearGroup}:${Math.sign(far.x - near.x)},${Math.sign(far.y - near.y)}`, + evidence: [`敌棋${pair.a.point}与${pair.b.point}属于不同正交连通串。`, `${move.point}处于两串代表点的联络方向之间，投影比例${t.toFixed(2)}、偏离约${off.toFixed(2)}格；附近己棋${support.map((p) => p.point).join('、')}有开放几何支持。`, '双方代表点到落点各有空路径；这是区域方向证据，不是原来已连接或必然可连接的证明。'], + limitations: [...LIMITATIONS, '两区域仍可能经其它方向联络；当前几何不能验证分断，也没有证明联络本来是其最佳计划。'], + wording: `${move.point}位于敌棋${pair.a.point}与${pair.b.point}这一侧区域的联络方向之间，可能有分割两片棋联系的意图；是否有效需检查敌方应对。` }) + break + } + } + const representedDirections = new Set<string>() + for (const { rank: _rank, directionKey, ...intent } of separations.sort((a, b) => a.rank - b.rank)) { + if (regionIntents.length >= 3) break + if (representedDirections.has(directionKey)) continue + representedDirections.add(directionKey) + regionIntents.push(intent) + } + const limitedIntents = regionIntents.slice(0, 3) + // Full coordinate facts remain structured and complete; prompt focuses on useful groups. + const related = connectedGroups.filter((g) => g.stones.some((p) => distance(points.get(p)!, move) <= 7)) + const other = connectedGroups.filter((g) => !related.includes(g)) + const lines = ['棋块/区域证据（来自完整落子前坐标，非图片猜测）；工具文本可能截断尾部图明细，不能把收到的图当作穷尽。', ...LIMITATIONS, + `实际${input.playerColor}落点=${move.point}。`, + ...ownDevelopment.map((i) => `自身发展[${i.kind}/hypothesis] ${i.wording}\n支持事实：${i.evidence.join('；')}\n反证/未验证：${i.counterEvidence.join('；')}`), + ...limitedIntents.map((i) => `区域意图[${i.kind}/hypothesis] ${i.wording}\n支持事实：${i.evidence.join('；')}\n未验证：${i.limitations.join('；')}`), + `相关正交串：${related.map((g) => `${g.id}={${g.stones.join('、')}}`).join('；') || '无'}`, + `相关潜在联系（不合并串）：${potentialLinks.filter((l) => related.some((g) => l.groups.includes(g.id))).map((l) => `${l.endpoints.join('↔')} ${l.kind}，一条空路径=${l.emptyPath.join('、')}`).join('；') || '无'}`, + `其它区域范围：${other.map((g) => `${g.id} ${name(g.bounds.minX, g.bounds.minY)}—${name(g.bounds.maxX, g.bounds.maxY)}(${g.stones.length}子)`).join('；') || '无'}`] + // Omit verbose graph context first. Hypotheses/facts are never fabricated to fill the budget. + let prompt = lines.join('\n') + if (prompt.length > 6000) prompt = [...lines.slice(0, 4 + ownDevelopment.length + limitedIntents.length), `相关串范围：${related.map((g) => `${g.id} ${g.stones.length}子 ${name(g.bounds.minX, g.bounds.minY)}—${name(g.bounds.maxX, g.bounds.maxY)}`).join('；')}`, '详细坐标与潜在联系保留于positionFacts，摘要未穷尽。'].join('\n').slice(0, 6000) + // Preserve the actionable summary and source coordinate list before verbose + // graph details when normal tool serialization imposes a text-size limit. + return { status: 'observed', ownDevelopment, regionIntents: limitedIntents, prompt, limitations: [...LIMITATIONS], + positionFacts: { timing: 'before-move', stonesByColor, connectedGroups, potentialLinks } } +} diff --git a/src/main/services/teacher/currentMoveResponsePolicy.ts b/src/main/services/teacher/currentMoveResponsePolicy.ts new file mode 100644 index 0000000..a14b044 --- /dev/null +++ b/src/main/services/teacher/currentMoveResponsePolicy.ts @@ -0,0 +1,28 @@ +interface CurrentMoveResponsePolicyInput { + intent: string + prompt?: string + explanationPace?: string +} + +export interface CurrentMoveResponsePolicy { + mode: 'concise' | 'detailed' + instruction: string +} + +const SHORT_REQUEST = /一句话|一两句话|简短|简洁|简要|不用展开|不必展开|不要展开|不需要展开|(?:不用|不必|不要|无需|不需要).{0,8}(?:详细|详解|细讲|展开|深挖|逐步|列变化)|no need\s*(?:详细|详解|展开)|\b(?:brief(?:ly)?|concise(?:ly)?|one sentence|no need(?: to)?(?: explain)?(?: in)? detail|(?:do not|don't) explain in detail|(?:do not|don't|no need to) (?:elaborate|expand))\b/i +const DETAIL_REQUEST = /详细|详解|细讲|展开|深挖|逐步|列(?:出)?变化|后续变化|后续怎么下|对方(?:怎么|如何)(?:应|下)|对方应手|应对变化|explain in detail|show (?:the )?variations|(?:opponent|opponent's) (?:reply|response)|what (?:happens|comes) next/i + +/** Evidence depth and teacher style do not determine the length of a move comment. */ +export function buildCurrentMoveResponsePolicy(input: CurrentMoveResponsePolicyInput): CurrentMoveResponsePolicy | undefined { + if (input.intent !== 'current-move') return undefined + const prompt = input.prompt ?? '' + const concise = SHORT_REQUEST.test(prompt) + const detailed = !concise && (DETAIL_REQUEST.test(prompt) || input.explanationPace === 'detailed') + const grounding = '内部仍须完整核对棋盘、KataGo、全局重点和棋块关系。先判断本手主要服务于哪片棋：己方出头、防被封锁、沿边展开与对敌攻击是竞争目的，不能因为落点在敌棋上方就默认讲限制对方出头。同一真实己串同时存在跳和飞关系时，应检查发展方向的前端锚点，不能按锚点名字或任选背侧棋子给整手命名。粘劫、连接和收官不自动等于弱棋补强；没有来源明确的未安定评估，不把气少、提单子或附近有敌棋当成弱龙证据。把“目的是封锁/试图分割”与“已经封锁成功/已经分断”分清；只有证据支持整片敌棋时才称“大龙”，不能凭局部串或棋形自动命名。若确需指出远处推荐，应尽量用已核验的棋盘效果或同一分支参考变化简短解释其用途，而不只报坐标；参考线不是强制应手，也不能把全局目差全算在一颗棋子的价值上。' + return { + mode: detailed ? 'detailed' : 'concise', + instruction: detailed + ? `【当前手讲解详略：按需详解】用户要求详细讲解或具体应手、后续变化时，围绕所问问题展开；先给本手的主要目的，再补必要的依据或已核验变化，不固定追加训练建议和无关栏目。${grounding}` + : `【当前手讲解详略：一句话】普通手严格只写一句，中文约20–60字；以“这手……”直接说棋形与主要目的，例如在证据支持时说“这手飞目的是封锁对方大龙”。内部比较多个目的后，输出只选证据最充分的主目的，不罗列次要棋形、空路线潜力或“尚未分断”等无关保留判断。AI推荐不同但差距不大时，省略推荐点和选择评价，不要先报AI推荐再讲目的。同一落子前比较若 significantGap 与 reliableSearch 均为 true，应在本句简短指出具体本手不佳、首选及已核验用途；这一点不要求首选距离远，也不等于整个局部不重要。只有全局证据显示远处首选明显更好、且没有接近首选的局部替代时，才最多补第二句关键提醒；仅距离远不能触发第二句。不要固定展开棋盘描述、坐标清单、胜率目差、PV、对方应手、训练建议或免责声明；不确定意图可在本句用“意在/可能”，无需追加一段保留判断。用户明确追问时再展开。teachingDensity、严谨风格和中盘复杂度只影响内部核验，不得覆盖这条篇幅约束。“为什么这手”只问目的时也保持简短。${grounding}` + } +} diff --git a/src/main/services/teacher/evidenceToolGate.ts b/src/main/services/teacher/evidenceToolGate.ts new file mode 100644 index 0000000..891957b --- /dev/null +++ b/src/main/services/teacher/evidenceToolGate.ts @@ -0,0 +1,36 @@ +import type { TeacherRunRequest, VisionEvidenceReport } from '@main/lib/types' +import type { ChatMessage } from '../llm/provider' +import { buildVisionEvidenceReport, validateVisionEvidenceForIntent } from './visionEvidence' + +/** Only a validated current-position image actually sent in the initial user message replaces a capture call. */ +export function hasInitialCurrentBoardImage(request: TeacherRunRequest, report: VisionEvidenceReport | undefined, message: ChatMessage): boolean { + if (request.mode !== 'current-move' || !request.gameId || !Number.isInteger(request.moveNumber) || request.moveNumber! < 1 || + !request.boardImageDataUrl || !report?.attached || report.source !== 'initial-attachment' || + !validateVisionEvidenceForIntent(report, 'current-move').ok || message.role !== 'user' || !Array.isArray(message.content)) return false + if (request.boardContext === 'trial' && (!request.trialBranch?.active || + request.moveNumber !== request.trialBranch.baseMoveNumber + request.trialBranch.moves.length)) return false + // Recompute bytes, MIME and dimensions from this request; an attached/valid flag alone is insufficient. + const actual = buildVisionEvidenceReport(request, 'current-move') + const image = actual.images.find(image => image.role === 'current-board' && image.moveNumber === request.moveNumber && image.index === 0) + const declared = report.images.find(image => image.role === 'current-board' && image.moveNumber === request.moveNumber && image.index === 0) + return Boolean(image?.valid && image.width && image.height && declared?.valid && declared.source === 'initial-attachment' && + declared.bytes === image.bytes && declared.mimeType === image.mimeType && + message.content.some(part => part.type === 'image_url' && part.image_url.url === request.boardImageDataUrl)) +} + +export function missingTeacherEvidenceTools(input: { + intent: string; request: TeacherRunRequest; report?: VisionEvidenceReport + initialMessage: ChatMessage; successfulTools: Set<string> +}): string[] { + const groups: Record<string, string[][]> = { + 'current-move': [ + ...(hasInitialCurrentBoardImage(input.request, input.report, input.initialMessage) ? [] : [['board_captureTeachingImage']]), + ['katago_analyzePosition'], ['knowledge_matchPosition', 'knowledge_searchLocal'] + ], + 'game-review': [['sgf_readGameRecord'], ['katago_analyzeGameBatch'], ['board_captureTeachingImage'], + ['knowledge_matchPosition', 'knowledge_searchLocal', 'knowledge_searchJoseki', 'knowledge_searchLifeDeath', 'knowledge_searchTesuji']], + 'move-range': [['katago_analyzeMoveRangeKeyMoves'], ['board_captureTeachingImage'], + ['knowledge_matchPosition', 'knowledge_searchLocal', 'knowledge_searchJoseki', 'knowledge_searchLifeDeath', 'knowledge_searchTesuji']] + } + return (groups[input.intent] ?? []).filter(group => !group.some(name => input.successfulTools.has(name))).map(group => group.join(' / ')) +} diff --git a/src/main/services/teacher/globalPriorityEvidence.ts b/src/main/services/teacher/globalPriorityEvidence.ts new file mode 100644 index 0000000..75c77e7 --- /dev/null +++ b/src/main/services/teacher/globalPriorityEvidence.ts @@ -0,0 +1,180 @@ +import type { KataGoMoveAnalysis } from '@main/lib/types' + +export interface GlobalPriorityInput { + gameId?: string + moveNumber: number + boardSize: number + boardTiming: 'before-move' + boardSnapshot?: Array<{ point: string; color: 'B' | 'W' }> + playedMove: string + playerColor: 'B' | 'W' + /** Pass the expected active branch hash; omission expects the main line. */ + trialBranchHash?: string + analysis?: KataGoMoveAnalysis +} + +export interface GlobalPriorityEvidence { + status: 'compared' | 'unavailable' + narrativePriority: 'global-priority' | 'local-or-undetermined' + boardTiming: 'before-move' + comparison?: { + bestMove: string + playedMove: string + playerColor: 'B' | 'W' + source: 'before-root-candidates' | 'before-root-and-forced-played' + valuePerspective: 'current-player' + distanceMetric: 'chebyshev' + distance: number + distantThreshold: number + bestWinrate: number + playedWinrate: number + winrateGapPercentagePoints: number + bestScoreLead: number + playedScoreLead: number + scoreGap: number + bestVisits: number + playedVisits: number + totalVisits: number + rootVisitSource: 'analysis-quality' | 'unavailable' + significantGap: boolean + reliableSearch: boolean + winrateSaturated: boolean + } + bestLocalAlternative?: { + move: string + source: 'before-root-candidate' + distanceFromPlayed: number + visits: number + winrate: number + scoreLead: number + scoreGapFromBest: number + winrateGapFromBestPercentagePoints: number + winrateSaturated: boolean + nearBest: boolean + } + warnings: string[] + prompt: string +} + +const columns = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +function coordinate(value: string, size: number): { x: number; y: number; gtp: string } | undefined { + const match = /^([A-HJ-Z])([1-9]\d*)$/.exec(value.trim().toUpperCase()) + if (!match) return undefined + const x = columns.indexOf(match[1]) + const y = Number(match[2]) - 1 + return x >= 0 && x < size && y >= 0 && y < size ? { x, y, gtp: `${match[1]}${y + 1}` } : undefined +} + +/** Compare alternative root choices, never the before/after root evaluations. */ +export function buildGlobalPriorityEvidence(input: GlobalPriorityInput): GlobalPriorityEvidence { + const unavailable = (reason: string): GlobalPriorityEvidence => ({ + status: 'unavailable', narrativePriority: 'local-or-undetermined', boardTiming: 'before-move', + warnings: [reason], prompt: `全局优先级证据不足：${reason} 不可把落子前后估值差当作同局面候选损失。` + }) + const analysis = input.analysis + if (input.boardTiming !== 'before-move' || !Number.isInteger(input.boardSize) || input.boardSize < 2 || input.boardSize > 25 || + !Number.isInteger(input.moveNumber) || input.moveNumber < 1 || !analysis || analysis.moveNumber !== input.moveNumber || + analysis.boardSize !== input.boardSize || (input.gameId && analysis.gameId !== input.gameId) || + (analysis.trialContext?.active ? analysis.trialContext.branchHash : undefined) !== input.trialBranchHash) { + return unavailable('分析的棋谱、手数、棋盘大小、分支或落子时点不匹配。') + } + const played = coordinate(input.playedMove, input.boardSize) + const current = analysis.currentMove + if (!played || !current || current.pass || current.color !== input.playerColor || + current.moveNumber !== input.moveNumber || coordinate(current.gtp, input.boardSize)?.gtp !== played.gtp) { + return unavailable('当前落点或颜色身份无效（停一手不能用于空间距离比较）。') + } + if (!input.boardSnapshot) return unavailable('缺少落子前棋盘，无法核对落点是否为空。') + const occupied = new Set<string>() + for (const stone of input.boardSnapshot) { + const point = coordinate(stone.point, input.boardSize) + if (!point || !['B', 'W'].includes(stone.color) || occupied.has(point.gtp)) return unavailable('落子前棋盘坐标或棋子数据无效。') + occupied.add(point.gtp) + } + const topMoves = analysis.before.topMoves + // Preserve the engine's root ranking; sorting by score would manufacture a different recommendation. + const best = topMoves[0] + const bestPoint = best && coordinate(best.move, input.boardSize) + if (!bestPoint || occupied.has(bestPoint.gtp) || occupied.has(played.gtp)) return unavailable('首选或实战坐标无效、停一手或已占据。') + const rootPlayed = topMoves.find((candidate) => coordinate(candidate.move, input.boardSize)?.gtp === played.gtp) + const matchingPlayedValue = coordinate(analysis.playedMove?.move ?? '', input.boardSize)?.gtp === played.gtp ? analysis.playedMove : undefined + if (matchingPlayedValue?.source === 'after-root') return unavailable('实战估值来源是 after-root，与展示候选来源冲突，不能用于同局面比较。') + const forced = matchingPlayedValue?.source === 'forced' ? matchingPlayedValue : undefined + // Forced queries are merged into display topMoves. The explicit provenance must + // survive that merge; merely finding the coordinate in the array is not proof + // that the value was produced by the ordinary unrestricted root search. + const actual = forced ?? rootPlayed + if (!actual) return unavailable('缺少同一落子前局面的实战候选估值；after-root 不能替代。') + for (const value of [best, actual]) { + if (!Number.isFinite(value.winrate) || value.winrate < 0 || value.winrate > 100 || !Number.isFinite(value.scoreLead) || + !Number.isFinite(value.visits) || (value.visits ?? 0) < 0) return unavailable('候选估值或搜索次数无效。') + } + const sign = input.playerColor === 'B' ? 1 : -1 + const perspectiveWinrate = (value: number): number => sign === 1 ? value : 100 - value + const bestWinrate = perspectiveWinrate(best.winrate) + const playedWinrate = perspectiveWinrate(actual.winrate) + const scoreGap = sign * (best.scoreLead - actual.scoreLead) + const winrateGap = bestWinrate - playedWinrate + const distance = Math.max(Math.abs(bestPoint.x - played.x), Math.abs(bestPoint.y - played.y)) + const distantThreshold = Math.max(4, Math.ceil(input.boardSize / 3)) + const winrateSaturated = [bestWinrate, playedWinrate].some((value) => value <= 2 || value >= 98) + const significantGap = scoreGap >= 3 || (!winrateSaturated && winrateGap >= 8) + const localCandidates = topMoves.flatMap((candidate) => { + const point = coordinate(candidate.move, input.boardSize) + if (!point || point.gtp === played.gtp || occupied.has(point.gtp) || !Number.isFinite(candidate.visits) || candidate.visits < 32 || + !Number.isFinite(candidate.winrate) || candidate.winrate < 0 || candidate.winrate > 100 || !Number.isFinite(candidate.scoreLead)) return [] + const distanceFromPlayed = Math.max(Math.abs(point.x - played.x), Math.abs(point.y - played.y)) + if (distanceFromPlayed >= distantThreshold) return [] + const winrate = perspectiveWinrate(candidate.winrate) + const scoreLead = sign * candidate.scoreLead + const scoreGapFromBest = sign * best.scoreLead - scoreLead + const winrateGapFromBestPercentagePoints = bestWinrate - winrate + const saturated = [bestWinrate, winrate].some((value) => value <= 2 || value >= 98) + return [{ move: point.gtp, source: 'before-root-candidate' as const, distanceFromPlayed, visits: candidate.visits, winrate, scoreLead, + scoreGapFromBest, winrateGapFromBestPercentagePoints, + winrateSaturated: saturated, + nearBest: scoreGapFromBest <= 1 || (!saturated && winrateGapFromBestPercentagePoints <= 2) }] + }) + // A single strong alternative in the same area defeats a region-wide inference + // from the poor played point. Otherwise preserve the engine's root ordering. + const bestLocalAlternative = localCandidates.find((candidate) => candidate.nearBest) ?? localCandidates[0] + // before.topMoves may include a separately forced played query with many visits. + // Summing its display rows would incorrectly upgrade a shallow root search. + const reportedRootVisits = analysis.analysisQuality?.totalVisits + const hasRootVisits = typeof reportedRootVisits === 'number' && Number.isFinite(reportedRootVisits) && reportedRootVisits >= 0 + const totalVisits = hasRootVisits ? reportedRootVisits : 0 + const bestVisits = best.visits + const playedVisits = actual.visits ?? 0 + const reliableSearch = ['medium', 'high'].includes(analysis.analysisQuality?.confidence ?? '') && bestVisits >= 32 && playedVisits >= 32 && totalVisits >= 160 && + analysis.runtimeEvidence?.teachingReadiness?.canUseInFinalReport !== false + const warnings: string[] = [] + if (!hasRootVisits) warnings.push('根搜索总次数未知，不能把单独强制实战查询的次数加进根搜索。') + if (!reliableSearch) warnings.push('搜索证据偏弱或教学就绪检查未通过，先加深分析；不能确定全局急所。') + if (analysis.analysisQuality?.deepenRecommended) warnings.push('引擎建议加深搜索，结论应表述为当前搜索倾向。') + if (winrateSaturated) warnings.push('胜率处于极端区间，显著差距只采用目差，不能单靠胜率定性。') + if (scoreGap < 0 || winrateGap < 0) warnings.push('首选并非所有估值指标都占优，不应夸大差距。') + if (!bestLocalAlternative) warnings.push('未找到有足够搜索次数的其他本地候选；实战点较差不等于整个局部不值得下。') + warnings.push('距离与估值支持优先级比较，不能单独证明分断、封锁大龙、死活或先手原因。') + const globalPriority = distance >= distantThreshold && significantGap && reliableSearch && !bestLocalAlternative?.nearBest + const comparison: NonNullable<GlobalPriorityEvidence['comparison']> = { + bestMove: bestPoint.gtp, playedMove: played.gtp, playerColor: input.playerColor, + source: forced ? 'before-root-and-forced-played' : 'before-root-candidates', valuePerspective: 'current-player', + distanceMetric: 'chebyshev', distance, distantThreshold, bestWinrate, playedWinrate, + winrateGapPercentagePoints: winrateGap, bestScoreLead: sign * best.scoreLead, playedScoreLead: sign * actual.scoreLead, + scoreGap, bestVisits, playedVisits, totalVisits, rootVisitSource: hasRootVisits ? 'analysis-quality' : 'unavailable', significantGap, reliableSearch, winrateSaturated + } + const prompt = [ + `全局优先级（第${input.moveNumber}手落子前，同一落子前局面估值；${input.playerColor === 'B' ? '黑' : '白'}方视角${forced ? '；实战估值来自单独强制搜索' : '；两点均为根候选'}）：首选 ${bestPoint.gtp}，实战 ${played.gtp}；棋盘最大轴距离 ${distance}（远处阈值 ${distantThreshold}）。`, + `首选/实战胜率 ${bestWinrate.toFixed(1)}% / ${playedWinrate.toFixed(1)}%，差 ${winrateGap.toFixed(1)} 个百分点；目差 ${comparison.bestScoreLead.toFixed(1)} / ${comparison.playedScoreLead.toFixed(1)}，差 ${scoreGap.toFixed(1)} 目；候选搜索次数 ${bestVisits} / ${playedVisits}，根搜索总次数 ${hasRootVisits ? totalVisits : '未知'}（来源 ${comparison.rootVisitSource}）。估值来源 ${comparison.source}。`, + ...(significantGap && reliableSearch ? [`具体着手评价：按当前搜索，实战 ${played.gtp} 相比首选 ${bestPoint.gtp} 有明显估值损失；简短解说也应点出这一选择问题，并用已核验的棋盘效果解释首选用途。是否距离远、是否存在接近首选的本地替代，只影响整个局部优先级的判断，不能抹去具体实战点的损失。`] : []), + ...(bestLocalAlternative ? [`本地其他候选 ${bestLocalAlternative.move}（距实战 ${bestLocalAlternative.distanceFromPlayed}，搜索 ${bestLocalAlternative.visits} 次）；较首选差 ${bestLocalAlternative.scoreGapFromBest.toFixed(1)} 目 / ${bestLocalAlternative.winrateGapFromBestPercentagePoints.toFixed(1)} 个百分点。`] : []), + bestLocalAlternative?.nearBest ? `本局部仍可处理，实战点选择有差别：本地 ${bestLocalAlternative.move} 与首选接近，不能把实战 ${played.gtp} 的损失归因于整个局部不重要。` + : globalPriority ? bestLocalAlternative && (bestLocalAlternative.scoreGapFromBest >= 3 || (!bestLocalAlternative.winrateSaturated && bestLocalAlternative.winrateGapFromBestPercentagePoints >= 8)) + ? `解说主线应先说明：按当前 KataGo 搜索，首选 ${bestPoint.gtp} 也明显优于已搜索的本地替代 ${bestLocalAlternative.move}，本手更值得优先处理远处；再解释实战局部。` + : `解说主线应先说明：当前前列搜索更倾向远处，先比较 ${bestPoint.gtp} 一带，再解释实战 ${played.gtp}；不能据此断言整个局部不重要。` + : distance >= distantThreshold && significantGap ? `当前搜索更建议远处 ${bestPoint.gtp} 一带，相比具体实战点 ${played.gtp} 估值差明显；搜索证据偏弱，需要加深确认，不能说确定错过急所，也不能排除本局部其他好点。` + : '尚不满足远处首选且估值明显更好的条件，不能仅因首选距离远就说本局部不重要。', + ...warnings + ].join('\n') + return { status: 'compared', narrativePriority: globalPriority ? 'global-priority' : 'local-or-undetermined', boardTiming: 'before-move', comparison, bestLocalAlternative, warnings, prompt } +} diff --git a/src/main/services/teacher/initialBoardCapture.ts b/src/main/services/teacher/initialBoardCapture.ts new file mode 100644 index 0000000..ee5c789 --- /dev/null +++ b/src/main/services/teacher/initialBoardCapture.ts @@ -0,0 +1,33 @@ +import type { TeacherRunRequest, VisionEvidenceReport } from '@main/lib/types' +import type { AgentToolExecutionResult } from '../llm/agentRuntime' +import type { ChatMessage } from '../llm/provider' + +interface InitialCaptureInput { + provider: string + intent: string + request: TeacherRunRequest + report?: VisionEvidenceReport + hasCaptureHandler: boolean +} + +export function needsInitialBoardCapture(input: InitialCaptureInput): boolean { + const { request, report } = input + return input.provider === 'codex-app-server' && input.intent === 'current-move' && + request.mode === 'current-move' && report?.required === true && !report.attached && + (request.toolPolicy === 'auto' || request.toolPolicy === undefined) && + Boolean(request.gameId) && Number.isInteger(request.moveNumber) && request.moveNumber! > 0 && + input.hasCaptureHandler +} + +export async function prepareInitialBoardMessages( + input: InitialCaptureInput, + capture: () => Promise<AgentToolExecutionResult> +): Promise<ChatMessage[]> { + if (!needsInitialBoardCapture(input)) return [] + const result = await capture() + if (!result.ok) throw new Error(`初始棋盘图获取失败：${result.toolResult}`) + const hasImage = result.followupMessages.some(message => Array.isArray(message.content) && + message.content.some(part => part.type === 'image_url' && /^data:image\/(?:png|jpeg);base64,/.test(part.image_url.url))) + if (!hasImage) throw new Error('初始棋盘截图没有返回真实图片输入。') + return [{ role: 'user', content: `GoAgent 已通过棋盘截图工具取得当前局面：\n${result.toolResult}` }, ...result.followupMessages] +} diff --git a/src/main/services/teacher/moveBoardEffects.ts b/src/main/services/teacher/moveBoardEffects.ts new file mode 100644 index 0000000..f1339c4 --- /dev/null +++ b/src/main/services/teacher/moveBoardEffects.ts @@ -0,0 +1,181 @@ +/** Immediate board transitions, not life/death judgements or history-aware legality. */ +type Color = 'B' | 'W' +type Stone = { point: string; color: Color } +export interface EffectChain { + id: string + color: Color + stoneCount: number + neighboringStones: string[] + liberties: string[] +} +interface FullChain { id: string; color: Color; stones: string[]; liberties: string[] } +export interface KoConnectionEvidence { + kind: 'ko-connection' + status: 'observed-structure' + connectedSingleton: string + otherFriendlyChainIds: string[] + opponentProbe: { color: Color; move: string; capturedStones: string[]; resultingChain: EffectChain } + structuralRecapture: { color: Color; move: string; capturedStones: string[]; restoresBeforeBoard: true } + wording: string + limitations: string[] +} +export interface MoveBoardEffects { + /** Keep the useful explanation before verbose chain inventories in tool serialization. */ + prompt: string + status: 'observed' | 'abstained' + boardTiming: 'before-move' + legality: 'board-only-not-history-verified' + actual?: { + color: Color + move: string + adjacentFriendlyChains: EffectChain[] + adjacentEnemyChains: EffectChain[] + capturedStones: string[] + mergedFriendlyChainIds: string[] + resultingChain: EffectChain + libertyChanges: Array<{ beforeChainId: string; before: string[]; after: string[]; lost: string[]; gained: string[] }> + } + koConnection?: KoConnectionEvidence + limitations: string[] +} +const COLUMNS = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +const LIMITATIONS = [ + '这里只验证占点、提子、自杀及正交串变化；未结合完整棋谱和规则验证劫禁、超级劫或轮次合法性。', + '气数与连接事实不证明棋块死活、弱龙补强、整龙已活或本手目的；这些判断需要独立证据。' +] +function parse(raw: unknown, size: number): string | undefined { + if (typeof raw !== 'string') return undefined + const match = /^([A-HJ-Z])([1-9]\d?)$/.exec(raw.trim().toUpperCase()) + if (!match || COLUMNS.indexOf(match[1]) >= size || Number(match[2]) > size) return undefined + return `${match[1]}${Number(match[2])}` +} +function neighbors(point: string, size: number): string[] { + const x = COLUMNS.indexOf(point[0]); const y = Number(point.slice(1)) - 1 + return [[x - 1, y], [x + 1, y], [x, y - 1], [x, y + 1]] + .filter(([a, b]) => a >= 0 && a < size && b >= 0 && b < size) + .map(([a, b]) => `${COLUMNS[a]}${b + 1}`) +} +function ordered(points: Iterable<string>): string[] { + return [...points].sort((a, b) => Number(a.slice(1)) - Number(b.slice(1)) || COLUMNS.indexOf(a[0]) - COLUMNS.indexOf(b[0])) +} +function chainAt(board: Map<string, Color>, point: string, size: number): FullChain { + const color = board.get(point)! + const stones = new Set<string>(); const liberties = new Set<string>(); const pending = [point] + while (pending.length) { + const p = pending.pop()! + if (stones.has(p)) continue + stones.add(p) + for (const q of neighbors(p, size)) { + if (!board.has(q)) liberties.add(q) + else if (board.get(q) === color && !stones.has(q)) pending.push(q) + } + } + const sorted = ordered(stones) + return { id: `${color}:${sorted[0]}`, color, stones: sorted, liberties: ordered(liberties) } +} +function adjacentChains(board: Map<string, Color>, point: string, size: number): FullChain[] { + const found = new Map<string, FullChain>() + for (const q of neighbors(point, size)) if (board.has(q)) { + const chain = chainAt(board, q, size); found.set(chain.id, chain) + } + return [...found.values()] +} +function compact(chain: FullChain, move: string, size: number): EffectChain { + const adjacent = new Set(neighbors(move, size)) + return { id: chain.id, color: chain.color, stoneCount: chain.stones.length, + neighboringStones: chain.stones.filter((point) => adjacent.has(point)), liberties: chain.liberties } +} +/** Repetition is deliberately ignored: a ko recapture is a structural probe, not a legal PV. */ +function simulate(board: Map<string, Color>, point: string, color: Color, size: number): + { board: Map<string, Color>; captured: string[]; chain: FullChain } | undefined { + if (board.has(point)) return undefined + const next = new Map(board); next.set(point, color) + const captured = new Set<string>() + for (const chain of adjacentChains(next, point, size)) if (chain.color !== color && !chain.liberties.length) { + for (const stone of chain.stones) { next.delete(stone); captured.add(stone) } + } + const chain = chainAt(next, point, size) + if (!chain.liberties.length) return undefined + return { board: next, captured: ordered(captured), chain } +} +function sameBoard(a: Map<string, Color>, b: Map<string, Color>): boolean { + return a.size === b.size && [...a].every(([point, color]) => b.get(point) === color) +} +function abstain(reason: string): MoveBoardEffects { + return { prompt: `落子效果证据弃权：${reason}`, status: 'abstained', boardTiming: 'before-move', + legality: 'board-only-not-history-verified', limitations: [...LIMITATIONS, reason] } +} + +export function buildMoveBoardEffects(input: { + boardSize: number + boardSnapshot?: Stone[] + playerColor: Color + playedMove: string + /** Accepted for callers; partial history cannot establish rule-aware repetition legality. */ + moveHistory?: Stone[] +}): MoveBoardEffects { + const size = input.boardSize + if (!Number.isInteger(size) || size < 2 || size > COLUMNS.length) return abstain('棋盘尺寸无效。') + if (!Array.isArray(input.boardSnapshot)) return abstain('缺少完整落子前棋盘。') + if (input.playerColor !== 'B' && input.playerColor !== 'W') return abstain('落子颜色无效。') + const move = parse(input.playedMove, size) + if (!move) return abstain('落点无效或为停着。') + const board = new Map<string, Color>() + for (const stone of input.boardSnapshot) { + const point = parse(stone?.point, size) + if (!point || (stone.color !== 'B' && stone.color !== 'W')) return abstain('棋盘包含无效棋子。') + if (board.has(point)) return abstain('棋盘包含重复或冲突落点。') + board.set(point, stone.color) + } + if (board.has(move)) return abstain('落点已被占用；必须提供落子前棋盘。') + const checked = new Set<string>() + for (const point of board.keys()) if (!checked.has(point)) { + const chain = chainAt(board, point, size) + if (!chain.liberties.length) return abstain('落子前棋盘含无气串，无法可靠模拟。') + for (const stone of chain.stones) checked.add(stone) + } + const adjacent = adjacentChains(board, move, size) + const friends = adjacent.filter((chain) => chain.color === input.playerColor) + const enemies = adjacent.filter((chain) => chain.color !== input.playerColor) + const actual = simulate(board, move, input.playerColor, size) + if (!actual) return abstain('落子在不允许自杀的局部模拟中无气；无法确认本手效果。') + let koConnection: KoConnectionEvidence | undefined + // A connection needs two genuinely different strings, not multiple neighbors of one string. + if (friends.length >= 2 && actual.captured.length === 0) { + const opponent: Color = input.playerColor === 'B' ? 'W' : 'B' + const probe = simulate(board, move, opponent, size) + if (probe?.captured.length === 1 && probe.chain.stones.length === 1 && probe.chain.liberties.length === 1) { + const target = probe.captured[0] + const singleton = friends.find((chain) => chain.stones.length === 1 && chain.stones[0] === target && + chain.liberties.length === 1 && chain.liberties[0] === move) + const recapture = singleton && probe.chain.liberties[0] === target ? simulate(probe.board, target, input.playerColor, size) : undefined + if (recapture?.captured.length === 1 && recapture.captured[0] === move && sameBoard(recapture.board, board)) { + const other = friends.filter((chain) => chain.id !== singleton!.id) + koConnection = { + kind: 'ko-connection', status: 'observed-structure', connectedSingleton: target, + otherFriendlyChainIds: other.map((chain) => chain.id), + opponentProbe: { color: opponent, move, capturedStones: probe.captured, resultingChain: compact(probe.chain, target, size) }, + structuralRecapture: { color: input.playerColor, move: target, capturedStones: recapture.captured, restoresBeforeBoard: true }, + wording: `${input.playerColor}${move}把仅剩${move}一气的单子${target}与其它己方正交串连接，可称粘劫，消除这里的单子劫争结构。`, + limitations: ['对手占点与回提只是在落子前棋盘上的反事实结构探测，不是实战后应手，也不是立即合法回提或必然变化。', + '回提模拟忽略重复局面禁令；完整恢复原盘只证明劫循环形状，不证明劫争价值、单片劫分类或整块棋的死活。'] + } + } + } + } + const merged = friends.length >= 2 ? friends.map((chain) => chain.id) : [] + const libertyChanges = friends.map((chain) => ({ beforeChainId: chain.id, before: chain.liberties, after: actual.chain.liberties, + lost: chain.liberties.filter((point) => !actual.chain.liberties.includes(point)), + gained: actual.chain.liberties.filter((point) => !chain.liberties.includes(point)) })) + const prompt = [ + `落子即时效果（完整落子前棋盘，局部模拟）：实际${input.playerColor}${move}；${actual.captured.length ? `提掉${actual.captured.join('、')}` : '没有提子'}。`, + ...(koConnection ? [`结构[ko-connection] ${koConnection.wording}`, ...koConnection.limitations] : []), + `落点相邻己方真实串：${friends.map((chain) => `${chain.id}(${chain.stones.length}子)，相邻棋子={${compact(chain, move, size).neighboringStones.join('、')}}，原气={${chain.liberties.join('、')}}`).join('；') || '无'}。`, + merged.length ? `本手连接${merged.length}个此前分开的己方正交串；这些是连接事实，不能直接解释为弱龙补强。` : '本手没有合并多个此前分开的己方正交串。', + `落子后所在串${actual.chain.stones.length}子，气={${actual.chain.liberties.join('、')}}。`, ...LIMITATIONS + ].join('\n') + return { prompt, status: 'observed', boardTiming: 'before-move', legality: 'board-only-not-history-verified', + koConnection, actual: { color: input.playerColor, move, adjacentFriendlyChains: friends.map((chain) => compact(chain, move, size)), + adjacentEnemyChains: enemies.map((chain) => compact(chain, move, size)), capturedStones: actual.captured, + mergedFriendlyChainIds: merged, resultingChain: compact(actual.chain, move, size), libertyChanges }, limitations: LIMITATIONS.slice() } +} diff --git a/src/main/services/teacher/moveConceptEvidence.ts b/src/main/services/teacher/moveConceptEvidence.ts new file mode 100644 index 0000000..60c2501 --- /dev/null +++ b/src/main/services/teacher/moveConceptEvidence.ts @@ -0,0 +1,103 @@ +import type { KataGoMoveAnalysis } from '@main/lib/types' +import { recognizeMoveConcepts, formatMoveConceptsForPrompt, type MoveConceptResult } from '../knowledge/moveConcepts' +import { buildGlobalPriorityEvidence, type GlobalPriorityEvidence } from './globalPriorityEvidence' +import { buildBoardGroupRelations, type BoardGroupRelations } from './boardGroupRelations' +import { buildMoveBoardEffects, type MoveBoardEffects } from './moveBoardEffects' + +export interface MoveConceptEvidence { + moveNumber: number + boardTiming: 'before-move' + boardCoordinates?: { + timing: 'before-move'; boardSize: number; moveNumber: number + stonesByColor: { B: string[]; W: string[] } + } + globalPriority?: GlobalPriorityEvidence + boardGroups?: BoardGroupRelations + moveEffects?: MoveBoardEffects + candidateOwnDevelopment: Array<Pick<MoveConceptResult['ownDevelopment'][number], + 'kind' | 'status' | 'anchor' | 'groupStones' | 'direction' | 'corridor' | 'pressurePoints' | 'counterEvidence'> & { + move: string; color: 'B' | 'W' + }> + candidateMoveEffects: Array<{ move: string; color: 'B' | 'W'; effects: MoveBoardEffects }> + entries: Array<{ role: 'played' | 'candidate'; move: string; color: 'B' | 'W'; concepts: MoveConceptResult }> + prompt: string +} + +/** Only root candidates share this board. PV continuations require separate replay. */ +export function buildMoveConceptEvidence(input: { + moveNumber: number + gameId?: string + trialBranchHash?: string + boardSize: number + boardSnapshot?: Array<{ point: string; color: 'B' | 'W' }> + moveHistory?: Array<{ point: string; color: 'B' | 'W' }> + playedMove?: string + playerColor?: 'B' | 'W' + analysis?: KataGoMoveAnalysis +}): MoveConceptEvidence { + const proposed = input.analysis + const samePoint = (a: string | undefined, b: string | undefined) => Boolean(a && b && a.trim().toUpperCase() === b.trim().toUpperCase()) + // Use one identity gate for every analysis-derived field, not only globalPriority. + const analysis = proposed?.moveNumber === input.moveNumber && proposed.boardSize === input.boardSize && + (!input.gameId || proposed.gameId === input.gameId) && + (proposed.trialContext?.active ? proposed.trialContext.branchHash : undefined) === input.trialBranchHash && + proposed.currentMove && (proposed.currentMove.moveNumber === undefined || proposed.currentMove.moveNumber === input.moveNumber) && + (!input.playerColor || proposed.currentMove.color === input.playerColor) && + (!input.playedMove || samePoint(proposed.currentMove.gtp, input.playedMove)) + ? proposed : undefined + const color = input.playerColor ?? analysis?.currentMove?.color + const playedMove = input.playedMove ?? analysis?.currentMove?.gtp + const globalPriority = color && playedMove ? buildGlobalPriorityEvidence({ + ...input, playedMove, playerColor: color, boardTiming: 'before-move', analysis + }) : undefined + const boardGroups = color && playedMove ? buildBoardGroupRelations({ + boardSize: input.boardSize, boardSnapshot: input.boardSnapshot, playedMove, playerColor: color + }) : undefined + const moveEffects = color && playedMove ? buildMoveBoardEffects({ + boardSize: input.boardSize, boardSnapshot: input.boardSnapshot, playedMove, playerColor: color, moveHistory: input.moveHistory + }) : undefined + const candidateMoveEffects: MoveConceptEvidence['candidateMoveEffects'] = [] + const entries: MoveConceptEvidence['entries'] = [] + if (color && playedMove) entries.push({ role: 'played', move: playedMove, color, concepts: recognizeMoveConcepts({ + boardSize: input.boardSize, boardSnapshot: input.boardSnapshot, moveHistory: input.moveHistory, playerColor: color, playedMove + }) }) + if (color && input.boardSnapshot && analysis) { + for (const candidate of analysis.before.topMoves.slice(0, 3)) { + if (candidate.move === playedMove) continue + candidateMoveEffects.push({ move: candidate.move, color, effects: buildMoveBoardEffects({ + boardSize: input.boardSize, boardSnapshot: input.boardSnapshot, playedMove: candidate.move, + playerColor: color, moveHistory: input.moveHistory + }) }) + entries.push({ role: 'candidate', move: candidate.move, color, concepts: recognizeMoveConcepts({ + boardSize: input.boardSize, boardSnapshot: input.boardSnapshot, moveHistory: input.moveHistory, playerColor: color, playedMove: candidate.move + }) }) + } + } + // Project only already-recognized root candidates on this before-move board. + // Keep compact purpose facts ahead of effects/graphs; PV continuations never enter here. + const candidateOwnDevelopment: MoveConceptEvidence['candidateOwnDevelopment'] = entries + .filter(entry => entry.role === 'candidate') + .flatMap(entry => entry.concepts.ownDevelopment.slice(0, 2).map(candidate => ({ + move: entry.move, color: entry.color, kind: candidate.kind, status: candidate.status, + anchor: candidate.anchor, groupStones: candidate.groupStones, direction: candidate.direction, + corridor: candidate.corridor, pressurePoints: candidate.pressurePoints, + counterEvidence: candidate.counterEvidence + }))) + return { + // The complete coordinate inventory and global comparison survive verbose effects/graph truncation. + boardCoordinates: boardGroups?.status === 'observed' ? { + timing: 'before-move', boardSize: input.boardSize, moveNumber: input.moveNumber, + stonesByColor: boardGroups.positionFacts.stonesByColor + } : undefined, + globalPriority, + candidateOwnDevelopment, + moveEffects, + candidateMoveEffects, + boardGroups, + moveNumber: input.moveNumber, + boardTiming: 'before-move', + prompt: entries.length ? entries.map((entry) => `${entry.role === 'played' ? '实战/当前手' : 'AI根候选'} ${entry.color} ${entry.move}（第${input.moveNumber}手落子前）\n${formatMoveConceptsForPrompt(entry.concepts)}`).join('\n\n') + : '缺少当前手颜色或落子前棋盘证据，不能确定跳、拆、飞攻或飞压等术语。', + entries + } +} diff --git a/src/main/services/teacher/ownDevelopment.ts b/src/main/services/teacher/ownDevelopment.ts new file mode 100644 index 0000000..010e628 --- /dev/null +++ b/src/main/services/teacher/ownDevelopment.ts @@ -0,0 +1,95 @@ +export interface OwnDevelopmentCandidate { + kind: 'own-development' | 'side-extension' + status: 'hypothesis' + groupStones: string[] + anchor: string + move: string + geometry: 'jump' | 'knight' + direction: 'inward' | 'along-side' + pressurePoints: string[] + contactPressurePoints: string[] + corridor: string[] + evidence: string[] + counterEvidence: string[] + wording: string +} +const COLS = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +type P = { point: string; x: number; y: number } +/** Position facts only: development is not an assessment that a group is weak or alive. */ +export function buildOwnDevelopment(input: { boardSize: number; boardSnapshot?: Array<{point: string; color: 'B' | 'W'}>; playedMove: string; playerColor: 'B' | 'W' }): OwnDevelopmentCandidate[] { + const size = input.boardSize + const parse = (raw: string): P | undefined => { + const m = /^([A-HJ-Z])([1-9]\d?)$/.exec(raw.trim().toUpperCase()) + if (!m) return undefined + const x = COLS.indexOf(m[1]); const y = Number(m[2]) - 1 + return x >= 0 && x < size && y >= 0 && y < size ? { point: `${COLS[x]}${y + 1}`, x, y } : undefined + } + if (!Number.isInteger(size) || size < 2 || size > COLS.length || !input.boardSnapshot || !['B', 'W'].includes(input.playerColor)) return [] + const move = parse(input.playedMove); if (!move) return [] + const board = new Map<string, 'B' | 'W'>(); const friends: P[] = []; const enemies: P[] = [] + for (const stone of input.boardSnapshot) { + const p = parse(stone.point) + if (!p || !['B', 'W'].includes(stone.color) || board.has(p.point)) return [] + board.set(p.point, stone.color); (stone.color === input.playerColor ? friends : enemies).push(p) + } + if (board.has(move.point)) return [] + const dist = (a: P, b: P) => Math.max(Math.abs(a.x - b.x), Math.abs(a.y - b.y)) + const touch = (a: P, b: P) => Math.abs(a.x - b.x) + Math.abs(a.y - b.y) === 1 + const groups: P[][] = []; const visited = new Set<string>() + for (const seed of friends) { + if (visited.has(seed.point)) continue + const group = [seed]; visited.add(seed.point) + for (let i = 0; i < group.length; i++) for (const p of friends) if (!visited.has(p.point) && touch(group[i], p)) { visited.add(p.point); group.push(p) } + groups.push(group) + } + const result: OwnDevelopmentCandidate[] = [] + for (const group of groups) for (const anchor of group) { + const dx = move.x - anchor.x; const dy = move.y - anchor.y + const axes = [Math.abs(dx), Math.abs(dy)].sort((a, b) => a - b) + const jump = axes[0] === 0 && axes[1] >= 2 && axes[1] <= 4 + const knight = axes[0] === 1 && axes[1] >= 2 && axes[1] <= 3 + if (!jump && !knight) continue + const corridor: string[] = [] + if (jump) for (let i = 1; i < axes[1]; i++) corridor.push(`${COLS[anchor.x + Math.sign(dx) * i]}${anchor.y + Math.sign(dy) * i + 1}`) + if (corridor.some(p => board.has(p))) continue + // Knight shapes have no unique corridor. Require at least one open monotone route. + const route = (x: number, y: number): boolean => { + if (x === move.x && y === move.y) return true + for (const [nx, ny] of [[x + Math.sign(move.x - x), y], [x, y + Math.sign(move.y - y)]]) { + if (nx === x && ny === y) continue + if (!board.has(`${COLS[nx]}${ny + 1}`) && route(nx, ny)) return true + } + return false + } + if (knight && !route(anchor.x, anchor.y)) continue + const pressure = enemies.filter(p => group.some(a => dist(a, p) <= 4)) + const contacts = pressure.filter(p => group.some(a => touch(a, p))) + const frames = [{depth:anchor.x,nx:1,ny:0},{depth:size-1-anchor.x,nx:-1,ny:0},{depth:anchor.y,nx:0,ny:1},{depth:size-1-anchor.y,nx:0,ny:-1}] + const min = Math.min(...frames.map(f => f.depth)) + if (min > 3) continue + for (const f of frames.filter(f => f.depth === min)) { + const inward = dx*f.nx + dy*f.ny; const tangent = dx*f.ny - dy*f.nx + const along = inward === 0 && jump + const toward = inward > 0 && inward >= Math.abs(tangent) + const ux = along ? Math.sign(dx) : dx; const uy = along ? Math.sign(dy) : dy + const projection = (p: P) => (p.x-anchor.x)*ux + (p.y-anchor.y)*uy + if (group.some(p => projection(p) > 0)) continue // the anchor must be this real string's frontier + const tangentEscape = along && contacts.length >= 2 && pressure.filter(p => dist(p, anchor) <= 2).every(p => projection(p) < projection(move)) + const inwardDevelopment = toward && pressure.length >= 2 && pressure.some(p => projection(p) < 0) + if (!along && !inwardDevelopment) continue + const pressuredDevelopment = tangentEscape || inwardDevelopment + const endpointEnemies = enemies.filter(p => dist(p, move) <= 2) + const evidence = [`己方真实正交串={${group.map(p=>p.point).sort().join('、')}}；${anchor.point}是本次展开方向的前端。`, `${move.point}相对${anchor.point}为${jump?'开放直跳':'飞的几何关系'}，${along?'沿邻近边线展开':'向邻近边线的中腹一侧展开'}${corridor.length?`；中间${corridor.join('、')}为空`:''}。`] + if (pressure.length) evidence.push(`该串四格内敌棋=${pressure.map(p=>p.point).sort().join('、')}；其中直接接触敌棋=${contacts.map(p=>p.point).sort().join('、')||'无'}，只是压力配置事实。`) + if (tangentEscape) evidence.push('落点沿边越过锚点两格内敌方压迫点的前沿；边距不变也可能构成自身出头。') + result.push({ kind: pressuredDevelopment?'own-development':'side-extension', status:'hypothesis', groupStones:group.map(p=>p.point).sort(), anchor:anchor.point, move:move.point, geometry:jump?'jump':'knight', direction:along?'along-side':'inward', pressurePoints:pressure.map(p=>p.point).sort(), contactPressurePoints:contacts.map(p=>p.point).sort(), corridor, evidence, + counterEvidence:['没有评估棋块死活、根据地或唯一逃路；未验证连接和对手封锁应对。', ...(endpointEnemies.length?[`落点附近仍有敌棋${endpointEnemies.map(p=>p.point).sort().join('、')}，发展空间和拆边距离仍有竞争。`]:[])], + wording:pressuredDevelopment?`${move.point}从己串${group.map(p=>p.point).sort().join('、')}的${anchor.point}一端${along?'沿边跳出':'向中腹发展'}，可先检查自身出头、争取发展空间的意图；不证明已逃出或安定。`:`${move.point}相对己串前端${anchor.point}沿边开放直跳，可作为拆边、展开自身的候选；不证明已安定或成空。` }) + if (along && pressuredDevelopment) { + const development = result[result.length - 1] + result.push({ ...development, kind: 'side-extension', wording: `${move.point}相对己串前端${anchor.point}沿边开放直跳，也可检查拆边、展开自身的意图；近终点敌棋仍需处理，不证明已安定或成空。` }) + } + } + } + return result.sort((a,b)=>(a.kind==='own-development'?0:1)-(b.kind==='own-development'?0:1)||(a.geometry==='jump'?0:1)-(b.geometry==='jump'?0:1)||a.anchor.localeCompare(b.anchor)) +} diff --git a/src/main/services/teacher/teachingEvidence.ts b/src/main/services/teacher/teachingEvidence.ts index 3e42e45..b0038ec 100644 --- a/src/main/services/teacher/teachingEvidence.ts +++ b/src/main/services/teacher/teachingEvidence.ts @@ -319,10 +319,10 @@ export function buildTeachingPacingAdvice( whyThisMuchExplanation = 'KataGo 搜索或实战手证据还不够强，只能讲判断倾向，不能下绝对结论。' } else if (tactical || phase === 'middle' || severity === 'mistake' || severity === 'blunder' || winrateLoss >= 7 || scoreLoss >= 4) { teachingDensity = 'detailed' - whyThisMuchExplanation = '这是中盘战、急所计算或明显损失局面，需要讲清这手目的、对方应手、PV 后续和实战代价。' + whyThisMuchExplanation = '这是中盘战、急所计算或明显损失局面，内部应核验目的、对方应手、PV 后续和实战代价；最终篇幅遵守用户的输出策略，默认当前手只讲主要目的。' } else if (hasJosekiBranch || focus === 'joseki-branch' || (phase === 'opening' && winrateLoss >= 2)) { teachingDensity = 'branch' - whyThisMuchExplanation = '这是定式分支、布局选择或相似型局面，适合列 1-2 个关键变化和选择条件。' + whyThisMuchExplanation = '这是定式分支、布局选择或相似型局面，内部应比较关键变化和选择条件；用户要求展开时再列变化。' } else if ((focus === 'joseki-normal' && winrateLoss < 2) || (severity === 'good' && winrateLoss < 2)) { teachingDensity = 'minimal' whyThisMuchExplanation = '这是常规定式或损失很小的正常选择，只点明棋形方向即可，不需要长篇讲解。' diff --git a/src/main/services/teacher/visionEvidenceVerifier.ts b/src/main/services/teacher/visionEvidenceVerifier.ts index c47e2ea..e79b169 100644 --- a/src/main/services/teacher/visionEvidenceVerifier.ts +++ b/src/main/services/teacher/visionEvidenceVerifier.ts @@ -6,7 +6,11 @@ const NO_BOARD_IMAGE_PATTERNS = [ /看不到(?:棋盘|图片|图像)/, /未提供(?:棋盘|图片|图像)/, /无法看到(?:棋盘|图片|图像)/, + /(?:工具|截图工具).{0,8}(?:没有|未|没能).{0,8}返回.{0,6}(?:图像|图片|棋盘图|截图)/, + /(?:没有|未能|没(?:有|能)?)(?:取得|获取|收到|拿到)(?:棋盘)?(?:图像|图片|截图)/, + /(?:本轮|本次|这次).{0,4}(?:未|没有)(?:附图|附上.{0,4}(?:图片|图像|截图|棋盘图))/, /no\s+(?:board\s+)?image/i, + /(?:tool\s+)?(?:did\s+not|didn't|has\s+not|hasn't)\s+return\s+(?:an?\s+)?(?:board\s+)?(?:image|screenshot)/i, /cannot\s+see\s+(?:the\s+)?(?:board|image)/i, /no\s+visual\s+input/i ] diff --git a/src/main/services/teacherAgent.ts b/src/main/services/teacherAgent.ts index dbeec38..70f20ff 100644 --- a/src/main/services/teacherAgent.ts +++ b/src/main/services/teacherAgent.ts @@ -30,7 +30,7 @@ import type { VisionEvidenceReport } from '@main/lib/types' import type { ChatContentPart, ChatMessage, ChatTool, ChatToolCall, ChatTurnResult } from './llm/provider' -import { analyzePosition, cancelKataGoAnalysis } from './katago' +import { analyzePosition, analyzeTrialPositionWithProgress, cancelKataGoAnalysis } from './katago' import { analyzeGameQuickRuntime } from './analysis/runtimeIntegration' import { MOVE_RANGE_KEY_MOVE_LIMIT, MOVE_RANGE_MAX_MOVES, parseMoveRangeFromPrompt, validateMoveRange } from '@shared/moveRange' import { formatMoveRangeSummaryForPrompt, selectMoveNumbersForRangeRefine } from './teacher/moveRangeReview' @@ -49,6 +49,11 @@ import { classifyTeacherIntent, type TeacherIntent } from './teacher/intentClass import { buildTeachingPacingAdvice } from './teacher/teachingEvidence' import { buildTeacherArtifact, validateTeachingArtifact } from './teacher/teachingArtifact' import { buildTeachingEvidenceBundle, formatTeachingEvidenceBundleForPrompt } from './teacher/evidenceBundle' +import { buildMoveConceptEvidence, type MoveConceptEvidence } from './teacher/moveConceptEvidence' +import { canReuseTeacherAnalysis } from './teacher/analysisReuse' +import { buildCurrentMoveResponsePolicy } from './teacher/currentMoveResponsePolicy' +import { prepareInitialBoardMessages } from './teacher/initialBoardCapture' +import { missingTeacherEvidenceTools } from './teacher/evidenceToolGate' import { scoreLeadForColor, scoreSummaryFromBlackLead } from './teacher/scorePerspective' import { buildTeacherPersonaInstruction, @@ -68,7 +73,7 @@ import { validateVisionEvidenceForIntent } from './teacher/visionEvidence' import { buildVisionEvidenceRepairNote, verifyVisionEvidenceMarkdown } from './teacher/visionEvidenceVerifier' -import { runProviderTurn } from './llm/providerRegistry' +import { resolveLlmConnection, runProviderTurn } from './llm/providerRegistry' import { isLlmSetupConfigurationError } from './llm/openaiCompatibleProvider' type TeacherProgressEmitter = (progress: TeacherRunProgress) => void @@ -355,7 +360,7 @@ function teacherLanguageName(locale: unknown): string { return '简体中文' } -function systemPrompt(level: CoachUserLevel): string { +function systemPrompt(level: CoachUserLevel, responsePolicyInstruction?: string): string { const settings = getSettings() return [ '你是 GoAgent 的围棋老师。', @@ -363,25 +368,31 @@ function systemPrompt(level: CoachUserLevel): string { '帮助学生理解棋局，并提升下一次判断。', '需要信息时调用工具；不要靠印象猜局面。', '你具备真正的工具调用能力：棋谱、棋盘截图、KataGo、知识库、学生画像、报告和本机工具都应按需调用；不要靠按钮预处理或印象猜局面。', - '分析当前手、整盘复盘或区间复盘时，必须通过工具取得棋盘图、KataGo 证据和知识库匹配。当前手至少调用 board.captureTeachingImage、katago.analyzePosition、knowledge.matchPosition 或 knowledge.searchLocal；整盘复盘先调用 sgf.readGameRecord、katago.analyzeGameBatch，再截图 3-6 个关键手；区间复盘先精读区间关键手，再截图关键手。', + '分析当前手、整盘复盘或区间复盘时，必须通过工具取得棋盘图、KataGo 证据和知识库匹配。当前手若尚未附图，先调用 board.captureTeachingImage；已提供的当前棋盘图无需重复截图，再调用 katago.analyzePosition、knowledge.matchPosition 或 knowledge.searchLocal；整盘复盘先调用 sgf.readGameRecord、katago.analyzeGameBatch，再截图 3-6 个关键手；区间复盘先精读区间关键手，再截图关键手。', '工具结果和 KataGo 是事实依据。', '如果请求或工具结果标记 boardContext=trial / trialContext，说明这是用户临时“试下”的变化分支；讲解时必须明确说“如果这样下/这个试下分支”，不要把它说成实战主线。', '如果 KataGo 结果包含 tracePacket，优先使用 tracePacket.searchSummary、candidateComparison、policySearchDelta、pvSupport、ownershipSummary、humanPolicySignals 和 shallowSearchTree 来解释“为什么”。', 'tracePacket 是给老师的搜索证据摘要，不要把原始 MCTS/搜索字段生硬堆给学生；请翻译成“自然但被搜索否定”“不直观但搜索支持”“PV 支撑弱所以只能参考”等教学语言。', '如果 tracePacket 的置信度或 PV 支撑不足，必须降级措辞，不能说唯一、必杀、必败或绝对。', '如果工具结果包含 runtimeEvidence.teachingReadiness，必须遵守其中的 level、shouldDeepen、safeWording、warnings 和 blockingIssues。', - '如果 runtimeEvidence.cacheStatus 是 hit，可以说明这是已缓存证据；如果是 written/miss/lower-quality，要说明当前证据是本次分析或仍需加深。', + responsePolicyInstruction ? '缓存状态和搜索次数用于内部判断；默认不向学生解释缓存命中、工具流程或技术字段。需要加深时先核验，最终只保留影响结论的简短限定。' : '如果 runtimeEvidence.cacheStatus 是 hit，可以说明这是已缓存证据；如果是 written/miss/lower-quality，要说明当前证据是本次分析或仍需加深。', '讲胜负、领先、落后和目数时，优先使用工具结果里的 teacherScore.text；没有 teacherScore 时再使用 scoreSummary.text/leader/leadPoints。blackScoreLead 是黑棋为正，负数表示白棋领先，不要自己用裸 scoreLead 符号猜胜负。', '内部核验时仍可用 scoreSummary.leader 和 scoreSummary.leadPoints 判断方向和数字，但对学生优先输出 teacherScore.text 这种简洁说法。', '棋谱的 result / game.result / rawResult 是终局记录，不是 KataGo 当前目差；Fox 数字结果还有平台口径换算，只有 resultSummary.displayLeadPoints / comparisonLeadPoints 才能和 KataGo/LizzieYzy 风格的目差比较。', '如果当前手是终局，且工具返回 resultSummary.confidence=recorded-result，请把 resultSummary.teacherText 当作终局目数来源。', '对学生输出时保持简洁：直接说“黑领先 X 目”或“白领先 X 目”；不要主动解释“棋谱记录、KataGo估值、Fox平台口径、换算口径”，除非用户追问来源。', '不要编造坐标、胜率、PV、定式名或来源。', + '棋子颜色与位置以 moveConceptEvidence.boardCoordinates.stonesByColor 的完整落子前坐标清单为准（兼容 boardGroups.positionFacts.stonesByColor）；它在详细效果和棋块图之前，避免尾部截断丢失盘面。完整清单没有的点是空点，只能作为本手、候选、路径或后续着手，不得称为已经存在的棋子；若完整清单缺失或证据弃权，不能按空盘处理。棋盘图片用于交叉核对，不用图片猜测覆盖棋谱坐标。', '每个关键结论都应能回指到工具证据；数字、坐标、PV、定式名、死活结论和先后手判断没有证据时必须降级成假设。', - '当 analysisQuality.confidence 不是 high，必须使用“AI 更倾向 / 更像 / 不宜下绝对结论”等低风险措辞。', - '强匹配才能明确说定式、死活型或手筋名；相似匹配只能说“像某某型”。', - '把握讲解火候：常规定式少讲，分支列变化，中盘战详细讲目的和后续。', - '如果工具结果给出 teachingDensity，就按它控制详略：minimal 很短，branch 讲 1-2 个关键变化，detailed 讲目的、应手、后续变化和实战评价，caution 只说倾向。', + responsePolicyInstruction ? '搜索置信度偏低时，评价优劣才用“AI更倾向”等措辞；有坐标支持的目的推断可用“意在/可能”，不必先报AI推荐点，也不必额外追加免责声明。' : '当 analysisQuality.confidence 不是 high，必须使用“AI 更倾向 / 更像 / 不宜下绝对结论”等低风险措辞。', + '知识检索的 exact/strong 只表示相似度，不能单独授权当前局面的死活、手筋、先后手或攻击结论。题库答案属于训练题，不是本局合法变化；明确命名战术必须有独立读棋证据，相似模式用于提示该检查什么。', + '先看全局再解释局部：moveConceptEvidence.globalPriority 比较同一落子前的首选和实战。先核对 bestLocalAlternative：nearBest=true 表示本局部仍可处理，只是实战点选择有差别，不得说整个局部不重要。首选在远处、差距明显且无接近首选的本地替代时，先说明当前搜索更建议处理哪里；没有已搜索的本地替代只能说前列搜索倾向，不能断言局部不急。差距小也不能仅凭距离说重点不在此处。搜索浅时保留这个方向提醒，并用 katago.analyzePosition 的显式 maxVisits 加深核对，不要因置信度低就把整段写成无法判断。不要用落子前后估值相减代替同局面候选差距。', + '具体实战点的优劣与整个局部的优先级分开判断：comparison.significantGap 和 reliableSearch 均为 true 时，即使 narrativePriority 是 local-or-undetermined 或本地另有好点，也应在简短解说中指出本手选择不佳。首选用途可参考 candidateMoveEffects 的实际棋盘效果及 candidateOwnDevelopment 的自身发展假设；后者不证明已经安定，开放路线不证明双方能够强制联络。只选最主要的有据目的，不罗列竞争假设或无关保留判断。', + '局部目的要先确定主体棋块：moveConceptEvidence.boardGroups 的正交串、潜在联系、自身发展候选和 regionIntents 提供竞争解释。先检查己方是否借本手出头、防被封锁或沿边展开，再与对敌攻击、争头、分割的证据比较；不是固定选防守，也不能因落点在敌棋上方或两串之间就固定选攻击。自身发展候选不证明己棋未活、已经逃出或已经安定，对敌候选不证明已经分断、封锁成功或杀棋。潜在联系不能说成已经连接，正交串不能直接当战略整龙。同一真实己串同时给出跳、飞时，检查发展方向最前端的锚点，背侧的小飞不能自动覆盖前端开放直跳。不要仅报与单颗棋子的飞、尖关系，也不要把“尚未验证成功”写成“没有任何可讲的目的”。没有区域候选时仍可结合全盘坐标与独立变化提出有明确来源的目的推断，不得凭空补坐标或效果。', + '当前手先取 KataGo 再取知识位置证据；若知识工具的 globalPriority 提示缺少匹配分析，而随后才取得或加深 KataGo，应重新调用 knowledge.matchPosition 更新全局比较。工具文本截断时不要把后部图明细的缺席当作棋盘没有其它棋子，优先用完整坐标清单与前部证据摘要。', + 'moveConceptEvidence 区分落子前的相对几何与意图：多个己棋锚点保留其关系；hypothesis 表示目的推断，不能升级为战术结果。跳不能自动改称拆，飞不能自动改称飞攻或飞压。全局主线之后围绕实际受益的己方或敌方棋块解释目的，再用跳、尖、飞、边角关系说明支持；有角部历史才能说“被夹后”。moveEffects/candidateMoveEffects 若给出实际提子、连接或局部单子劫循环，应优先讲这一具体棋盘效果；粘劫不自动是弱龙补强，棋盘结构的反事实回提也不表示可无视劫禁立即回提。没有独立未安定证据，不断言补强弱棋；局部模式分数、气数少、ownership、候选距离、目差或PV长度不能单独证明死活或强制性。若远处推荐确实明显更好，可由推荐点的实际连接、提子等效果或同一分支已核验参考线解释用途，不得凭坐标猜防进角，也不得把全局目差归为一颗棋子的固定价值。不要把不同候选分支的PV串成一条变化。对同一未验证效果只需一句简短保留，不要反复用“不能断定”取代具体解释。', + responsePolicyInstruction ? '当前手的 teachingDensity 只指导内部核验深度，中盘战或 detailed 标记不要求长回答；最终篇幅遵守当前手输出策略。' : '把握讲解火候：常规定式少讲，分支列变化，中盘战详细讲目的和后续。', + responsePolicyInstruction ? '工具事实用于选出最重要的一条棋理，不要求逐项向学生复述图像、几何锚点、搜索统计、PV和训练建议。' : '如果工具结果给出 teachingDensity，就按它控制详略：minimal 很短，branch 讲 1-2 个关键变化，detailed 讲目的、应手、后续变化和实战评价，caution 只说倾向。', '像老师讲棋：先帮学生看懂棋形和判断方法，再自然引用必要证据；不要按固定栏目或机器报告口吻堆字段。', '区间复盘要先讲区间走势，再聚焦 3-5 个关键手；不要逐手流水账；每个关键手必须引用 KataGo、analysisQuality、棋形识别或战术信号。', '区间过长或证据不足时要建议缩小范围或只做抽样总结，不能把低 visits 区间分析说成最终结论。', @@ -395,7 +406,8 @@ function systemPrompt(level: CoachUserLevel): string { terminologyDensity: normalizeTerminologyDensity(settings.teacherTerminologyDensity), explanationPace: normalizeExplanationPace(settings.teacherExplanationPace), variationDetail: normalizeVariationDetail(settings.teacherVariationDetail) - }) + }), + responsePolicyInstruction ?? '' ].join('\n') } @@ -549,8 +561,8 @@ export function cancelTeacherRun(payload: { runId?: string } = {}): { cancelled: return { cancelled } } -function agentSystemPrompt(level: CoachUserLevel): string { - return systemPrompt(level) +function agentSystemPrompt(level: CoachUserLevel, responsePolicyInstruction?: string): string { + return systemPrompt(level, responsePolicyInstruction) } function stringInput(input: JsonObject, key: string, fallback = ''): string { @@ -832,6 +844,8 @@ function defaultArtifactTitleForState(state: TeacherAgentSessionState): string { } function initialAgentUserMessage(state: TeacherAgentSessionState): ChatMessage { + const responsePolicy = buildCurrentMoveResponsePolicy({ intent: state.intent, prompt: state.request.prompt, + explanationPace: getSettings().teacherExplanationPace }) const visionEvidence = state.request.visionEvidence ?? buildVisionEvidenceReport(state.request, state.intent) const visionValidation = validateVisionEvidenceForIntent(visionEvidence, state.intent) if (!visionValidation.ok && !allowToolFirstVision(state.request)) { @@ -870,11 +884,11 @@ function initialAgentUserMessage(state: TeacherAgentSessionState): ChatMessage { '任务说明：请根据 intent 完成用户请求。', '你可以自主调用工具获取棋谱、棋盘图、KataGo 数据、知识库和学生画像；不要等待程序替你预处理。', 'board.captureTeachingImage 是用来看棋盘图片的工具；拿到图后再调用 KataGo、调用知识库，匹配棋形、定式、死活、手筋或常见错误类型。', - '如果 intent 是 current-move，请调用 board.captureTeachingImage 获取当前手棋盘图，再调用 katago.analyzePosition 和 knowledge.matchPosition/searchLocal 核对事实。', + '如果 intent 是 current-move 且尚未附图，请调用 board.captureTeachingImage 获取当前手棋盘图；已有当前手棋盘图时无需重复截图。再调用 katago.analyzePosition 和 knowledge.matchPosition/searchLocal 核对事实。', '如果 boardContext=trial，请把本轮当作用户“试下”的变化图：截图和 prefetchedAnalysis 已对应试下分支；讲解时只能说“这个试下分支/如果这样下”，不能说成实战。', '如果 intent 是 game-review，请先调用 sgf.readGameRecord 和 katago.analyzeGameBatch 找关键问题手，再调用 board.captureTeachingImage(selection=top-loss,maxImages=3-6) 获取关键手图，最后调用知识库工具讲解。', '如果 intent 是 move-range，请调用 katago.analyzeMoveRangeKeyMoves 精读区间关键手，再调用 board.captureTeachingImage(selection=move-range-top-loss,maxImages=3-6) 获取关键手图。', - '当前手讲解要按工具返回的 teachingDensity 掌握详略：常规定式少讲；定式分支或相似型列关键变化；中盘战、攻杀、转换要讲目的、对方应手、后续变化和实战评价。', + responsePolicy?.instruction ?? '当前手讲解要按工具返回的 teachingDensity 掌握详略：常规定式少讲；定式分支或相似型列关键变化；中盘战、攻杀、转换要讲目的、对方应手、后续变化和实战评价。', 'boardImageAttached=true 表示本轮用户消息已附棋盘图；否则请先调用 board.captureTeachingImage。请把图片中的棋形、厚薄、急所和全局方向作为局面判断依据。', '如果 visionEvidence.attached=true，本轮已经提供棋盘图，严禁说“没有棋盘图”“看不到棋盘”“未提供图片”。', '如果 visionEvidence.required=true 但证据不完整，程序会阻止任务执行；你不需要猜测缺失图片。', @@ -886,7 +900,8 @@ function initialAgentUserMessage(state: TeacherAgentSessionState): ChatMessage { formatKataGoTraceForPrompt(state.request.prefetchedAnalysis?.tracePacket ?? state.lastAnalysis?.tracePacket), 'prefetchedAnalysisAvailable=true 表示 katago.analyzePosition 可复用已缓存的 KataGo 分析结果。', '上下文JSON：', - JSON.stringify(context) + JSON.stringify(context), + responsePolicy?.instruction ?? '' ].join('\n') const visionParts = buildVisionImageContentParts(state.request, visionEvidence) if (visionParts.length > 0) { @@ -1059,23 +1074,32 @@ async function knowledgeBundleForState(state: TeacherAgentSessionState, input: J knowledgeMatches: KnowledgeMatch[] recommendedProblems: RecommendedProblem[] teachingPacing?: TeachingPacingAdvice + moveConceptEvidence: MoveConceptEvidence }> { const record = await ensureSessionRecord(state).catch(() => undefined) - const analysis = state.lastAnalysis - const moveNumber = numberInput(input, 'moveNumber', analysis?.moveNumber ?? state.request.moveNumber ?? record?.moves.length ?? 80, 0, record?.moves.length ?? 400) + const trial = state.request.boardContext === 'trial' && state.request.trialBranch?.active ? state.request.trialBranch : undefined + const positionMoves: GameMove[] | undefined = record ? trial ? [ + ...record.moves.slice(0, trial.baseMoveNumber), + ...trial.moves.map((move, index) => ({ ...move, moveNumber: trial.baseMoveNumber + index + 1, point: move.gtp, pass: move.gtp.toLowerCase() === 'pass' })) + ] : record.moves : undefined + const defaultMoveNumber = trial ? trial.baseMoveNumber + trial.moves.length : state.lastAnalysis?.moveNumber ?? state.request.moveNumber ?? positionMoves?.length ?? 80 + const moveNumber = numberInput(input, 'moveNumber', defaultMoveNumber, 0, positionMoves?.length ?? 400) + const matchingAnalysis = analysisForMoveNumber(state, moveNumber) + const analysis = matchingAnalysis && (!record || matchingAnalysis.gameId === record.game.id) && (trial ? matchingAnalysis.trialContext?.branchHash === trial.branchHash : !matchingAnalysis.trialContext?.active) + ? matchingAnalysis : undefined const boardSize = record?.boardSize ?? analysis?.boardSize ?? 19 const boardState = record ? buildBoardState({ boardSize, - moves: record.moves, + moves: positionMoves ?? [], uptoMoveNumber: Math.max(0, moveNumber - 1), initialStones: record.initialStones }) : undefined - const boardSnapshot = boardState ? boardStateToSnapshot(boardState) : undefined + const boardSnapshot = boardState && boardState.warnings.length === 0 ? boardStateToSnapshot(boardState) : undefined + const recordMove = positionMoves?.[moveNumber - 1] const anchors = (analysis ? [ analysis.playedMove?.move ?? analysis.currentMove?.gtp, - ...analysis.before.topMoves.slice(0, 6).map((candidate) => candidate.move), - ...analysis.before.topMoves.slice(0, 2).flatMap((candidate) => candidate.pv.slice(0, 4)) + ...analysis.before.topMoves.slice(0, 6).map((candidate) => candidate.move) ] : arrayInput(input, 'candidateMoves')).filter((move): move is string => typeof move === 'string' && move.length > 0) const localWindows = boardSnapshot ? buildLocalWindows(boardSnapshot, anchors, boardSize) : undefined @@ -1086,18 +1110,18 @@ async function knowledgeBundleForState(state: TeacherAgentSessionState, input: J const query = { text: stringInput(input, 'text', state.request.prompt), moveNumber, - totalMoves: record?.moves.length ?? moveNumber, + totalMoves: positionMoves?.length ?? moveNumber, boardSize, - recentMoves: record?.moves.slice(Math.max(0, moveNumber - 40), moveNumber) ?? [], + recentMoves: positionMoves?.slice(Math.max(0, moveNumber - 40), moveNumber) ?? [], userLevel: state.profile.userLevel, studentLevel: state.profile.userLevel, - playerColor: analysis?.currentMove?.color, + playerColor: recordMove?.color ?? analysis?.currentMove?.color, lossScore: analysis?.playedMove?.scoreLoss ?? numberInput(input, 'lossScore', 2), judgement: analysis?.judgement ?? 'mistake', contextTags: analysis ? tagsFromAnalysis(analysis, analysis.currentMove) : themesFromProfile(state.profile), - playedMove: analysis?.playedMove?.move ?? analysis?.currentMove?.gtp ?? stringInput(input, 'playedMove'), + playedMove: recordMove?.gtp ?? analysis?.playedMove?.move ?? analysis?.currentMove?.gtp ?? stringInput(input, 'playedMove'), candidateMoves: analysis?.before.topMoves.slice(0, 8).map((candidate) => candidate.move) ?? arrayInput(input, 'candidateMoves'), - principalVariation: analysis?.before.topMoves.slice(0, 3).flatMap((candidate) => candidate.pv.slice(0, 8)) ?? arrayInput(input, 'principalVariation'), + principalVariation: analysis?.before.topMoves[0]?.pv.slice(0, 8) ?? arrayInput(input, 'principalVariation'), boardSnapshot, localWindows, maxResults: numberInput(input, 'maxResults', 4, 1, 8) @@ -1105,12 +1129,19 @@ async function knowledgeBundleForState(state: TeacherAgentSessionState, input: J const knowledgeMatches = searchKnowledgeMatches({ ...query, maxResults: 8 }) const recommendedProblems = recommendedProblemsFromMatches(knowledgeMatches, 3, { includeWeakFallback: true, includeJosekiFallback: true, includeDrillFallback: true }) const knowledge = searchKnowledge(query) + const moveConceptEvidence = buildMoveConceptEvidence({ ...query, analysis, + gameId: record?.game.id, + trialBranchHash: trial?.branchHash, + moveHistory: boardSnapshot ? positionMoves?.slice(0, Math.max(0, moveNumber - 1)).map((move) => ({ point: move.gtp, color: move.color })) : undefined + }) const teachingPacing = analysis ? buildTeachingPacingAdvice(analysis, knowledgeMatches) : undefined state.knowledge = knowledge state.knowledgeMatches = knowledgeMatches state.recommendedProblems = recommendedProblems state.teachingPacing = teachingPacing - return { knowledge, knowledgeMatches, recommendedProblems, teachingPacing } + // Keep the current-position evidence ahead of potentially large retrieval packets. + // Tool messages have a size limit; trailing evidence can otherwise disappear entirely. + return { moveConceptEvidence, knowledge, knowledgeMatches, recommendedProblems, teachingPacing } } function analysisForMoveNumber(state: TeacherAgentSessionState, moveNumber: number): KataGoMoveAnalysis | undefined { @@ -1118,6 +1149,18 @@ function analysisForMoveNumber(state: TeacherAgentSessionState, moveNumber: numb return state.rangeAnalyses?.find((analysis) => analysis.moveNumber === moveNumber) } +async function analyzeTeacherPosition(state: TeacherAgentSessionState, gameId: string, moveNumber: number, maxVisits: number): Promise<KataGoMoveAnalysis> { + const trial = state.request.boardContext === 'trial' && state.request.trialBranch?.active ? state.request.trialBranch : undefined + if (!trial) return analyzePosition(gameId, moveNumber, maxVisits, { runId: state.id, group: 'teacher' }) + if (gameId !== state.request.gameId || moveNumber !== trial.baseMoveNumber + trial.moves.length) { + throw new Error('试下分析必须对应当前试下棋谱与分支末手，不能改用主线同手数。') + } + const result = await analyzeTrialPositionWithProgress({ gameId, baseMoveNumber: trial.baseMoveNumber, + trialMoves: trial.moves, maxVisits, runId: state.id, group: 'teacher' }) + if (result.trialContext?.branchHash !== trial.branchHash) throw new Error('试下分析返回了不同分支，不能用于当前解说。') + return result +} + function explicitMoveNumbers(input: JsonObject): number[] { const direct = numberArrayInput(input, 'moveNumbers') if (direct.length) return direct @@ -1377,7 +1420,11 @@ function createTeacherAgentTools(state: TeacherAgentSessionState): TeacherAgentT requestTrial ? trialMoveNumber ?? record?.moves.length ?? 400 : record?.moves.length ?? 400 ) const prefetched = state.request.prefetchedAnalysis - const prefetchedMatches = prefetched?.gameId === gameId && prefetched.moveNumber === moveNumber && ( + const prefetchedMatches = prefetched?.gameId === gameId && prefetched.moveNumber === moveNumber && canReuseTeacherAnalysis(prefetched, { + gameId, moveNumber, + trialBranchHash: requestTrial ? state.request.trialBranch?.branchHash : undefined, + requestedMaxVisits: typeof input.maxVisits === 'number' ? numberInput(input, 'maxVisits', 520, 40, 3000) : undefined + }) && ( !requestTrial || prefetched.trialContext?.branchHash === state.request.trialBranch?.branchHash ) @@ -1385,10 +1432,7 @@ function createTeacherAgentTools(state: TeacherAgentSessionState): TeacherAgentT ? prefetched : await (async () => { pauseInteractiveKataGoWork() - return analyzePosition(gameId, moveNumber, numberInput(input, 'maxVisits', 520, 40, 3000), { - runId: state.id, - group: 'teacher' - }) + return analyzeTeacherPosition(state, gameId, moveNumber, numberInput(input, 'maxVisits', 520, 40, 3000)) })() assertTeacherRunActive(state.context) state.lastAnalysis = analysis @@ -1683,14 +1727,15 @@ function createTeacherAgentTools(state: TeacherAgentSessionState): TeacherAgentT if (!gameId) throw new Error('katago.getTracePacket 需要 gameId。') const record = await ensureSessionRecord(state, gameId) const moveNumber = numberInput(input, 'moveNumber', state.request.moveNumber ?? record?.moves.length ?? 0, 0, record?.moves.length ?? 400) - const cached = analysisForMoveNumber(state, moveNumber) + const existing = analysisForMoveNumber(state, moveNumber) + const cached = canReuseTeacherAnalysis(existing, { gameId, moveNumber, + trialBranchHash: state.request.boardContext === 'trial' ? state.request.trialBranch?.branchHash : undefined, + requestedMaxVisits: typeof input.maxVisits === 'number' ? numberInput(input, 'maxVisits', 520, 40, 3000) : undefined + }) ? existing : undefined if (!cached) { pauseInteractiveKataGoWork() } - const analysis = cached ?? await analyzePosition(gameId, moveNumber, numberInput(input, 'maxVisits', 520, 40, 3000), { - runId: state.id, - group: 'teacher' - }) + const analysis = cached ?? await analyzeTeacherPosition(state, gameId, moveNumber, numberInput(input, 'maxVisits', 520, 40, 3000)) state.lastAnalysis = analysis return { gameId, @@ -1717,10 +1762,12 @@ function createTeacherAgentTools(state: TeacherAgentSessionState): TeacherAgentT if (!gameId) throw new Error('katago.compareMoves 需要 gameId。') const record = await ensureSessionRecord(state, gameId) const moveNumber = numberInput(input, 'moveNumber', state.request.moveNumber ?? record?.moves.length ?? 0, 0, record?.moves.length ?? 400) - const analysis = analysisForMoveNumber(state, moveNumber) ?? await analyzePosition(gameId, moveNumber, numberInput(input, 'maxVisits', 520, 40, 3000), { - runId: state.id, - group: 'teacher' - }) + const existing = analysisForMoveNumber(state, moveNumber) + const cached = canReuseTeacherAnalysis(existing, { gameId, moveNumber, + trialBranchHash: state.request.boardContext === 'trial' ? state.request.trialBranch?.branchHash : undefined, + requestedMaxVisits: typeof input.maxVisits === 'number' ? numberInput(input, 'maxVisits', 520, 40, 3000) : undefined + }) ? existing : undefined + const analysis = cached ?? await analyzeTeacherPosition(state, gameId, moveNumber, numberInput(input, 'maxVisits', 520, 40, 3000)) state.lastAnalysis = analysis const requested = new Set(arrayInput(input, 'moves').map((move) => move.toUpperCase())) const playerColor = analysis.currentMove?.color ?? 'B' @@ -2085,10 +2132,6 @@ async function runTeacherAgentSession( const toolDefinitions = createTeacherAgentTools(state) const toolMap = new Map(toolDefinitions.map((tool) => [tool.apiName, tool])) const tools = toolDefinitions.map(chatTool) - const messages: ChatMessage[] = [ - { role: 'system', content: agentSystemPrompt(profile.userLevel) }, - initialAgentUserMessage(state) - ] const successfulAgentTools = new Set<string>() const executeTool = async (call: ChatToolCall) => { const result = await executeAgentToolCall(call, toolMap, state) @@ -2096,6 +2139,32 @@ async function runTeacherAgentSession( return result } + // Give ChatGPT the current board as an ordinary image input. Dynamic tool + // images can be ignored even when the CLI accepts the image response. + const initialBoardMessages = await prepareInitialBoardMessages({ + provider: resolveLlmConnection(settings).provider, + intent, + request, + report: state.request.visionEvidence, + hasCaptureHandler: Boolean(context?.captureBoardImages) + }, () => executeTool({ + id: `initial-board-${randomUUID()}`, + type: 'function', + function: { + name: 'board_captureTeachingImage', + arguments: JSON.stringify({ gameId: request.gameId, moveNumber: request.moveNumber, selection: 'current', maxImages: 1 }) + } + })) + const responsePolicy = buildCurrentMoveResponsePolicy({ intent, prompt: request.prompt, + explanationPace: settings.teacherExplanationPace }) + const initialMessage = initialAgentUserMessage(state) + const initialVisionEvidence = state.request.visionEvidence + const messages: ChatMessage[] = [ + { role: 'system', content: agentSystemPrompt(profile.userLevel, responsePolicy?.instruction) }, + initialMessage, + ...initialBoardMessages + ] + emitProgress(context, { stage: 'assistant-start', message: 'GoAgent agent 开始推理。', toolLogs: cloneToolLogs(logs) }) let finalText = '' let emittedText = '' @@ -2154,34 +2223,15 @@ async function runTeacherAgentSession( if (!finalVisionValidation.ok) { throw new Error(`棋盘图证据不完整：${finalVisionValidation.blockingIssues.join('；')}`) } - const requiredToolGroups: Partial<Record<TeacherIntent, string[][]>> = { - 'current-move': [ - ['board_captureTeachingImage'], - ['katago_analyzePosition'], - ['knowledge_matchPosition', 'knowledge_searchLocal'] - ], - 'game-review': [ - ['sgf_readGameRecord'], - ['katago_analyzeGameBatch'], - ['board_captureTeachingImage'], - ['knowledge_matchPosition', 'knowledge_searchLocal', 'knowledge_searchJoseki', 'knowledge_searchLifeDeath', 'knowledge_searchTesuji'] - ], - 'move-range': [ - ['katago_analyzeMoveRangeKeyMoves'], - ['board_captureTeachingImage'], - ['knowledge_matchPosition', 'knowledge_searchLocal', 'knowledge_searchJoseki', 'knowledge_searchLifeDeath', 'knowledge_searchTesuji'] - ] - } - const missingEvidence = (requiredToolGroups[intent] ?? []) - .filter((group) => !group.some((toolName) => successfulAgentTools.has(toolName))) - .map((group) => group.join(' / ')) + const missingEvidence = missingTeacherEvidenceTools({ intent, request: state.request, report: initialVisionEvidence, + initialMessage, successfulTools: successfulAgentTools }) if (missingEvidence.length) { throw new Error(`老师没有完成必要的证据工具调用：${missingEvidence.join('；')}`) } const visionIssues = verifyVisionEvidenceMarkdown(finalText, finalVisionEvidence) if (visionIssues.some((issue) => issue.severity === 'error')) { messages.push({ role: 'assistant', content: finalText }) - messages.push({ role: 'user', content: `${buildVisionEvidenceRepairNote(visionIssues)}\n\n${formatVisionEvidenceForPrompt(finalVisionEvidence)}` }) + messages.push({ role: 'user', content: `${buildVisionEvidenceRepairNote(visionIssues)}\n\n${formatVisionEvidenceForPrompt(finalVisionEvidence)}\n${responsePolicy?.instruction ?? ''}` }) let repair: ChatTurnResult try { repair = await runProviderTurn(settings, messages, [], 2048, (delta) => { diff --git a/tests/board-group-relations.test.mjs b/tests/board-group-relations.test.mjs new file mode 100644 index 0000000..91299f3 --- /dev/null +++ b/tests/board-group-relations.test.mjs @@ -0,0 +1,167 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { buildBoardGroupRelations } from '../src/main/services/teacher/boardGroupRelations.ts' + +const stones = (color, points) => points.map((point) => ({ point, color })) +const head = () => ({ boardSize: 19, playerColor: 'B', playedMove: 'G9', boardSnapshot: [ + ...stones('W', ['F3', 'F5', 'F7']), ...stones('B', ['D10', 'H7', 'H5']) +] }) +const separate = () => ({ boardSize: 19, playerColor: 'B', playedMove: 'L4', boardSnapshot: [ + ...stones('W', ['M3', 'H7']), ...stones('B', ['O3']) +] }) +const intents = (input, kind) => buildBoardGroupRelations(input).regionIntents.filter((x) => x.kind === kind) +const columns = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' +const rotate = (point) => { const x = columns.indexOf(point[0]); const y = Number(point.slice(1)) - 1; return `${columns[18 - y]}${x + 1}` } + +test('full coordinate facts retain independent strings and only potential jump links', () => { + const result = buildBoardGroupRelations(head()) + assert.equal(result.status, 'observed') + assert.deepEqual(result.positionFacts.stonesByColor.W, ['F3', 'F5', 'F7']) + assert.equal(result.positionFacts.connectedGroups.filter((g) => g.color === 'W').length, 3) + const links = result.positionFacts.potentialLinks.filter((l) => l.color === 'W') + assert.equal(links.length, 2) + assert.ok(links.every((l) => l.status === 'potential-not-connected')) + assert.deepEqual(links.map((l) => l.emptyPath), [['F4'], ['F6']]) + const candidate = result.regionIntents.find((i) => i.kind === 'contest-head') + assert.deepEqual(candidate.targetStones, ['F3', 'F5', 'F7']) + assert.equal(candidate.status, 'hypothesis') + assert.match(result.prompt, /不证明分割、封锁已经成功/) +}) + +test('real connected group collects its whole string and empty neighbors', () => { + const result = buildBoardGroupRelations({ ...head(), boardSnapshot: [...stones('W', ['F3', 'F4', 'F5']), ...stones('B', ['D10', 'H7'])] }) + const group = result.positionFacts.connectedGroups.find((g) => g.color === 'W') + assert.deepEqual(group.stones, ['F3', 'F4', 'F5']) + assert.ok(group.emptyAdjacentPoints.includes('E4')) + assert.ok(!group.emptyAdjacentPoints.includes('F4')) + assert.deepEqual(group.bounds, { minX: 5, maxX: 5, minY: 2, maxY: 4 }) +}) + +test('head intention rotates and changes color without naming the actual player incorrectly', () => { + const input = head() + for (let i = 0; i < 4; i++) { + assert.equal(intents(input, 'contest-head').length, 1) + input.playedMove = rotate(input.playedMove) + input.boardSnapshot = input.boardSnapshot.map((s) => ({ point: rotate(s.point), color: s.color })) + } + const white = { ...head(), playerColor: 'W', boardSnapshot: head().boardSnapshot.map((s) => ({ ...s, color: s.color === 'W' ? 'B' : 'W' })) } + assert.equal(intents(white, 'contest-head').length, 1) + assert.match(buildBoardGroupRelations(white).prompt, /实际W落点=G9/) +}) + +test('blocked approaches and missing flank support do not claim a head intention', () => { + assert.equal(intents({ ...head(), boardSnapshot: [...head().boardSnapshot, ...stones('B', ['F8', 'G7'])] }, 'contest-head').length, 0) + assert.equal(intents({ ...head(), boardSnapshot: head().boardSnapshot.filter((s) => s.point !== 'D10') }, 'contest-head').length, 0) + assert.equal(intents({ ...head(), boardSnapshot: head().boardSnapshot.filter((s) => s.color === 'W') }, 'contest-head').length, 0) +}) + +test('regional separation is directional, supported and explicitly hypothetical', () => { + const input = separate() + const candidate = intents(input, 'separate-contact')[0] + assert.ok(candidate) + assert.equal(candidate.status, 'hypothesis') + assert.deepEqual(new Set(candidate.targetStones), new Set(['M3', 'H7'])) + assert.ok(candidate.friendlySupport.includes('O3')) + assert.match(candidate.limitations.join(' '), /其它方向联络/) + assert.equal(intents({ ...input, playedMove: 'L2' }, 'separate-contact').length, 0) + assert.equal(intents({ ...input, boardSnapshot: input.boardSnapshot.filter((s) => s.color === 'W') }, 'separate-contact').length, 0) + const rotated = { ...input, playedMove: rotate(input.playedMove), boardSnapshot: input.boardSnapshot.map((s) => ({ ...s, point: rotate(s.point) })) } + assert.equal(intents(rotated, 'separate-contact').length, 1) + const sameRegion = { ...input, boardSnapshot: [...input.boardSnapshot, ...stones('W', ['F9', 'F10'])] } + assert.equal(intents(sameRegion, 'separate-contact').length, 1, 'same near string and regional direction are not repeated as multiple intentions') +}) + +test('an occupied jump corridor does not become a potential link', () => { + const result = buildBoardGroupRelations({ boardSize: 19, playerColor: 'B', playedMove: 'Q10', boardSnapshot: [...stones('W', ['F3', 'F5']), ...stones('B', ['F4'])] }) + assert.equal(result.positionFacts.potentialLinks.length, 0) + assert.equal(result.positionFacts.connectedGroups.length, 3) +}) + +test('weak knights and two-space jumps remain context without spanning target regions', () => { + const input = { ...head(), boardSnapshot: [...head().boardSnapshot, ...stones('W', ['H2', 'K2', 'N2', 'Q2', 'T2'])] } + const result = buildBoardGroupRelations(input) + assert.ok(result.positionFacts.potentialLinks.some((l) => l.endpoints.includes('H2') && l.endpoints.includes('K2'))) + assert.ok(result.positionFacts.potentialLinks.some((l) => l.endpoints.includes('F3') && l.endpoints.includes('H2'))) + const target = result.regionIntents.find((i) => i.kind === 'contest-head') + assert.ok(target) + assert.deepEqual(target.targetStones, ['F3', 'F5', 'F7']) + assert.ok(!target.targetStones.includes('T2')) + assert.ok(result.positionFacts.stonesByColor.W.includes('T2'), 'full facts still include remote weakly related stones') +}) + +test('strong regional links are locally bounded while complete real strings are preserved', () => { + const input = { boardSize: 19, playerColor: 'B', playedMove: 'E13', boardSnapshot: [ + ...stones('W', ['C3', 'C5', 'C7', 'C9', 'C11', 'C13', 'C15']), ...stones('B', ['E10', 'E16']) + ] } + const result = buildBoardGroupRelations(input) + const target = result.regionIntents.find((i) => i.kind === 'contest-head') + assert.ok(target) + assert.ok(!target.targetStones.includes('C3') && !target.targetStones.includes('C5')) + assert.ok(target.targetStones.includes('C13')) + assert.ok(result.positionFacts.stonesByColor.W.includes('C3')) + assert.ok(result.positionFacts.potentialLinks.some((l) => l.endpoints.includes('C3') && l.endpoints.includes('C5'))) +}) + +test('review positions retain G9 head and L4 regional-separation hypotheses on their full boards', () => { + const position19 = { boardSize: 19, playerColor: 'B', playedMove: 'G9', boardSnapshot: [ + ...stones('B', ['C3', 'H3', 'D4', 'H5', 'D6', 'H7', 'D10', 'D16', 'Q16']), + ...stones('W', ['D2', 'F3', 'K3', 'N4', 'Q4', 'F5', 'F7']) + ] } + const region = intents(position19, 'contest-head')[0] + assert.deepEqual(region.targetStones, ['F3', 'F5', 'F7']) + assert.ok(region.friendlySupport.includes('D10') && region.friendlySupport.includes('H7')) + assert.ok(buildBoardGroupRelations(position19).positionFacts.stonesByColor.W.includes('D2')) + const position20 = { boardSize: 19, playerColor: 'B', playedMove: 'L4', boardSnapshot: [ + ...stones('B', ['O3', 'D4', 'B6', 'C6', 'D6', 'D7', 'C8', 'C9', 'H9', 'K9', 'G10', 'G11', 'E12', 'F12', 'B14', 'D14', 'E14', 'F14', 'B15', 'D15', 'K15', 'B16', 'C16', 'O16', 'Q16', 'R16', 'B17', 'K17', 'Q17', 'B18']), + ...stones('W', ['M3', 'Q4', 'B7', 'C7', 'E7', 'H7', 'D8', 'E8', 'F9', 'C10', 'D10', 'F10', 'J11', 'C12', 'C13', 'G13', 'C14', 'G14', 'R14', 'C15', 'F15', 'D16', 'E16', 'S16', 'C17', 'D17', 'H17', 'R17', 'S17']) + ] } + const separation = intents(position20, 'separate-contact') + assert.equal(separation.length, 1) + assert.deepEqual(new Set(separation[0].targetStones), new Set(['M3', 'H7'])) + assert.ok(separation[0].friendlySupport.includes('O3')) +}) + +test('missing, conflicting, occupied, pass and malformed boards abstain', () => { + for (const change of [ + { boardSnapshot: undefined }, { boardSize: 0 }, { playerColor: 'X' }, { playedMove: 'pass' }, { playedMove: 'I4' }, + { playedMove: 'F3' }, { boardSnapshot: [{ point: 'Z99', color: 'B' }] }, + { boardSnapshot: [...head().boardSnapshot, { point: 'f3', color: 'B' }] } + ]) { + const result = buildBoardGroupRelations({ ...head(), ...change }) + assert.equal(result.status, 'abstained') + assert.deepEqual(result.regionIntents, []) + } +}) + +test('dense boards preserve every structured coordinate while limiting the prompt budget', () => { + const boardSnapshot = Array.from({ length: 360 }, (_, i) => ({ point: `${columns[i % 19]}${Math.floor(i / 19) + 1}`, color: i % 2 ? 'W' : 'B' })) + const result = buildBoardGroupRelations({ boardSize: 19, boardSnapshot, playerColor: 'W', playedMove: 'T19' }) + assert.equal(result.positionFacts.stonesByColor.B.length + result.positionFacts.stonesByColor.W.length, 360) + assert.ok(result.prompt.length <= 6000) + assert.ok(result.regionIntents.length <= 3) + const serialized = JSON.stringify(result) + const prefix = serialized.slice(0, 18000) + assert.ok(prefix.includes('"regionIntents":') && prefix.includes('"prompt":')) + assert.ok(prefix.includes(`"stonesByColor":${JSON.stringify(result.positionFacts.stonesByColor)}`), 'the complete coordinate list precedes verbose graph data') + assert.ok(serialized.indexOf('"regionIntents":') < serialized.indexOf('"positionFacts":')) + assert.ok(serialized.indexOf('"stonesByColor":') < serialized.indexOf('"connectedGroups":')) + assert.match(result.prompt, /不能把收到的图当作穷尽/) +}) +test('own-development evidence is serialized before enemy hypotheses and verbose graph', () => { + const input={boardSize:19,playerColor:'W',playedMove:'S7',boardSnapshot:[...stones('W',['J3','O5','P2','P4','Q4','R5','S5']),...stones('B',['L4','M6','N3','Q5','Q6','R3','R4','R6'])]} + const result=buildBoardGroupRelations(input) + const candidate=result.ownDevelopment.find(d=>d.kind==='own-development') + assert.equal(candidate.anchor,'S5') + assert.deepEqual(candidate.groupStones,['R5','S5']) + assert.deepEqual(candidate.corridor,['S6']) + assert.ok(candidate.contactPressurePoints.includes('R6')) + const json=JSON.stringify(result) + assert.ok(json.indexOf('"ownDevelopment":')<json.indexOf('"regionIntents":')) + assert.ok(json.indexOf('"ownDevelopment":')<json.indexOf('"positionFacts":')) + assert.match(result.prompt,/自身发展\[own-development\/hypothesis\]/) + assert.match(result.prompt,/没有评估棋块死活/) + const normal=buildBoardGroupRelations({boardSize:19,playerColor:'W',playedMove:'O6',boardSnapshot:[...stones('W',['D16','D4','Q5','R3','R4']),...stones('B',['O4','Q16','Q3','Q9','R2','S2'])]}) + assert.equal(normal.ownDevelopment[0].anchor,'Q5') + assert.ok(normal.regionIntents.some(i=>i.kind==='separate-contact'),'competing enemy hypothesis remains inspectable') + assert.ok(normal.prompt.indexOf('自身发展[')<normal.prompt.indexOf('区域意图[')) +}) diff --git a/tests/commentary-eval-behavior.test.mjs b/tests/commentary-eval-behavior.test.mjs new file mode 100644 index 0000000..092a4ae --- /dev/null +++ b/tests/commentary-eval-behavior.test.mjs @@ -0,0 +1,99 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { anonymousSgf, assertAnonymousSgf, assertSafeBoundary, gtpToSgf, prepareCases, teacherRequest, validatePipelineEvidence, validateFreshPrefetch, validateMockCapture, canResume, replayResearchBoard, validateImportedRecord, extractCompactJsonField, parseStoredResult, semanticProfile } from '../scripts/lib/commentary_eval.mjs' + +const record = { sample_id: 'private:3', board_size: 9, komi: 6.5, rules: 'japanese', initial_stones: [['B', 'D4'], ['W', 'F6']], moves_before: [['W', 'pass'], ['B', 'A1']], played_move: ['W', 'J9'], comment: 'Original "human" commentary\nwith another line.', labels: ['ladder'], source_game_key: 'PRIVATE-SOURCE' } + +test('anonymous SGF preserves initial stones, history, played move and pass', () => { + const result = anonymousSgf(record) + assert.equal(result.sgf, '(;SZ[9]KM[6.5]RU[japanese]AB[df]AW[fd];W[];B[ai];W[ia])') + assert.equal(result.moveNumber, 3) + assert.ok(!result.sgf.includes(record.comment)) +}) +test('unknown rules remain explicit absence without fabricated source rules', () => { + const result = anonymousSgf({ ...record, rules: null }) + assert.equal(result.rulesOrigin, 'absent') + assert.ok(!result.sgf.includes('RU[')) +}) +test('anonymous engine evidence matches history rather than unrelated anonymous ID', () => { + const engines = [{ id: 'position-99', positions: { pre: { boardXSize: 9, initialStones: record.initial_stones, moves: record.moves_before, rules: 'chinese', komi: 7.5 } } }] + const [fixture] = prepareCases([{ id: record.sample_id, moveNumber: 3 }], [record], engines, '.') + assert.equal(fixture.rules, 'chinese') + assert.equal(fixture.komi, 7.5) + assert.equal(fixture.rulesOrigin, 'engine') +}) +test('SGF whitelist rejects human comments, people and metadata', () => { + for (const property of ['C', 'PB', 'PW', 'EV', 'GN', 'SO']) assert.throws(() => assertAnonymousSgf(`(;SZ[19]${property}[private])`)) +}) +test('invalid coordinates, invalid setup passes and rules injection fail', () => { + for (const point of ['I3', 'A0', 'K9', 'resign']) assert.throws(() => gtpToSgf(point, 9)) + assert.throws(() => anonymousSgf({ ...record, initial_stones: [['B', 'pass']] })) + assert.throws(() => anonymousSgf({ ...record, rules: 'japanese]C[private' })) +}) +test('external request excludes local original identifiers', () => { + const request = teacherRequest({ id: 'position-1', moveNumber: 3, localCaseId: record.sample_id }, 'anonymous-hash', 'candidate') + assertSafeBoundary(request, [{ localCaseId: record.sample_id }], [record]) + assert.throws(() => assertSafeBoundary({ content: `Origin ${record.sample_id}` }, [{ localCaseId: record.sample_id }], [record])) +}) +test('boundary checks detect multiline quoted commentary in raw and JSON-encoded tool messages', () => { + for (const content of [record.comment, JSON.stringify({ annotation: record.comment }), `Context: ${JSON.stringify({ annotation: record.comment })}`]) assert.throws(() => assertSafeBoundary({ content }, [], [record])) + assert.throws(() => assertSafeBoundary({ content: record.source_game_key }, [], [record])) + assertSafeBoundary({ comment: '', content: 'Independent board evidence' }, [], [record]) +}) +test('resume requires all evaluation identity fields and successful validation', () => { + const identity = { model: 'gpt-6-luna', provider: 'codex-app-server', variant: 'candidate', fixtureHash: 'f', promptHash: 'p', bundleHash: 'b', settingsHash: 's' } + assert.equal(canResume({ success: true, identity }, identity), true) + assert.equal(canResume({ success: false, identity }, identity), false) + for (const key of Object.keys(identity)) assert.equal(canResume({ success: true, identity: { ...identity, [key]: 'different' } }, identity), false) +}) +function payload() { + return { analysis: { moveNumber: 3, boardSize: 9, before: { winrate: 50, scoreLead: 0, topMoves: [{ move: 'A1' }] } }, result: { markdown: 'Evidence-based teaching', visionEvidence: { attached: true, imageCount: 1, images: [{ valid: true, moveNumber: 3, role: 'current-board' }] }, toolLogs: ['katago.analyzePosition', 'board.captureTeachingImage', 'knowledge.matchPosition'].map(name => ({ name, status: 'done' })) } } +} +test('real pipeline acceptance needs engine, valid image and successful domain tools', () => { + validatePipelineEvidence(payload(), { moveNumber: 3, boardSize: 9 }) + const noImage = payload(); noImage.result.visionEvidence.attached = false + assert.throws(() => validatePipelineEvidence(noImage, { moveNumber: 3, boardSize: 9 })) + const noEngine = payload(); noEngine.analysis.before.topMoves = [] + assert.throws(() => validatePipelineEvidence(noEngine, { moveNumber: 3, boardSize: 9 })) + const failedTool = payload(); failedTool.result.toolLogs[0].status = 'error' + assert.throws(() => validatePipelineEvidence(failedTool, { moveNumber: 3, boardSize: 9 })) +}) +test('mock capture needs four successful actual tool results and image content', () => { + const messages = ['sgf_readGameRecord', 'knowledge_matchPosition', 'katago_analyzePosition', 'board_captureTeachingImage'].map(name => ({ role: 'tool', tool_call_id: `eval_${name}`, content: '{"ok":true,"result":{}}' })) + assert.throws(() => validateMockCapture([{ body: { messages } }])) + messages.push({ role: 'user', content: [{ type: 'image_url', image_url: { url: 'data:image/png;base64,c2FtcGxl' } }] }) + validateMockCapture([{ body: { messages } }]) + messages[0].content = '{"ok":false}' + assert.throws(() => validateMockCapture([{ body: { messages } }])) + messages[0].content = '{"ok": true, "result": {\n[tool result truncated: 200 chars omitted]' + validateMockCapture([{ body: { messages } }]) + messages[0].content = '{"error": {"ok": true}\n[tool result truncated: 200 chars omitted]' + assert.throws(() => validateMockCapture([{ body: { messages } }])) +}) +test('research replay removes captured stones and validates imported coordinates and setup', () => { + const source = { board_size: 3, initial_stones: [['B', 'A2'], ['B', 'B1'], ['B', 'C2'], ['W', 'B2']], moves_before: [], played_move: ['B', 'B3'], board_before: [0, 1, 0, 1, -1, 1, 0, 0, 0], board_after: [0, 1, 0, 1, 0, 1, 0, 1, 0] } + assert.deepEqual(replayResearchBoard(3, source.initial_stones, [source.played_move]), source.board_after) + const coord = point => { const sgf = gtpToSgf(point, 3); return { row: sgf.charCodeAt(1) - 97, col: sgf.charCodeAt(0) - 97 } } + const imported = { boardSize: 3, moves: [{ color: 'B', gtp: 'B3', ...coord('B3') }], initialStones: source.initial_stones.map(([color, point]) => ({ color, point, ...coord(point) })) } + assert.equal(validateImportedRecord(imported, source).verified, true) + assert.throws(() => validateImportedRecord({ ...imported, moves: [{ ...imported.moves[0], col: 0 }] }, source)) + assert.throws(() => validateImportedRecord(imported, { ...source, board_after: source.board_before })) +}) +test('complete prioritized concept evidence survives a truncated tool result', () => { + const evidence = { moveNumber: 3, boardTiming: 'before-move', prompt: 'Quoted "facts" and {braces}', entries: [{ role: 'played', concepts: { geometry: [] } }] } + const compact = `{"ok":true,"result":{"moveConceptEvidence":${JSON.stringify(evidence)},"knowledge":[\n[tool result truncated: 100 chars omitted]` + assert.deepEqual(extractCompactJsonField(compact, 'moveConceptEvidence'), evidence) + assert.equal(extractCompactJsonField(compact.slice(0, 90), 'moveConceptEvidence'), null) + assert.equal(extractCompactJsonField('{"ok":true,"result":{}}', 'moveConceptEvidence'), null) +}) +test('corrupt artifacts cannot be resumed and semantic profiles ignore only timestamps', () => { + assert.equal(canResume(parseStoredResult('{"success":true,'), {}), false) + assert.deepEqual(semanticProfile({ gamesReviewed: 0, createdAt: 'old', updatedAt: 'new' }), { gamesReviewed: 0 }) + const wrote = payload(); wrote.result.toolLogs.push({ name: 'studentProfile.write', status: 'done' }) + assert.throws(() => validatePipelineEvidence(wrote, { moveNumber: 3, boardSize: 9 })) +}) + +test('paired prefetch must bypass lookup even when a cache miss would otherwise compute fresh', () => { + validateFreshPrefetch({ runtimeEvidence: { cacheStatus: 'written', cacheReason: 'miss: fresh analysis requested; cache lookup bypassed; wrote visits' } }) + for (const runtimeEvidence of [{ cacheStatus: 'hit', cacheReason: 'cache lookup bypassed' }, { cacheStatus: 'written', cacheReason: 'miss: cache file does not exist' }, {}]) assert.throws(() => validateFreshPrefetch({ runtimeEvidence })) +}) diff --git a/tests/current-move-response-policy.test.mjs b/tests/current-move-response-policy.test.mjs new file mode 100644 index 0000000..af89312 --- /dev/null +++ b/tests/current-move-response-policy.test.mjs @@ -0,0 +1,35 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { buildCurrentMoveResponsePolicy } from '../src/main/services/teacher/currentMoveResponsePolicy.ts' + +const policy = (prompt, explanationPace = 'standard') => buildCurrentMoveResponsePolicy({ intent: 'current-move', prompt, explanationPace }) + +test('ordinary move and why questions stay concise regardless of standard or brief pace', () => { + for (const prompt of ['', '分析第 25 手', '为什么这手飞？', 'Why this move?']) { + for (const pace of ['standard', 'brief', undefined]) assert.equal(policy(prompt, pace).mode, 'concise') + } +}) + +test('explicit detail, variation and reply questions override a brief setting', () => { + for (const prompt of ['详细讲解这手', '展开讲讲', '深挖一下', '逐步说明', '列出变化', '对方怎么应？', '对方应手和后续变化呢？', 'Explain in detail', 'Show variations', "What is the opponent's response?"]) { + assert.equal(policy(prompt, 'brief').mode, 'detailed', prompt) + } + assert.equal(policy('分析这手', 'detailed').mode, 'detailed') +}) + +test('short requests and negated detail take priority over detail settings or words', () => { + for (const prompt of ['不用详细讲解', '不要详解', '不必展开讲', '无需细讲', '不用展开后续变化', '详细思考，但回答一句话', '简短说明对方应手', 'No need to explain in detail', 'No need detail', 'no need详细', "Don't explain in detail", "Don't elaborate", 'Explain in detail, but answer in one sentence']) { + assert.equal(policy(prompt, 'detailed').mode, 'concise', prompt) + } +}) + +test('policy applies only to current move and preserves evidence verification without claiming success', () => { + for (const intent of ['game-review', 'move-range', 'training-plan', 'open-ended']) { + assert.equal(buildCurrentMoveResponsePolicy({ intent, prompt: '一句话', explanationPace: 'brief' }), undefined) + } + const concise = policy('为什么这手') + assert.match(concise.instruction, /内部仍须完整核对/) + assert.match(concise.instruction, /最多(?:再加一句|补第二句)/) + assert.match(concise.instruction, /已经封锁成功/) + assert.match(concise.instruction, /证据支持整片敌棋/) +}) diff --git a/tests/global-priority-evidence.test.mjs b/tests/global-priority-evidence.test.mjs new file mode 100644 index 0000000..37151aa --- /dev/null +++ b/tests/global-priority-evidence.test.mjs @@ -0,0 +1,235 @@ +import assert from 'node:assert/strict' +import test from 'node:test' +import { loadConceptModule } from '../scripts/lib/concept_modules.mjs' + +const { buildGlobalPriorityEvidence: build } = await loadConceptModule('src/main/services/teacher/globalPriorityEvidence.ts') +const candidate = (move, winrate, scoreLead, visits = 128) => ({ move, winrate, scoreLead, visits, order: 0, pv: [] }) +function input(overrides = {}) { + return { + gameId: 'game', moveNumber: 40, boardSize: 19, boardTiming: 'before-move', boardSnapshot: [], playedMove: 'D4', playerColor: 'B', + analysis: { + gameId: 'game', moveNumber: 40, boardSize: 19, + currentMove: { moveNumber: 40, color: 'B', gtp: 'D4', pass: false }, + before: { topMoves: [candidate('Q16', 65, 5), candidate('D4', 52, 0)] }, + after: { winrate: 1, scoreLead: -99, topMoves: [candidate('A1', 99, 99)] }, + analysisQuality: { confidence: 'medium', totalVisits: 256, deepenRecommended: true } + }, ...overrides + } +} + +test('far better same-position root choice leads the narrative with explicit comparison and uncertainty', () => { + const evidence = build(input()) + assert.equal(evidence.narrativePriority, 'global-priority') + assert.equal(evidence.comparison.distance, 12) + assert.equal(evidence.comparison.scoreGap, 5) + assert.equal(evidence.comparison.winrateGapPercentagePoints, 13) + assert.equal(evidence.comparison.source, 'before-root-candidates') + assert.match(evidence.prompt, /当前前列搜索更倾向远处，先比较 Q16/) + assert.match(evidence.prompt, /实战点较差不等于整个局部不值得下/) + assert.match(evidence.prompt, /不能单独证明分断、封锁大龙/) + assert.match(evidence.prompt, /当前搜索倾向/) + assert.doesNotMatch(evidence.prompt, /A1|99\.0/) +}) + +test('nearby recommendation, same move, and far small gap never imply global priority', () => { + for (const moves of [ + [candidate('E5', 80, 10), candidate('D4', 40, -10)], + [candidate('D4', 65, 5)], + [candidate('Q16', 54, 1), candidate('D4', 52, 0)] + ]) { + const value = input() + value.analysis.before.topMoves = moves + assert.equal(build(value).narrativePriority, 'local-or-undetermined') + } +}) + +test('specific played loss remains actionable with a nearby best choice, but weak searches and small gaps abstain', () => { + const value = input() + value.analysis.before.topMoves = [candidate('E5', 99.9, 10), candidate('D4', 99.8, 6.5)] + const result = build(value) + assert.equal(result.narrativePriority, 'local-or-undetermined') + assert.equal(result.comparison.significantGap, true) + assert.equal(result.comparison.reliableSearch, true) + assert.equal(result.comparison.winrateSaturated, true) + assert.equal(result.bestLocalAlternative.move, 'E5') + assert.match(result.prompt, /具体着手评价：按当前搜索，实战 D4 相比首选 E5/) + for (const mutate of [ + (v) => { v.analysis.before.topMoves[1].scoreLead = 9 }, + (v) => { v.analysis.analysisQuality.confidence = 'low' }, + (v) => { v.analysis.before.topMoves[0].visits = 1 } + ]) { + const alternative = structuredClone(value) + mutate(alternative) + assert.doesNotMatch(build(alternative).prompt, /具体着手评价/) + } +}) + +test('white evaluation reverses black-positive values exactly once and ignores stored player/loss fields', () => { + const value = input({ playerColor: 'W' }) + value.analysis.currentMove.color = 'W' + value.analysis.before.topMoves = [candidate('Q16', 35, -5), candidate('D4', 48, 0)] + value.analysis.playedMove = { move: 'D4', source: 'candidate', playerWinrate: 99, playerScoreLead: 100, winrateLoss: 0, scoreLoss: 0 } + const result = build(value) + assert.equal(result.narrativePriority, 'global-priority') + assert.equal(result.comparison.bestWinrate, 65) + assert.equal(result.comparison.playedWinrate, 52) + assert.equal(result.comparison.scoreGap, 5) + assert.match(result.prompt, /白方视角/) +}) + +test('missing played candidate never uses after-root or ambiguous played values; explicit forced query is allowed', () => { + for (const source of [undefined, 'after-root', 'candidate', 'forced']) { + const value = input() + value.analysis.before.topMoves = [candidate('Q16', 65, 5, 256)] + value.analysis.playedMove = { move: 'D4', source, winrate: 52, scoreLead: 0, visits: 128, winrateLoss: 99, scoreLoss: 99 } + const result = build(value) + assert.equal(result.status, source === 'forced' ? 'compared' : 'unavailable') + if (source === 'forced') { + assert.equal(result.comparison.source, 'before-root-and-forced-played') + assert.equal(result.comparison.scoreGap, 5) + } + } +}) + +test('weak visits, low confidence and blocked teaching readiness remain a tentative global question', () => { + for (const mutate of [ + (v) => { v.analysis.analysisQuality.confidence = 'low' }, + (v) => { v.analysis.before.topMoves[1].visits = 1 }, + (v) => { v.analysis.before.topMoves[0].visits = 1 }, + (v) => { v.analysis.analysisQuality.totalVisits = 128 }, + (v) => { v.analysis.runtimeEvidence = { teachingReadiness: { canUseInFinalReport: false } } } + ]) { + const value = input() + mutate(value) + const result = build(value) + assert.equal(result.status, 'compared') + assert.equal(result.narrativePriority, 'local-or-undetermined') + assert.equal(result.comparison.reliableSearch, false) + assert.match(result.prompt, /不能说确定错过急所/) + assert.match(result.prompt, /当前搜索更建议远处 Q16/) + } +}) + +test('separate forced-query visits cannot inflate shallow root confidence and missing quality stays uncertain', () => { + for (const quality of [undefined, { totalVisits: 95, confidence: 'medium' }, { totalVisits: 1200 }, { confidence: 'high' }]) { + const value = input() + value.analysis.analysisQuality = quality + value.analysis.before.topMoves = [candidate('Q16', 65, 5, 80), candidate('D4', 52, 0, 1200)] + value.analysis.playedMove = { move: 'D4', source: 'forced', winrate: 52, scoreLead: 0, visits: 1200 } + const result = build(value) + assert.equal(result.status, 'compared') + assert.equal(result.narrativePriority, 'local-or-undetermined') + assert.equal(result.comparison.reliableSearch, false) + assert.equal(result.comparison.totalVisits, quality?.totalVisits ?? 0) + assert.equal(result.comparison.source, 'before-root-and-forced-played') + assert.match(result.prompt, /当前搜索更建议远处 Q16/) + } +}) + +test('merged played display row retains explicit forced provenance and cannot bypass after-root provenance', () => { + const value = input() + value.analysis.playedMove = { move: 'D4', source: 'forced', winrate: 52, scoreLead: 0, visits: 1200 } + // Even a matching display row with different visits does not override the source. + const forced = build(value) + assert.equal(forced.status, 'compared') + assert.equal(forced.comparison.source, 'before-root-and-forced-played') + assert.equal(forced.comparison.playedVisits, 1200) + assert.match(forced.prompt, /同一落子前局面估值/) + assert.match(forced.prompt, /实战估值来自单独强制搜索/) + value.analysis.playedMove.source = 'after-root' + const after = build(value) + assert.equal(after.status, 'unavailable') + assert.match(after.prompt, /after-root/) +}) + +test('near-optimal local alternative defeats global-region attribution even when the played point is poor', () => { + for (const local of [candidate('E4', 64, 4.5), candidate('E4', 63.5, 2), candidate('E4', 50, 4.5)]) { + const value = input() + value.analysis.before.topMoves.push(candidate('F5', 55, 1), local) + const result = build(value) + assert.equal(result.narrativePriority, 'local-or-undetermined') + assert.equal(result.bestLocalAlternative.move, 'E4') + assert.match(result.prompt, /本局部仍可处理，实战点选择有差别/) + } +}) + +test('searched weak local alternative supports a qualified preference for the remote area', () => { + const value = input() + value.analysis.before.topMoves.push(candidate('E4', 54, 0.5)) + const result = build(value) + assert.equal(result.narrativePriority, 'global-priority') + assert.equal(result.bestLocalAlternative.move, 'E4') + assert.match(result.prompt, /也明显优于已搜索的本地替代 E4/) +}) + +test('unsearched, occupied or forced played rows do not become local alternatives', () => { + const value = input() + value.analysis.before.topMoves.push(candidate('E4', 65, 5, 0), candidate('F4', 65, 5, 128)) + value.boardSnapshot.push({ point: 'F4', color: 'W' }) + value.analysis.playedMove = { move: 'D4', source: 'forced', winrate: 52, scoreLead: 0, visits: 1200 } + assert.equal(build(value).bestLocalAlternative, undefined) +}) + +test('saturated winrate cannot establish significance without a meaningful score gap', () => { + const value = input() + value.analysis.before.topMoves = [candidate('Q16', 99.9, 20), candidate('D4', 90, 19)] + assert.equal(build(value).narrativePriority, 'local-or-undetermined') + value.analysis.before.topMoves[1].scoreLead = 15 + assert.equal(build(value).narrativePriority, 'global-priority') + assert.match(build(value).prompt, /只采用目差/) +}) + +test('ordinary winrate gap can establish importance even with less than three points score gap', () => { + const value = input() + value.analysis.before.topMoves = [candidate('Q16', 60, 1), candidate('D4', 50, 0)] + assert.equal(build(value).narrativePriority, 'global-priority') +}) + +test('wrong move, game, size, timing, color or trial branch never compares unrelated positions', () => { + for (const mutate of [ + (v) => { v.analysis.moveNumber++ }, (v) => { v.analysis.boardSize = 9 }, + (v) => { v.analysis.gameId = 'other' }, (v) => { v.boardTiming = 'after-move' }, + (v) => { v.analysis.currentMove.color = 'W' }, (v) => { v.analysis.currentMove.gtp = 'E4' }, + (v) => { v.analysis.currentMove.moveNumber++ }, + (v) => { v.analysis.trialContext = { active: true, branchHash: 'other' } }, + (v) => { v.trialBranchHash = 'expected' } + ]) { + const value = input(); mutate(value) + assert.equal(build(value).status, 'unavailable') + } + const branch = input({ trialBranchHash: 'expected' }) + branch.analysis.trialContext = { active: true, branchHash: 'expected' } + assert.equal(build(branch).status, 'compared') +}) + +test('pass, invalid, occupied and unavailable pre-move boards cannot imply a global mistake', () => { + for (const mutate of [ + (v) => { v.playedMove = 'pass' }, (v) => { v.playedMove = 'I4' }, (v) => { v.playedMove = 'T99' }, + (v) => { v.analysis.currentMove.pass = true }, (v) => { v.analysis.before.topMoves[0].move = 'pass' }, + (v) => { v.boardSnapshot = undefined }, (v) => { v.boardSnapshot = [{ point: 'D4', color: 'B' }] }, + (v) => { v.boardSnapshot = [{ point: 'Q16', color: 'W' }] }, + (v) => { v.boardSnapshot = [{ point: 'I4', color: 'B' }] } + ]) { + const value = input(); mutate(value) + assert.equal(build(value).status, 'unavailable') + } +}) + +test('nonfinite/out-of-range evaluation and malformed visits cannot manufacture a significant gap', () => { + for (const [key, invalid] of [['winrate', NaN], ['winrate', 101], ['scoreLead', Infinity], ['visits', -1], ['visits', undefined]]) { + const value = input(); value.analysis.before.topMoves[1][key] = invalid + assert.equal(build(value).status, 'unavailable') + } +}) + +test('distance threshold scales down on small boards and uses engine rank rather than score sorting', () => { + const value = input({ boardSize: 9, playedMove: 'A1' }) + value.analysis.boardSize = 9 + value.analysis.currentMove.gtp = 'A1' + value.analysis.before.topMoves = [candidate('H8', 65, 5), candidate('A1', 52, 0), candidate('D4', 99, 90)] + const result = build(value) + assert.equal(result.narrativePriority, 'local-or-undetermined') + assert.equal(result.comparison.bestMove, 'H8') + assert.equal(result.bestLocalAlternative.move, 'D4') + assert.equal(result.comparison.distantThreshold, 4) +}) diff --git a/tests/initial-board-capture.test.mjs b/tests/initial-board-capture.test.mjs new file mode 100644 index 0000000..0256dfc --- /dev/null +++ b/tests/initial-board-capture.test.mjs @@ -0,0 +1,53 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { needsInitialBoardCapture, prepareInitialBoardMessages } from '../src/main/services/teacher/initialBoardCapture.ts' + +const input = () => ({ + provider: 'codex-app-server', intent: 'current-move', hasCaptureHandler: true, + request: { mode: 'current-move', gameId: 'game', moveNumber: 12, toolPolicy: 'auto' }, + report: { required: true, attached: false } +}) +const image = { role: 'user', content: [{ type: 'text', text: '第12手棋盘' }, { type: 'image_url', image_url: { url: 'data:image/png;base64,AA==' } }] } + +test('initial capture is limited to explicit automatic ChatGPT board review', async () => { + const expected = input() + assert.equal(needsInitialBoardCapture(expected), true) + for (const changed of [ + { provider: 'openai-compatible' }, { intent: 'freeform' }, { hasCaptureHandler: false }, + { request: { ...expected.request, mode: 'freeform' } }, + { request: { ...expected.request, toolPolicy: 'manual' } }, + { request: { ...expected.request, gameId: '' } }, + { request: { ...expected.request, moveNumber: 0 } }, + { request: { ...expected.request, moveNumber: 2.5 } }, + { report: { required: false, attached: false } }, { report: { required: true, attached: true } } + ]) { + let calls = 0 + assert.deepEqual(await prepareInitialBoardMessages({ ...expected, ...changed }, async () => { calls++; return { ok: true, toolResult: '', followupMessages: [image] } }), []) + assert.equal(calls, 0) + } +}) +test('one real capture supplies metadata before its initial image, including trial requests', async () => { + let calls = 0 + const expected = input() + expected.request.boardContext = 'trial' + expected.request.trialBranch = { active: true, baseMoveNumber: 11, moves: [{ color: 'W', gtp: 'C15' }] } + const messages = await prepareInitialBoardMessages(expected, async () => { + calls++ + return { ok: true, toolResult: '{"moveNumber":12,"visionEvidence":{"attached":true}}', followupMessages: [image] } + }) + assert.equal(calls, 1) + assert.match(messages[0].content, /"moveNumber":12/) + assert.deepEqual(messages[1], image) + assert.equal(messages.filter(message => Array.isArray(message.content)).length, 1) +}) +test('failed, empty-image, and cancelled captures prevent a subsequent model call', async () => { + for (const capture of [ + async () => ({ ok: false, toolResult: 'renderer failed', followupMessages: [] }), + async () => ({ ok: true, toolResult: 'metadata only', followupMessages: [] }), + async () => { throw new Error('cancelled') } + ]) { + let modelCalls = 0 + await assert.rejects(async () => { await prepareInitialBoardMessages(input(), capture); modelCalls++ }) + assert.equal(modelCalls, 0) + } +}) diff --git a/tests/katago-ownership-behavior.test.mjs b/tests/katago-ownership-behavior.test.mjs new file mode 100644 index 0000000..ef6bc19 --- /dev/null +++ b/tests/katago-ownership-behavior.test.mjs @@ -0,0 +1,83 @@ +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import test from 'node:test' +import ts from 'typescript' + +const ownershipSource = readFileSync(new URL('../src/main/services/analysis/ownership.ts', import.meta.url), 'utf8') +const katagoSource = readFileSync(new URL('../src/main/services/katago.ts', import.meta.url), 'utf8') +const ast = ts.createSourceFile('katago.ts', katagoSource, ts.ScriptTarget.Latest, true) +const functionNames = new Set(['blackWinrateFromSideToMove', 'blackScoreLeadFromSideToMove', 'responseSideToMove', 'root', 'candidates']) +const functions = ast.statements + .filter((statement) => ts.isFunctionDeclaration(statement) && functionNames.has(statement.name?.text)) + .map((statement) => statement.getText(ast)) +assert.equal(functions.length, functionNames.size) +const compiled = ts.transpileModule(`${ownershipSource}\n${functions.join('\n')}\nexport { root, candidates };`, { + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ES2022 } +}).outputText +const { readRootOwnership, root, candidates } = await import(`data:text/javascript;base64,${Buffer.from(compiled).toString('base64')}`) + +const field = (value) => Array(81).fill(value) +const response = (extra = {}, currentPlayer = 'B') => ({ rootInfo: { currentPlayer, winrate: 0.6, scoreLead: 2 }, ...extra }) + +test('official top-level root ownership and stdev reach the parsed analysis root', () => { + const parsed = root(response({ ownership: field(0.75), ownershipStdev: field(0.2) }), 'B', 9) + assert.deepEqual(parsed.ownership, field(0.75)) + assert.deepEqual(parsed.ownershipStdev, field(0.2)) + assert.equal(parsed.winrate, 60) + assert.equal(parsed.scoreLead, 2) +}) + +test('top-level fields take priority and missing fields retain legacy nested compatibility', () => { + const nested = { rootInfo: { ownership: field(-0.8), ownershipStdev: field(0.3) } } + assert.deepEqual(readRootOwnership(nested, 9), { ownership: field(-0.8), ownershipStdev: field(0.3) }) + assert.deepEqual(readRootOwnership({ ...nested, ownership: field(0.8), ownershipStdev: field(0.1) }, 9), { + ownership: field(0.8), ownershipStdev: field(0.1) + }) + assert.deepEqual(readRootOwnership({ ...nested, ownership: field(0.8) }, 9), { + ownership: field(0.8), ownershipStdev: field(0.3) + }) +}) + +test('malformed official arrays are discarded without falling back or fabricating zeros', () => { + const invalid = [null, [], field(0).slice(1), Array(361).fill(0), field(NaN), field(Infinity), field('0'), field(null), field(1.01), field(-1.01), new Array(81)] + for (const ownership of invalid) { + const parsed = readRootOwnership({ ownership, rootInfo: { ownership: field(0.8) } }, 9) + assert.equal(parsed.ownership, undefined) + } + assert.equal(readRootOwnership({}, 9).ownership, undefined) + assert.equal(readRootOwnership({ rootInfo: { ownership: field('0') } }, 9).ownership, undefined) +}) + +test('ownership length matches the contextual board size, independent of optional response dimensions', () => { + assert.equal(root(response({ ownership: Array(361).fill(0), boardXSize: 19, boardYSize: 19 }), 'B', 9).ownership, undefined) + assert.deepEqual(root(response({ ownership: field(0), boardXSize: 19, boardYSize: 19 }), 'B', 9).ownership, field(0)) + assert.equal(readRootOwnership({ ownership: field(0) }, 9.5).ownership, undefined) + assert.equal(readRootOwnership({ ownership: [] }, 0).ownership, undefined) +}) + +test('standard deviations require finite numeric values between zero and one', () => { + for (const ownershipStdev of [field(-0.1), field(1.01), field(NaN), field(Infinity), field('0.1'), field(null), field(0.1).slice(1)]) { + assert.equal(readRootOwnership({ ownershipStdev, rootInfo: { ownershipStdev: field(0.2) } }, 9).ownershipStdev, undefined) + } + assert.deepEqual(readRootOwnership({ ownership: field(-1), ownershipStdev: field(1) }, 9), { + ownership: field(-1), ownershipStdev: field(1) + }) +}) + +test('before and after ownership preserve the reported player perspective, as do candidates', () => { + const before = response({ ownership: field(0.8), moveInfos: [{ move: 'A1', ownership: field(0.7) }] }, 'B') + const after = response({ ownership: field(-0.8), moveInfos: [{ move: 'B1', ownership: field(-0.7) }] }, 'W') + assert.deepEqual(root(before, 'W', 9).ownership, field(0.8)) + assert.deepEqual(root(after, 'B', 9).ownership, field(-0.8)) + assert.deepEqual(candidates(before, 'W')[0].ownership, field(0.7)) + assert.deepEqual(candidates(after, 'B')[0].ownership, field(-0.7)) + assert.equal(root(after, 'B', 9).scoreLead, -2) + assert.equal(root(after, 'B', 9).winrate, 40) +}) + +test('valid arrays are copied so parsed root data cannot mutate the response', () => { + const raw = field(0.5) + const parsed = readRootOwnership({ ownership: raw }, 9) + parsed.ownership[0] = -1 + assert.equal(raw[0], 0.5) +}) diff --git a/tests/katago-ranking-contract.test.mjs b/tests/katago-ranking-contract.test.mjs index c03c657..7b73fb0 100644 --- a/tests/katago-ranking-contract.test.mjs +++ b/tests/katago-ranking-contract.test.mjs @@ -42,8 +42,8 @@ test('KataGo candidate ranking uses before-position choices and current-player l assert.match(katago, /forcedActualVisits\(maxVisits, FORCED_ACTUAL_EVIDENCE_MIN_VISITS\)/) assert.match(katago, /allowMoves\?:/) assert.match(katago, /payload\.allowMoves = query\.allowMoves/) - assert.match(katago, /root\(beforeResponse, beforeSideToMove\)/) - assert.match(katago, /root\(afterResponse, afterSideToMove\)/) + assert.match(katago, /root\(beforeResponse, beforeSideToMove, boardSize\)/) + assert.match(katago, /root\(afterResponse, afterSideToMove, boardSize\)/) assert.match(katago, /candidates\(beforeResponse, beforeSideToMove\)/) assert.match(katago, /playedMoveValue\(currentMove, searchMoves, afterRoot, forcedActual\)/) assert.match(katago, /playedLoss\(currentMove, best, actual\)/) diff --git a/tests/knowledge-semantic-grounding-behavior.test.mjs b/tests/knowledge-semantic-grounding-behavior.test.mjs new file mode 100644 index 0000000..97eb04f --- /dev/null +++ b/tests/knowledge-semantic-grounding-behavior.test.mjs @@ -0,0 +1,132 @@ +import assert from 'node:assert/strict' +import { readFileSync } from 'node:fs' +import test from 'node:test' +import ts from 'typescript' + +async function loadTs(path, prefix = '', suffix = '') { + const source = readFileSync(new URL(path, import.meta.url), 'utf8') + const ast = ts.createSourceFile(path, source, ts.ScriptTarget.Latest, true) + const body = ast.statements.filter((node) => !ts.isImportDeclaration(node)).map((node) => node.getText(ast)).join('\n') + const compiled = ts.transpileModule(`${prefix}\n${body}\n${suffix}`, { + compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ES2022 } + }).outputText + return import(`data:text/javascript;base64,${Buffer.from(compiled).toString('base64')}`) +} + +const matcher = await loadTs('../src/main/services/knowledge/matchEngine.ts', ` +let library; +export function setLibrary(value) { library = value } +function loadKnowledgeTrainingLibrary() { return library } +function loadKnowledgePatternCards() { return [] } +function searchKnowledgePatterns() { return [] } +function formatPatternForPrompt() { return '' } +`) +const { findLocalPatternMatches } = await loadTs('../src/main/services/knowledge/localPatternMatcher.ts') +const b = (point) => ({ color: 'B', point }) +const w = (point) => ({ color: 'W', point }) +const initialStones = [b('D5'), b('E4'), w('C4'), w('C3')] +const problem = { + id: 'eye', title: '真眼假眼', region: 'corner', toPlay: 'B', difficulty: 'standard', + objective: '判断真眼假眼死活', sourceKind: 'common-pattern', initialStones, + correctMoves: [{ move: 'D4', explanation: '训练答案' }], failureMoves: [], tags: ['真眼假眼', '死活'], + teaching: { recognition: '训练棋形', firstFeeling: '先读棋', explanation: '练习眼形', memoryCue: '眼形', firstHint: '先读棋', tesujiIdea: '读棋' } +} +const line = { + id: 'joseki', title: '星位点三三', family: '星位', phase: ['opening'], levels: ['intermediate'], + sourceKind: 'common-pattern', relativeSequence: ['D4', 'C3', 'D3', 'C4'], normalizedFeatures: [], + branches: [], decisionRules: [], commonMistakes: [], katagoEraJudgement: '训练参考', trainingFocus: [], tags: ['定式', '星位点三三'] +} +const query = (extra = {}) => ({ boardSize: 19, moveNumber: 80, totalMoves: 180, userLevel: 'intermediate', recentMoves: [], + text: '真眼假眼死活', playedMove: 'D4', candidateMoves: ['D4'], playerColor: 'B', lossScore: 10, judgement: 'blunder', maxResults: 20, ...extra }) +const search = (extra = {}) => matcher.searchKnowledgeMatchEngine('', query(extra)) +matcher.setLibrary({ josekiLines: [line], lifeDeathProblems: [problem], tesujiProblems: [{ ...problem, id: 'tesuji', title: '倒扑手筋', tags: ['倒扑', '手筋'] }] }) + +test('region, loss, tactical keywords and accidental answer coordinates remain training analogies', () => { + for (const text of ['真眼假眼死活', '倒扑手筋']) { + const matches = search({ text, boardSnapshot: [b('Q16'), w('R16')] }) + assert.ok(matches.length > 0) + for (const match of matches) { + assert.ok(['partial', 'weak'].includes(match.confidence), `${match.id}: ${match.confidence}`) + assert.ok(!match.reason.some((reason) => /answer-|sequence-overlap/.test(reason))) + assert.match(match.applicability, /训练|弱相关/) + } + } +}) + +test('absolute answer coincidences in history and whole-board stones do not change score', () => { + const baseline = search({ playedMove: undefined, candidateMoves: [], boardSnapshot: [b('Q16')] }).find((m) => m.id === 'eye') + const coincidence = search({ playedMove: undefined, candidateMoves: [], boardSnapshot: [b('D4')], + recentMoves: [{ moveNumber: 1, color: 'B', gtp: 'D4', row: 15, col: 3, pass: false }] }).find((m) => m.id === 'eye') + assert.equal(coincidence.score, baseline.score) + assert.equal(coincidence.confidence, 'partial') +}) + +test('PV future points and untyped local-window anchors do not establish before-position geometry', () => { + const before = search({ playedMove: undefined, candidateMoves: [], boardSnapshot: initialStones }) + const future = search({ playedMove: undefined, candidateMoves: [], boardSnapshot: initialStones, + principalVariation: ['D4', 'C3', 'E4'], localWindows: [{ anchor: 'D4', stones: initialStones }] }) + assert.deepEqual(future, before) + assert.ok(future.every((m) => !m.reason.some((reason) => reason.startsWith('geometry:')))) +}) + +test('real relative geometry with the matching player role supports strong shape relevance, never a solved-position claim', () => { + const matched = search({ boardSnapshot: initialStones }).find((m) => m.id === 'eye') + assert.equal(matched.confidence, 'strong') + assert.ok(matched.reason.some((reason) => reason.startsWith('geometry:'))) + assert.match(matched.applicability, /未验证.*是否合法/) + assert.match(matched.teachingPayload.recognition, /训练类比，尚未确认/) + const prompt = matcher.formatKnowledgeMatchForPrompt(matched) + assert.match(prompt, /不代表本局概念或战术成立/) + assert.match(prompt, /必须另有合法手顺和读棋证据/) + assert.doesNotMatch(prompt, /可以说“这是/) + const unknown = search({ boardSnapshot: initialStones, playerColor: undefined }).find((m) => m.id === 'eye') + assert.equal(unknown.confidence, 'partial') + const wrongPlayer = search({ boardSnapshot: initialStones, playerColor: 'W' }).find((m) => m.id === 'eye') + assert.equal(wrongPlayer.confidence, 'partial') + const windowOnly = search({ localWindows: [{ anchor: 'D4', stones: initialStones }] }).find((m) => m.id === 'eye') + assert.equal(windowOnly.confidence, 'partial') + assert.ok(windowOnly.reason.some((reason) => reason.startsWith('incomplete-local-window:'))) +}) + +test('unordered joseki coordinate overlap and explicit keywords cannot authorize exact or strong', () => { + const found = search({ text: '星位点三三定式', moveNumber: 10, playedMove: 'C3', candidateMoves: ['D3'], + principalVariation: ['D4', 'C4'], recentMoves: [{ gtp: 'D4', row: 15, col: 3, color: 'B' }] }).find((m) => m.id === 'joseki') + assert.ok(found) + assert.equal(found.confidence, 'partial') +}) + +const card = (anchorRole, points = [{ dx: 0, dy: 0, state: 'any-stone' }, { dx: 1, dy: 0, state: 'friendly' }]) => ({ + id: anchorRole, title: anchorRole, shapeType: 'test', category: 'shape', anchorRole, + phase: ['any'], regions: ['any'], tags: [], sourceRefs: [], sourceQuality: 'original', points, + teaching: { recognition: '', wrongThinking: '', correctThinking: '', drillPrompt: '' } +}) +const local = (cards, extra = {}) => findLocalPatternMatches(cards, { + boardSize: 19, boardSnapshot: [b('D4'), b('E4')], anchors: ['D4'], playerColor: 'B', ...extra +}) + +test('actual and candidate roles require typed source anchors rather than generic windows', () => { + assert.deepEqual(local([card('actual'), card('candidate')], { localWindows: [{ anchor: 'D4', stones: [] }] }), []) + assert.equal(local([card('actual')], { playedMove: 'D4' }).length, 1) + assert.equal(local([card('candidate')], { candidateMoves: ['D4'] }).length, 1) + assert.equal(local([card('actual')], { playedMove: 'Q16', candidateMoves: ['D4'] }).length, 0) + assert.equal(local([card('either')], { anchors: [], localWindows: [{ anchor: 'D4', stones: [] }] }).length, 0) +}) + +test('friendly, enemy and empty anchor roles obey the actual board and known player', () => { + assert.deepEqual(local([card('friendly-stone'), card('enemy-stone'), card('empty')]).map((m) => m.card.id), ['friendly-stone']) + assert.equal(local([card('enemy-stone')], { boardSnapshot: [w('D4'), b('E4')] }).length, 1) + const emptyCard = card('empty', [{ dx: 0, dy: 0, state: 'empty' }, { dx: 1, dy: 0, state: 'friendly' }]) + assert.equal(local([emptyCard], { boardSnapshot: [b('E4')] }).length, 1) + assert.equal(local([emptyCard]).length, 0) + assert.equal(local([card('either')], { boardSnapshot: [w('D4'), w('E4')] }).length, 0) +}) + +test('unknown player perspective is explicit and weak; required points and constraints cannot be ignored', () => { + const match = local([card('either')], { boardSnapshot: [w('D4'), w('E4')], playerColor: undefined })[0] + assert.equal(match.perspective, 'W') + assert.equal(match.confidence, 'weak') + assert.match(match.evidence.join(' '), /current-player unknown/) + assert.equal(local([card('either')], { boardSnapshot: undefined }).length, 0) + assert.equal(local([{ ...card('either'), constraints: [{ type: 'anchor-empty', value: 1 }] }]).length, 0) + assert.equal(local([card('either')], { boardSnapshot: [b('D4')] }).length, 0) +}) diff --git a/tests/knowledge-training.test.mjs b/tests/knowledge-training.test.mjs index 249b9ba..9664f69 100644 --- a/tests/knowledge-training.test.mjs +++ b/tests/knowledge-training.test.mjs @@ -103,7 +103,7 @@ test('matching engine feeds knowledge matches to the agent without forcing UI te assert.doesNotMatch(card, /关联训练题/) }) -test('matching engine ranks exact joseki, life-death, and tesuji matches ahead of broad patterns', () => { +test('matching engine ranks relevant training analogies without claiming keyword-only exact matches', () => { const fixture = join(repoRoot, 'tests', 'fixtures', 'knowledge-match-engine-smoke.ts') const output = execFileSync(process.execPath, [findJitiCli(), fixture], { cwd: repoRoot, @@ -112,12 +112,12 @@ test('matching engine ranks exact joseki, life-death, and tesuji matches ahead o const result = JSON.parse(output) assert.equal(result.star33.matches[0].matchType, 'joseki') - assert.equal(result.star33.matches[0].confidence, 'exact') + assert.equal(result.star33.matches[0].confidence, 'partial') assert.match(result.star33.matches[0].title, /星位点三三/) assert.deepEqual(result.star33.recommendedProblems, []) assert.equal(result.trueFalseEye.matches[0].matchType, 'life_death') - assert.equal(result.trueFalseEye.matches[0].confidence, 'exact') + assert.equal(result.trueFalseEye.matches[0].confidence, 'partial') assert.match(result.trueFalseEye.matches[0].title, /真眼假眼/) assert.equal(result.trueFalseEye.recommendedProblems[0].problemType, 'life_death') assert.match(result.trueFalseEye.recommendedProblems[0].title, /真眼假眼/) @@ -125,7 +125,7 @@ test('matching engine ranks exact joseki, life-death, and tesuji matches ahead o const firstTesujiIndex = result.snapback.matches.findIndex((match) => match.matchType === 'tesuji') const firstJosekiIndex = result.snapback.matches.findIndex((match) => match.matchType === 'joseki') assert.equal(firstTesujiIndex, 0) - assert.equal(result.snapback.matches[0].confidence, 'exact') + assert.equal(result.snapback.matches[0].confidence, 'partial') assert.match(result.snapback.matches[0].title, /倒扑/) assert.ok(firstJosekiIndex === -1 || firstJosekiIndex > firstTesujiIndex) assert.equal(result.snapback.recommendedProblems[0].problemType, 'tesuji') diff --git a/tests/move-board-effects.test.mjs b/tests/move-board-effects.test.mjs new file mode 100644 index 0000000..d5a281d --- /dev/null +++ b/tests/move-board-effects.test.mjs @@ -0,0 +1,157 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { buildMoveBoardEffects } from '../src/main/services/teacher/moveBoardEffects.ts' + +const stones = (color, points) => points.map((point) => ({ point, color })) +const ko = () => ({ boardSize: 19, playerColor: 'B', playedMove: 'D4', boardSnapshot: [ + ...stones('B', ['C3', 'D3', 'E3', 'C4', 'E4', 'D5']), ...stones('W', ['C5', 'E5', 'D6']) +] }) +const columns = 'ABCDEFGHJKLMNOPQRST' +function transform(point, turns, reflect) { + let x = columns.indexOf(point[0]); let y = Number(point.slice(1)) - 1 + if (reflect) x = 18 - x + for (let i = 0; i < turns; i++) [x, y] = [18 - y, x] + return `${columns[x]}${y + 1}` +} + +test('connection closes a structurally verified single-stone ko without claiming legal recapture', () => { + const input = ko(); const before = JSON.stringify(input) + const result = buildMoveBoardEffects(input) + assert.equal(result.status, 'observed') + assert.deepEqual(result.actual.capturedStones, []) + assert.equal(result.actual.adjacentFriendlyChains.length, 2) + assert.equal(result.actual.mergedFriendlyChainIds.length, 2) + assert.equal(result.actual.resultingChain.stoneCount, 7) + const singleton = result.actual.libertyChanges.find((change) => change.beforeChainId === 'B:D5') + assert.deepEqual(singleton.before, ['D4']) + assert.deepEqual(singleton.lost, ['D4']) + assert.equal(result.koConnection.kind, 'ko-connection') + assert.equal(result.koConnection.connectedSingleton, 'D5') + assert.deepEqual(result.koConnection.opponentProbe.capturedStones, ['D5']) + assert.deepEqual(result.koConnection.opponentProbe.resultingChain.liberties, ['D5']) + assert.deepEqual(result.koConnection.structuralRecapture.capturedStones, ['D4']) + assert.equal(result.koConnection.structuralRecapture.restoresBeforeBoard, true) + assert.equal(result.legality, 'board-only-not-history-verified') + assert.match(result.prompt, /粘劫/) + assert.match(result.prompt, /不是立即合法回提/) + assert.match(result.prompt, /不证明棋块死活/) + assert.equal(JSON.stringify(input), before) +}) + +test('ko structure and endpoints survive all rotations, reflection and color reversal', () => { + for (const reflect of [false, true]) for (let turns = 0; turns < 4; turns++) for (const swap of [false, true]) { + const input = ko() + input.playedMove = transform(input.playedMove, turns, reflect) + input.playerColor = swap ? 'W' : 'B' + input.boardSnapshot = input.boardSnapshot.map((stone) => ({ point: transform(stone.point, turns, reflect), + color: swap ? stone.color === 'B' ? 'W' : 'B' : stone.color })) + const result = buildMoveBoardEffects(input) + assert.equal(result.koConnection?.connectedSingleton, transform('D5', turns, reflect)) + assert.equal(result.koConnection?.structuralRecapture.color, input.playerColor) + assert.equal(result.koConnection?.opponentProbe.resultingChain.stoneCount, 1) + } +}) + +test('ordinary connection records distinct chain endpoints without inventing ko or weakness', () => { + const result = buildMoveBoardEffects({ boardSize: 19, playerColor: 'B', playedMove: 'D4', + boardSnapshot: stones('B', ['C4', 'E4']) }) + assert.equal(result.status, 'observed') + assert.equal(result.actual.mergedFriendlyChainIds.length, 2) + assert.deepEqual(result.actual.adjacentFriendlyChains.map((chain) => chain.neighboringStones), [['C4'], ['E4']]) + assert.equal(result.koConnection, undefined) + assert.match(result.prompt, /不能直接解释为弱龙补强/) +}) + +test('multiple neighbors from one string are not reported as a connection', () => { + const result = buildMoveBoardEffects({ boardSize: 19, playerColor: 'B', playedMove: 'D4', + boardSnapshot: stones('B', ['C4', 'C3', 'D3', 'E3', 'E4']) }) + assert.equal(result.actual.adjacentFriendlyChains.length, 1) + assert.deepEqual(result.actual.mergedFriendlyChainIds, []) + assert.equal(result.koConnection, undefined) +}) + +test('two-stone threatened chain is not classified as single-stone ko', () => { + const input = ko() + input.boardSnapshot = [ ...stones('B', ['C3', 'D3', 'E3', 'C4', 'E4', 'D5', 'D6']), + ...stones('W', ['C5', 'E5', 'C6', 'E6', 'D7']) ] + const result = buildMoveBoardEffects(input) + assert.equal(result.status, 'observed') + assert.equal(result.actual.mergedFriendlyChainIds.length, 2) + assert.equal(result.koConnection, undefined) +}) + +test('single capture without an isolated capturing stone / restoration cycle is not ko', () => { + // White D4 joins C4/C5 and captures D5; Black D5 would capture three white stones (snapback). + const result = buildMoveBoardEffects({ boardSize: 19, playerColor: 'B', playedMove: 'D4', boardSnapshot: [ + ...stones('B', ['B4', 'B5', 'C3', 'C6', 'D3', 'E4', 'D5']), + ...stones('W', ['C4', 'C5', 'E5', 'D6']) + ] }) + assert.equal(result.status, 'observed') + assert.deepEqual(result.actual.capturedStones, ['C4', 'C5']) + assert.equal(result.koConnection, undefined) +}) + +test('a singleton rescue with open neighboring points is not a ko connection', () => { + const input = ko() + input.boardSnapshot = input.boardSnapshot.filter((stone) => !['C3', 'D3', 'E3', 'C4', 'E4'].includes(stone.point)) + const result = buildMoveBoardEffects(input) + assert.equal(result.status, 'observed') + assert.equal(result.actual.adjacentFriendlyChains.length, 1) + assert.equal(result.koConnection, undefined) +}) + +test('capturing one stone with an additional liberty does not create a ko cycle', () => { + const input = ko() + input.boardSnapshot = input.boardSnapshot.filter((stone) => stone.point !== 'C4') + const result = buildMoveBoardEffects(input) + assert.equal(result.status, 'observed') + assert.equal(result.actual.mergedFriendlyChainIds.length, 2) + assert.equal(result.koConnection, undefined) +}) + +test('actual capture has exact enemy chain / liberty facts', () => { + const result = buildMoveBoardEffects({ boardSize: 19, playerColor: 'B', playedMove: 'D4', boardSnapshot: [ + ...stones('B', ['C5', 'E5', 'D6']), ...stones('W', ['D5']) + ] }) + assert.deepEqual(result.actual.capturedStones, ['D5']) + assert.equal(result.actual.adjacentEnemyChains[0].stoneCount, 1) + assert.deepEqual(result.actual.adjacentEnemyChains[0].liberties, ['D4']) + assert.ok(result.actual.resultingChain.liberties.includes('D5')) +}) + +test('missing, invalid, occupied, conflicting and suicide inputs abstain without false effects', () => { + const input = ko() + const invalid = [ + { ...input, boardSnapshot: undefined }, { ...input, boardSize: 1 }, { ...input, playedMove: 'I4' }, + { ...input, playedMove: 'pass' }, { ...input, playerColor: 'X' }, + { ...input, boardSnapshot: [...input.boardSnapshot, { point: 'D4', color: 'W' }] }, + { ...input, boardSnapshot: [...input.boardSnapshot, { point: 'D5', color: 'W' }] }, + { ...input, boardSnapshot: [...input.boardSnapshot, { point: 'A20', color: 'W' }] }, + { ...input, boardSnapshot: stones('W', ['C4', 'E4', 'D3', 'D5']) }, + { ...input, boardSnapshot: [...stones('B', ['A1']), ...stones('W', ['A2', 'B1'])] } + ] + for (const value of invalid) { + const result = buildMoveBoardEffects(value) + assert.equal(result.status, 'abstained') + assert.equal(result.actual, undefined) + assert.equal(result.koConnection, undefined) + } +}) + +test('partial move history never promotes structural probing to rule-aware legality', () => { + const result = buildMoveBoardEffects({ ...ko(), moveHistory: [{ point: 'pass', color: 'B' }, { point: 'D5', color: 'B' }] }) + assert.equal(result.koConnection.kind, 'ko-connection') + assert.equal(result.legality, 'board-only-not-history-verified') + assert.match(result.prompt, /未结合完整棋谱和规则/) +}) + +test('returned chains retain counts and neighboring representatives without duplicating large stone arrays', () => { + const input = ko() + input.boardSnapshot.push(...stones('B', ['B3', 'A3', 'A4', 'A5'])) + const result = buildMoveBoardEffects(input) + assert.ok(result.actual.adjacentFriendlyChains.some((chain) => chain.stoneCount > chain.neighboringStones.length)) + for (const chain of [...result.actual.adjacentFriendlyChains, result.actual.resultingChain]) { + assert.equal('stones' in chain, false) + assert.ok(chain.neighboringStones.length <= 4) + } +}) diff --git a/tests/move-concept-evidence.test.mjs b/tests/move-concept-evidence.test.mjs new file mode 100644 index 0000000..d774b99 --- /dev/null +++ b/tests/move-concept-evidence.test.mjs @@ -0,0 +1,200 @@ +import assert from 'node:assert/strict' +import test from 'node:test' +import { loadConceptModule } from '../scripts/lib/concept_modules.mjs' + +const { buildMoveConceptEvidence } = await loadConceptModule('src/main/services/teacher/moveConceptEvidence.ts') +const { extractKataGoShapeFeatures } = await loadConceptModule('src/main/services/knowledge/katagoShapeFeatures.ts') +const snapshot = [{ color: 'B', point: 'D4' }] +const analysis = { moveNumber: 10, boardSize: 19, currentMove: { color: 'B', gtp: 'F4' }, before: { topMoves: [ + { move: 'F4', pv: ['F4', 'Q16', 'E4'] }, { move: 'E6', pv: ['E6', 'E4'] } +] } } + +test('played and root candidates use before-move board and never label a PV continuation', () => { + const evidence = buildMoveConceptEvidence({ moveNumber: 10, boardSize: 19, boardSnapshot: snapshot, analysis }) + assert.equal(evidence.boardTiming, 'before-move') + assert.deepEqual(evidence.entries.map((entry) => [entry.role, entry.move]), [['played', 'F4'], ['candidate', 'E6']]) + assert.equal(evidence.entries[0].concepts.geometry[0].kind, 'one-space-jump') + assert.equal(evidence.entries[1].concepts.geometry[0].kind, 'small-knight') + assert.equal(evidence.moveEffects.actual.move, 'F4') + assert.deepEqual(evidence.candidateMoveEffects.map(entry => entry.move), ['E6']) + assert.match(evidence.prompt, /不能仅凭间隔确定是拆/) + assert.doesNotMatch(evidence.prompt, /Q16/) +}) + +test('actual ko connection and root candidate effects reach evidence without replaying PV as facts', () => { + const boardSnapshot = [ + ...['D4', 'E3', 'E5', 'F4'].map(point => ({ point, color: 'B' })), + ...['C4', 'D3', 'D5'].map(point => ({ point, color: 'W' })) + ] + const before = structuredClone(boardSnapshot) + const evidence = buildMoveConceptEvidence({ moveNumber: 20, boardSize: 9, playerColor: 'B', playedMove: 'E4', boardSnapshot, + analysis: { moveNumber: 20, boardSize: 9, currentMove: { color: 'B', gtp: 'E4' }, + before: { topMoves: [{ move: 'E4', pv: ['E4', 'D4'] }, { move: 'G7', pv: ['G7', 'D4'] }] } } }) + assert.equal(evidence.moveEffects.koConnection.kind, 'ko-connection') + assert.equal(evidence.moveEffects.koConnection.connectedSingleton, 'D4') + assert.deepEqual(evidence.moveEffects.actual.capturedStones, []) + assert.deepEqual(evidence.candidateMoveEffects.map(entry => entry.move), ['G7']) + assert.equal(evidence.candidateMoveEffects[0].effects.actual.move, 'G7') + assert.equal(evidence.candidateMoveEffects[0].effects.koConnection, undefined) + assert.deepEqual(boardSnapshot, before) + assert.deepEqual(evidence.boardCoordinates.stonesByColor, evidence.boardGroups.positionFacts.stonesByColor) + assert.ok(JSON.stringify(evidence).indexOf('"boardCoordinates"') < JSON.stringify(evidence).indexOf('"moveEffects"')) +}) + +test('complete coordinate facts remain readable before the normal tool truncation budget', () => { + const columns = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' + const boardSnapshot = Array.from({ length: 360 }, (_, i) => ({ + point: `${columns[i % 19]}${Math.floor(i / 19) + 1}`, color: i % 2 ? 'W' : 'B' + })) + const evidence = buildMoveConceptEvidence({ moveNumber: 300, boardSize: 19, playerColor: 'W', playedMove: 'T19', boardSnapshot }) + const transport = JSON.stringify({ moveConceptEvidence: evidence }, null, 2) + const end = transport.indexOf(',\n "globalPriority"') + assert.ok(end > 0 && end < 18000) + const received = JSON.parse(`${transport.slice(0, end)}\n }\n}`) + assert.deepEqual(received.moveConceptEvidence.boardCoordinates.stonesByColor, + evidence.boardGroups.positionFacts.stonesByColor) + assert.equal(received.moveConceptEvidence.boardCoordinates.stonesByColor.B.length + + received.moveConceptEvidence.boardCoordinates.stonesByColor.W.length, 360) +}) + +test('analysis from a different move or board size cannot provide candidate evidence', () => { + for (const stale of [{ ...analysis, moveNumber: 9 }, { ...analysis, boardSize: 9 }]) { + const evidence = buildMoveConceptEvidence({ moveNumber: 10, boardSize: 19, boardSnapshot: snapshot, playedMove: 'F4', playerColor: 'B', analysis: stale }) + assert.deepEqual(evidence.entries.map((entry) => entry.role), ['played']) + } +}) + +test('unavailable or occupied pre-move board cannot become a confident term', () => { + for (const boardSnapshot of [undefined, [...snapshot, { color: 'B', point: 'F4' }]]) { + const evidence = buildMoveConceptEvidence({ moveNumber: 10, boardSize: 19, boardSnapshot, playedMove: 'F4', playerColor: 'B' }) + assert.equal(evidence.entries[0].concepts.status, 'abstained') + } +}) + +test('concept tools carry actual prior moves into corner context without treating PV as history', () => { + const boardSnapshot = [{ color: 'W', point: 'D4' }, { color: 'B', point: 'F3' }, { color: 'W', point: 'H3' }] + const moveHistory = [{ color: 'B', point: 'F3' }, { color: 'W', point: 'H3' }] + const evidence = buildMoveConceptEvidence({ moveNumber: 41, boardSize: 19, boardSnapshot, + playedMove: 'C3', playerColor: 'B', moveHistory }) + assert.match(evidence.prompt, /三三/) + assert.match(evidence.prompt, /F3/) + assert.match(evidence.prompt, /H3/) + assert.doesNotMatch(evidence.prompt, /已经活棋|一定能活/) +}) + +test('pass, missing or invalid coordinates do not imply a distant global mistake', () => { + for (const playedMove of ['pass', undefined, 'I4', 'T99']) { + const features = extractKataGoShapeFeatures({ boardSize: 19, moveNumber: 90, totalMoves: 200, playedMove, candidateMoves: ['Q16'], lossScore: 8 }) + assert.equal(features.some((feature) => feature.shapeType === 'local_vs_global_shape'), false) + } +}) + +test('distance and PV length do not assert a shape cause or reliable reading', () => { + const features = extractKataGoShapeFeatures({ boardSize: 19, moveNumber: 90, totalMoves: 200, playedMove: 'F4', candidateMoves: ['E4'], lossScore: 4, principalVariation: ['E4', 'F5', 'G5', 'H5', 'J5', 'K5', 'L5', 'M5'] }) + assert.match(features.find((feature) => feature.shapeType === 'local_shape_detail').recognition, /距离和目差本身不能确定原因/) + assert.match(features.find((feature) => feature.shapeType === 'pv_supported_shape').recognition, /不能证明/) +}) + +test('teaching evidence carries whole-board facts and regional purpose alongside global comparison', () => { + const boardSnapshot = [{ color: 'W', point: 'M3' }, { color: 'W', point: 'H7' }, { color: 'B', point: 'O3' }] + const analysis = { gameId: 'game', moveNumber: 58, boardSize: 19, + currentMove: { moveNumber: 58, color: 'B', gtp: 'L4' }, + before: { topMoves: [{ move: 'Q16', winrate: 80, scoreLead: 8, visits: 700 }, { move: 'L4', winrate: 55, scoreLead: 1, visits: 200 }] }, + analysisQuality: { totalVisits: 900, confidence: 'high' } } + const evidence = buildMoveConceptEvidence({ moveNumber: 58, gameId: 'game', boardSize: 19, + boardSnapshot, playedMove: 'L4', playerColor: 'B', analysis }) + assert.equal(evidence.globalPriority.narrativePriority, 'global-priority') + assert.deepEqual(evidence.boardGroups.positionFacts.stonesByColor, { B: ['O3'], W: ['M3', 'H7'] }) + assert.ok(evidence.boardGroups.regionIntents.some(intent => intent.kind === 'separate-contact' && intent.status === 'hypothesis')) + assert.match(evidence.boardGroups.prompt, /分割/) + assert.equal(evidence.boardGroups.positionFacts.stonesByColor.B.includes('Q16'), false) + assert.equal(evidence.entries[0].move, 'L4') +}) + +test('global evidence rejects another game or trial while retaining actual board facts', () => { + const current = { gameId: 'other', moveNumber: 10, boardSize: 19, + currentMove: { moveNumber: 10, color: 'B', gtp: 'F4' }, before: { topMoves: [] } } + for (const analysis of [current, { ...current, gameId: 'game', trialContext: { active: true, branchHash: 'other' } }]) { + const result = buildMoveConceptEvidence({ gameId: 'game', moveNumber: 10, boardSize: 19, + playedMove: 'F4', playerColor: 'B', boardSnapshot: snapshot, analysis }) + assert.equal(result.globalPriority.status, 'unavailable') + assert.deepEqual(result.boardGroups.positionFacts.stonesByColor.B, ['D4']) + } +}) +test('rejected game, branch, move identity and color cannot leak AI candidate concepts or effects', () => { + const current = { gameId: 'game', moveNumber: 10, boardSize: 19, + currentMove: { moveNumber: 10, color: 'B', gtp: 'F4' }, + before: { topMoves: [{ move: 'E6', visits: 100, winrate: 70, scoreLead: 2 }] } } + for (const stale of [ + { ...current, gameId: 'other' }, + { ...current, trialContext: { active: true, branchHash: 'other' } }, + { ...current, currentMove: { ...current.currentMove, moveNumber: 9 } }, + { ...current, currentMove: { ...current.currentMove, color: 'W' } }, + { ...current, currentMove: { ...current.currentMove, gtp: 'Q16' } } + ]) { + const result = buildMoveConceptEvidence({ gameId: 'game', moveNumber: 10, boardSize: 19, + playedMove: 'F4', playerColor: 'B', boardSnapshot: snapshot, analysis: stale }) + assert.equal(result.globalPriority.status, 'unavailable') + assert.deepEqual(result.entries.map(entry => entry.role), ['played']) + assert.deepEqual(result.candidateMoveEffects, []) + assert.deepEqual(result.candidateOwnDevelopment, []) + assert.equal(result.moveEffects.actual.move, 'F4') + } + const main = buildMoveConceptEvidence({ gameId: 'game', trialBranchHash: 'active', moveNumber: 10, boardSize: 19, + playedMove: 'F4', playerColor: 'B', boardSnapshot: snapshot, analysis: current }) + assert.deepEqual(main.candidateMoveEffects, []) + const matching = buildMoveConceptEvidence({ gameId: 'game', trialBranchHash: 'active', moveNumber: 10, boardSize: 19, + playedMove: 'F4', playerColor: 'B', boardSnapshot: snapshot, + analysis: { ...current, trialContext: { active: true, branchHash: 'active' } } }) + assert.deepEqual(matching.candidateMoveEffects.map(entry => entry.move), ['E6']) +}) +test('compact root candidate development survives pretty JSON budget before verbose effects and graph', () => { + const boardSnapshot=[ + ...['D16','D4','Q5','R3','R4'].map(point=>({point,color:'W'})), + ...['O4','Q16','Q3','Q9','R2','S2'].map(point=>({point,color:'B'})) + ] + const evidence=buildMoveConceptEvidence({moveNumber:12,boardSize:19,playedMove:'O6',playerColor:'W',boardSnapshot, + analysis:{moveNumber:12,boardSize:19,currentMove:{color:'W',gtp:'O6'},before:{topMoves:[ + {move:'R8',pv:['R8','R9','Q8','P9']},{move:'Q7',pv:['Q7','S9','O7']},{move:'M3',pv:['M3','N3','R8']} + ]}}}) + const compact=evidence.candidateOwnDevelopment + const extension=compact.find(c=>c.move==='R8' && c.kind==='side-extension') + assert.ok(extension) + assert.equal(extension.anchor,'R4') + assert.equal(extension.status,'hypothesis') + assert.deepEqual(extension.groupStones,['R3','R4']) + assert.equal(extension.direction,'along-side') + assert.deepEqual(extension.corridor,['R5','R6','R7']) + assert.ok(extension.counterEvidence.some(e=>e.includes('Q9'))) + for (const entry of evidence.entries.filter(e=>e.role==='candidate')) { + const projected=compact.filter(c=>c.move===entry.move) + assert.ok(projected.length<=2) + assert.deepEqual(projected,entry.concepts.ownDevelopment.slice(0,2).map(c=>({ + move:entry.move,color:entry.color,kind:c.kind,status:c.status,anchor:c.anchor,groupStones:c.groupStones, + direction:c.direction,corridor:c.corridor,pressurePoints:c.pressurePoints,counterEvidence:c.counterEvidence + }))) + } + assert.ok(compact.every(c=>!('wording' in c) && !('evidence' in c))) + assert.ok(!compact.some(c=>c.move==='O6' || c.move==='R9' || c.move==='Q8'), 'played and PV moves are excluded') + const transport=JSON.stringify({moveConceptEvidence:evidence},null,2) + assert.ok(transport.indexOf('"globalPriority":')<transport.indexOf('"candidateOwnDevelopment":')) + assert.ok(transport.indexOf('"candidateOwnDevelopment":')<transport.indexOf('"moveEffects":')) + // Parse a complete transport prefix, not a substring that might cut a hypothesis in half. + const end=transport.indexOf(',\n "moveEffects":') + assert.ok(end>0 && end<18000) + const received=JSON.parse(`${transport.slice(0,end)}\n }\n}`) + assert.deepEqual(received.moveConceptEvidence.candidateOwnDevelopment,compact) +}) +test('missing, stale and unknown candidate development stays empty without inventing labels', () => { + const inputs=[ + {boardSnapshot:undefined,analysis}, + {boardSnapshot:snapshot,analysis:{...analysis,moveNumber:11}}, + {boardSnapshot:snapshot,analysis:{...analysis,boardSize:9}}, + {boardSnapshot:snapshot,analysis:{...analysis,before:{topMoves:[{move:'pass',pv:['pass','G4']},{move:'Q16',pv:['Q16','G4']}]}}}, + {boardSnapshot:snapshot} + ] + for (const input of inputs) { + const result=buildMoveConceptEvidence({moveNumber:10,boardSize:19,playedMove:'F4',playerColor:'B',...input}) + assert.deepEqual(result.candidateOwnDevelopment,[]) + } +}) diff --git a/tests/move-concepts-behavior.test.mjs b/tests/move-concepts-behavior.test.mjs new file mode 100644 index 0000000..7d1d520 --- /dev/null +++ b/tests/move-concepts-behavior.test.mjs @@ -0,0 +1,383 @@ +import assert from 'node:assert/strict' +import { test } from 'node:test' + +const { recognizeMoveConcepts, formatMoveConceptsForPrompt } = await (await import('../scripts/lib/concept_modules.mjs')).loadConceptModule('src/main/services/knowledge/moveConcepts.ts') +const b = (point) => ({ point, color: 'B' }) +const w = (point) => ({ point, color: 'W' }) +const run = (boardSnapshot, playedMove, extra = {}) => recognizeMoveConcepts({ boardSize: 19, boardSnapshot, playedMove, playerColor: 'B', ...extra }) + +test('actual geometry recognizes jump, diagonal and knight relations in every orientation', () => { + const columns = 'ABCDEFGHJKLMNOPQRSTUVWXYZ' + for (const [dx, dy, kind] of [[0, 2, 'one-space-jump'], [0, 3, 'two-space-jump'], [0, 4, 'three-space-jump'], [1, 1, 'diagonal'], [1, 2, 'small-knight'], [1, 3, 'large-knight']]) { + for (const swap of [false, true]) for (const sx of [-1, 1]) for (const sy of [-1, 1]) { + const x = 9 + sx * (swap ? dy : dx); const y = 9 + sy * (swap ? dx : dy) + const result = run([b('K10')], `${columns[x]}${y + 1}`) + assert.equal(result.geometry.length, 1) + assert.equal(result.geometry[0].kind, kind) + assert.equal(result.geometry[0].status, 'observed') + } + } +}) + +test('multiple friendly anchors remain ambiguous even when nearest anchor is unique', () => { + const result = run([b('D4'), b('E5')], 'F4') + assert.equal(result.status, 'ambiguous') + assert.deepEqual(result.geometry.map((g) => [g.anchor, g.kind]), [['D4', 'one-space-jump'], ['E5', 'diagonal']]) + assert.match(result.wording, /多个己棋/) + assert.equal(result.intents.length, 0) +}) + +test('a stone between jump endpoints blocks the named relation', () => { + for (const blocker of [w('E4'), b('E4')]) { + const result = run([b('D4'), blocker], 'F4') + const relation = result.geometry.find((g) => g.anchor === 'D4') + assert.equal(relation.status, 'blocked') + assert.deepEqual(relation.blockers, ['E4']) + assert.match(relation.wording, /不能按完整/) + } +}) + +test('jump on an open side yields only a qualified extension hypothesis', () => { + const result = run([b('D4')], 'G4') + assert.equal(result.geometry[0].kind, 'two-space-jump') + assert.equal(result.intents[0].kind, 'side-extension') + assert.equal(result.intents[0].status, 'hypothesis') + assert.match(result.intents[0].wording, /不能仅凭间隔确定是拆/) + assert.ok(result.intents[0].counterEvidence.length) + assert.equal(run([b('D10')], 'G10').intents.length, 0) + assert.equal(run([b('D4'), w('G6')], 'G4').intents.length, 0) + assert.equal(run([b('D4')], 'F4', { boardSize: 9 }).intents.length, 0) + assert.equal(run([b('A4')], 'D4').intents.length, 0) +}) + +test('enemy proximity and low liberties never independently become attack, press or escape', () => { + const result = run([b('D4'), w('C4'), w('D3'), w('E4')], 'D5') + assert.equal(result.geometry[0].kind, 'connected') + assert.equal(result.intents.length, 0) + assert.match(result.limitations.join(' '), /气数.*不能单独证明/) + const contact = run([w('D4')], 'D5') + assert.equal(contact.relations[0].kind, 'enemy-contact') + assert.equal(contact.intents.length, 0) +}) + +test('external unsettled assessments produce explicitly provisional intentions with provenance', () => { + const assessment = { stones: ['K4'], color: 'W', status: 'unsettled', source: 'reviewed-variation', evidence: ['应对变化尚未确认两眼'] } + const result = run([w('K4')], 'K5', { groupAssessments: [assessment] }) + assert.deepEqual(result.intents.map((i) => i.kind), ['attack', 'press']) + for (const intent of result.intents) { + assert.equal(intent.status, 'hypothesis') + assert.match(intent.evidence.join(' '), /reviewed-variation/) + assert.ok(intent.counterEvidence.length) + } + assert.equal(run([w('K4')], 'K5', { groupAssessments: [{ ...assessment, source: '' }] }).intents.length, 0) + assert.equal(run([w('K4')], 'K5', { groupAssessments: [{ ...assessment, status: 'settled' }] }).intents.length, 0) + assert.equal(run([w('K4')], 'K5', { groupAssessments: [{ ...assessment, stones: ['E4'] }] }).intents.length, 0) + const escape = run([b('K4')], 'K6', { groupAssessments: [{ ...assessment, color: 'B' }] }) + assert.deepEqual(escape.intents.map((i) => i.kind), ['escape']) +}) + +test('ownership indexing uses top row first and explicit perspective without classifying life or intent', () => { + const ownership = Array(361).fill(0) + ownership[(19 - 4) * 19 + 3] = 0.81 + for (const ownershipPerspective of ['black', 'white']) { + const result = run([w('D3')], 'D4', { engineFeatures: { ownership, ownershipPerspective, source: 'KataGo' } }) + const feature = result.relations.find((r) => r.kind === 'ownership-at-move') + assert.equal(feature.value, 0.81) + assert.match(feature.evidence, new RegExp(ownershipPerspective)) + assert.equal(result.intents.length, 0) + } + assert.ok(run([], 'D4', { engineFeatures: { ownership } }).limitations.some((s) => s.includes('已忽略'))) + assert.equal(run([], 'D4', { engineFeatures: { ownership: [1], ownershipPerspective: 'white' } }).relations.length, 0) +}) + +test('missing, stale, invalid, duplicate boards and pass fail closed', () => { + assert.equal(run(undefined, 'D4').status, 'abstained') + assert.equal(run([b('D4')], 'D4').status, 'abstained') + assert.equal(run([b('I4')], 'D4').status, 'abstained') + assert.equal(run([b('D4'), w('D4')], 'F4').status, 'abstained') + assert.equal(run([], 'pass').status, 'abstained') + assert.equal(run([], 'T20').status, 'abstained') + assert.equal(run([], 'D4', { boardSize: 1 }).status, 'abstained') +}) + +test('unrelated distant friendly stones do not manufacture a named relation', () => { + const result = run([b('D4')], 'Q16') + assert.equal(result.status, 'abstained') + assert.equal(result.geometry.length, 0) + assert.equal(result.intents.length, 0) +}) + +test('formatter retains all competing anchors, blockers, provisional intentions and limitations', () => { + const ambiguous = formatMoveConceptsForPrompt(run([b('D4'), b('E5')], 'F4')) + assert.match(ambiguous, /锚点=D4/) + assert.match(ambiguous, /锚点=E5/) + assert.match(ambiguous, /ambiguous/) + const provisional = formatMoveConceptsForPrompt(run([b('D4')], 'G4')) + assert.match(provisional, /side-extension\/hypothesis/) + assert.match(provisional, /反证\/未验证：尚未确认/) + assert.match(provisional, /限制：/) + const blocked = run([b('D4'), b('E4')], 'F4') + assert.match(blocked.wording, /相邻连接/) + assert.match(formatMoveConceptsForPrompt(blocked), /阻挡=E4/) +}) + +// Derived semantic fixtures from the six human-reviewed examples. These are development +// regressions, not an independent sample or an estimate of semantic accuracy. +const reviewedRelations = [ + { id: '004:30', color: 'W', move: 'P12', board: [w('P10'), w('Q10'), w('R10'), b('R12'), b('Q14')], kind: 'cap', target: 'R12', empty: ['Q12'] }, + { id: '018:11', color: 'B', move: 'R5', board: [b('Q3'), b('R9'), w('Q5')], kind: 'attachment-under', target: 'Q5', empty: [] }, + { id: '022:16', color: 'W', move: 'F4', board: [w('D4'), b('F3')], kind: 'contact-press', target: 'F3', empty: ['E4'] }, + { id: '023:2', color: 'B', move: 'E3', board: [b('D4'), b('Q4'), b('D16'), b('Q16'), w('F3')], kind: 'diagonal-contact', target: 'F3', empty: [] }, + { id: '033:41', color: 'B', move: 'C3', board: [w('D4'), b('F3'), w('H3')], kind: 'corner-entry', target: 'D4', empty: [], history: [b('F3'), w('H3')] }, + { id: '047:8', color: 'W', move: 'O16', board: [w('Q17'), b('Q15')], kind: 'knight-pressure', target: 'Q15', empty: ['Q16', 'P16'] } +] +const relationColumns = 'ABCDEFGHJKLMNOPQRST' +function transformPoint(point, rotation, reflection) { + if (point.toLowerCase() === 'pass') return point + let x = relationColumns.indexOf(point[0]); let y = Number(point.slice(1)) - 1 + if (reflection) x = 18 - x + for (let i = 0; i < rotation; i++) [x, y] = [18 - y, x] + return `${relationColumns[x]}${y + 1}` +} + +test('reviewed relationship candidates survive rotations, reflections and color swaps', () => { + for (const fixture of reviewedRelations) for (const rotation of [0, 1, 2, 3]) for (const reflection of [false, true]) for (const swapColor of [false, true]) { + const point = (p) => transformPoint(p, rotation, reflection) + const color = (c) => swapColor ? (c === 'B' ? 'W' : 'B') : c + const stone = (s) => ({ point: point(s.point), color: color(s.color) }) + const result = run(fixture.board.map(stone), point(fixture.move), { playerColor: color(fixture.color), moveHistory: fixture.history?.map(stone) }) + const candidate = result.relationCandidates.find((r) => r.kind === fixture.kind && r.enemyAnchor === point(fixture.target)) + assert.ok(candidate, `${fixture.id}, rotation=${rotation}, reflection=${reflection}, colors=${swapColor}`) + assert.equal(candidate.status, 'candidate') + assert.equal(candidate.geometryStatus, 'observed') + assert.equal(candidate.purposeHypothesis.status, 'hypothesis') + assert.ok(candidate.evidence.length && candidate.counterEvidence.length) + assert.deepEqual(candidate.emptyPoints.sort(), fixture.empty.map(point).sort()) + assert.ok(candidate.supportingFriendPoints.every((p) => fixture.board.map(stone).some((s) => s.point === p && s.color === color(fixture.color)))) + assert.ok(result.wording.startsWith(result.relationCandidates[0].wording)) + assert.equal(result.intents.length, 0) + if (fixture.history) assert.match(candidate.wording, /被夹后点三三/) + if (fixture.kind === 'cap' || fixture.kind === 'attachment-under') assert.equal(result.status, 'ambiguous') + } +}) + +test('supported enemy candidate remains available alongside all friendly geometry', () => { + const result = run([w('P10'), w('Q10'), b('R12')], 'P12', { playerColor: 'W' }) + assert.equal(result.status, 'ambiguous') + assert.equal(result.geometry.length, 2) + assert.match(result.wording, /敌棋R12.*镇/) + const prompt = formatMoveConceptsForPrompt(result) + assert.match(prompt, /比较自身发展与对敌关系/) + assert.match(prompt, /术语\[cap\/candidate\] 敌棋=R12/) + assert.match(prompt, /目的\[hypothesis\]/) + assert.match(prompt, /锚点=P10/) + assert.match(prompt, /锚点=Q10/) +}) + +test('cap requires an empty straight gap and supported knight requires open local space', () => { + for (const color of ['B', 'W']) { + assert.ok(!run([w('P10'), w('Q10'), b('R12'), { point: 'Q12', color }], 'P12', { playerColor: 'W' }).relationCandidates.some((r) => r.kind === 'cap')) + for (const point of ['P16', 'Q16']) assert.ok(!run([w('Q17'), b('Q15'), { point, color }], 'O16', { playerColor: 'W' }).relationCandidates.some((r) => r.kind === 'knight-pressure')) + } +}) + +test('contact direction, missing support and central isolated enemies do not manufacture terms', () => { + assert.ok(!run([b('Q3'), w('Q5')], 'Q6').relationCandidates.some((r) => r.kind === 'attachment-under')) + assert.ok(!run([b('Q3'), w('Q5')], 'Q4').relationCandidates.some((r) => r.kind === 'attachment-under' || r.kind === 'contact-press')) + assert.ok(!run([b('D4'), w('F3')], 'G3').relationCandidates.some((r) => r.kind === 'contact-press')) + // A diagonal towards E3 cannot label a contact behind it as尖顶. + assert.ok(!run([b('D4'), w('D3')], 'E3').relationCandidates.some((r) => r.kind === 'diagonal-contact')) + for (const move of ['K9', 'K11', 'K12']) assert.equal(run([w('K10')], move).relationCandidates.length, 0) + for (const move of ['R5', 'P5']) assert.equal(run([w('Q5')], move).relationCandidates.length, 0) + assert.equal(run([b('K10'), w('M9')], 'L9').relationCandidates.length, 0) +}) + +test('three-three entry needs an enemy star and pincer wording needs matching current history', () => { + assert.equal(run([b('D4')], 'C3').relationCandidates.length, 0) + assert.equal(run([], 'C3').relationCandidates.length, 0) + const board = [w('D4'), b('F3'), w('H3')] + const noHistory = run(board, 'C3').relationCandidates.find((r) => r.kind === 'corner-entry') + assert.equal(noHistory.enemyAnchor, 'D4') + assert.doesNotMatch(noHistory.wording, /被夹后/) + for (const moveHistory of [[w('H3'), b('F3')], [b('F3'), w('pass')], [w('F3'), b('H3')], [b('F3'), w('J3')]]) { + assert.doesNotMatch(run(board, 'C3', { moveHistory }).wording, /被夹后/) + } + assert.doesNotMatch(run([w('D4'), w('H3')], 'C3', { moveHistory: [b('F3'), w('H3')] }).wording, /被夹后/) + assert.equal(run([w('D4')], 'C3').status, 'observed') +}) + +test('reviewed knight target uses actual opponent color and does not assert a successful enclosure', () => { + const result = run([w('Q17'), b('Q15')], 'O16', { playerColor: 'W' }) + const candidate = result.relationCandidates.find((r) => r.kind === 'knight-pressure') + assert.equal(candidate.enemyAnchor, 'Q15') + assert.deepEqual(candidate.supportingFriendPoints, ['Q17']) + assert.match(candidate.purposeHypothesis.wording, /可能限制敌棋Q15/) + assert.doesNotMatch(result.wording, /封锁白角|白子.*封锁/) + assert.equal(run([b('Q15')], 'O16', { playerColor: 'W' }).relationCandidates.length, 0) +}) + +test('neutral enemy facts survive all board symmetries without manufacturing a relation term', () => { + for (const rotation of [0, 1, 2, 3]) for (const reflection of [false, true]) for (const swapColor of [false, true]) { + const point = (p) => transformPoint(p, rotation, reflection) + const result = run([{ point: point('D4'), color: swapColor ? 'B' : 'W' }], point('F4'), { playerColor: swapColor ? 'W' : 'B' }) + assert.equal(result.status, 'observed') + assert.equal(result.relationCandidates.length, 0) // Corner tie has no unique inward edge. + const fact = result.relations.find((r) => r.kind === 'enemy-position') + assert.deepEqual(fact.points, [point('D4')]) + assert.ok(Math.abs(fact.offset.dx) + Math.abs(fact.offset.dy) === 2) + assert.match(result.wording, /敌棋.*坐标差/) + } + const contact = run([b('K9'), w('L10')], 'K10') + assert.equal(contact.relationCandidates.length, 0) + assert.match(contact.wording, /敌棋L10直接相邻/) + assert.equal(contact.geometry[0].kind, 'connected') +}) + +function everySymmetry(board, move, playerColor, check) { + for (const rotation of [0, 1, 2, 3]) for (const reflection of [false, true]) for (const swapColor of [false, true]) { + const point = (p) => transformPoint(p, rotation, reflection) + const color = (c) => swapColor ? (c === 'B' ? 'W' : 'B') : c + check(run(board.map((s) => ({ point: point(s.point), color: color(s.color) })), point(move), { playerColor: color(playerColor) }), point) + } +} + +test('round-two press needs close along-side support; a large knight remains a neutral contact', () => { + everySymmetry([b('C12'), w('C9')], 'D9', 'B', (result, point) => { + assert.equal(result.relationCandidates.some((r) => r.kind === 'contact-press'), false) + assert.match(result.wording, /直接相邻/) + assert.equal(result.geometry.find((g) => g.anchor === point('C12')).kind, 'large-knight') + }) + everySymmetry([b('Q16'), w('R14')], 'Q14', 'B', (result, point) => { + const candidate = result.relationCandidates.find((r) => r.kind === 'contact-press') + assert.deepEqual(candidate.supportingFriendPoints, [point('Q16')]) + assert.deepEqual(candidate.emptyPoints, [point('Q15')]) + }) + for (const blocker of [b('Q15'), w('Q15')]) { + const candidate = run([b('Q16'), w('R14'), blocker], 'Q14').relationCandidates.find((r) => r.kind === 'contact-press') + if (blocker.color === 'W') assert.equal(candidate, undefined) + else assert.deepEqual(candidate.supportingFriendPoints, ['Q15']) // A friendly intervening stone is a new direct support, not an empty jump. + } + assert.equal(run([b('D11'), w('C9')], 'D9').relationCandidates.some((r) => r.kind === 'contact-press'), true) + assert.equal(run([b('F9'), w('C9')], 'D9').relationCandidates.some((r) => r.kind === 'contact-press'), false) // support points inward, not along side +}) + +test('small-knight enclosure has a real friendly corner focus and survives every symmetry', () => { + everySymmetry([w('D17')], 'C15', 'W', (result, point) => { + const candidate = result.relationCandidates.find((r) => r.kind === 'corner-enclosure') + assert.ok(candidate) + assert.equal(candidate.enemyAnchor, undefined) + assert.equal(candidate.friendlyAnchor, point('D17')) + assert.match(candidate.wording, /小飞守角/) + assert.equal(candidate.purposeHypothesis.status, 'hypothesis') + const prompt = formatMoveConceptsForPrompt(result) + assert.ok(prompt.includes(`己方角部锚点=${point('D17')}`)) + assert.doesNotMatch(prompt, /敌棋=undefined/) + }) + assert.equal(run([w('D17')], 'C15').relationCandidates.some((r) => r.kind === 'corner-enclosure'), false) + assert.equal(run([b('K10')], 'J8').relationCandidates.some((r) => r.kind === 'corner-enclosure'), false) + for (const point of ['C16', 'E16', 'F14']) assert.equal(run([w('D17'), b(point)], 'C15', { playerColor: 'W' }).relationCandidates.some((r) => r.kind === 'corner-enclosure'), false) +}) + +test('shoulder probe uses enemy diagonal, inward direction and open support from the other side', () => { + everySymmetry([b('J4'), b('Q3'), b('Q5'), w('L3')], 'M4', 'B', (result, point) => { + const candidate = result.relationCandidates.find((r) => r.kind === 'shoulder-probe') + assert.ok(candidate) + assert.equal(candidate.enemyAnchor, point('L3')) + assert.deepEqual(candidate.supportingFriendPoints, [point('J4')]) + assert.deepEqual(candidate.emptyPoints.sort(), [point('K4'), point('L4')].sort()) + assert.ok(candidate.evidence.join(' ').includes(point('Q3'))) + assert.ok(candidate.evidence.join(' ').includes(point('Q5'))) + assert.match(candidate.purposeHypothesis.wording, /可能.*扩张/) + }) + for (const board of [[w('L3')], [b('P4'), w('L3')], [b('J4'), w('L3'), w('K4')], [b('J4'), w('L3'), b('L4')]]) { + assert.equal(run(board, 'M4').relationCandidates.some((r) => r.kind === 'shoulder-probe'), false) + } + assert.equal(run([b('J2'), w('L3')], 'M2').relationCandidates.some((r) => r.kind === 'shoulder-probe'), false) + assert.equal(run([b('J10'), w('L9')], 'M10').relationCandidates.some((r) => r.kind === 'shoulder-probe'), false) +}) + +test('side expansion tolerates enemies behind its anchor but preserves forward pressure and contact', () => { + everySymmetry([b('L16'), w('O17'), w('D17'), w('C15')], 'H16', 'B', (result, point) => { + const intent = result.intents.find((i) => i.kind === 'side-extension') + assert.ok(intent) + assert.match(intent.wording, /争取区域/) + assert.ok(intent.counterEvidence.join(' ').includes(point('O17'))) + }) + for (const point of ['K16', 'J15', 'G16', 'L17', 'M16']) { + assert.equal(run([b('L16'), w(point)], 'H16').intents.some((i) => i.kind === 'side-extension'), false) + } +}) + +test('star diagonal contact describes corner restriction and same-side spacing without inventing replies', () => { + everySymmetry([b('D4'), w('C6'), w('C9'), b('C11')], 'C5', 'B', (result, point) => { + const candidate = result.relationCandidates.find((r) => r.kind === 'diagonal-contact') + assert.ok(candidate) + assert.match(candidate.purposeHypothesis.wording, /可能.*限制挂角子.*向角部发展/) + assert.ok(candidate.evidence.join(' ').includes(point('C9'))) + assert.ok(candidate.evidence.join(' ').includes(point('C11'))) + assert.match(candidate.purposeHypothesis.wording, /空间可能受限/) + assert.doesNotMatch(candidate.wording, /经典定式|守角成功/) + }) + const noSpacing = run([b('D4'), w('C6')], 'C5').relationCandidates.find((r) => r.kind === 'diagonal-contact') + assert.doesNotMatch(noSpacing.purposeHypothesis.wording, /空间可能受限/) + const blocked = run([b('D4'), w('C6'), w('C8'), w('C9'), b('C11')], 'C5').relationCandidates.find((r) => r.kind === 'diagonal-contact') + assert.doesNotMatch(blocked.purposeHypothesis.wording, /空间可能受限/) +}) +// Full reviewed boards are development regressions, never runtime purpose labels. +const developmentFixtures = [ + { move: 'O6', anchor: 'Q5', color: 'W', friends: ['D16','D4','Q5','R3','R4'], enemies: ['O4','Q16','Q3','Q9','R2','S2'] }, + { move: 'S7', anchor: 'S5', color: 'W', friends: ['J3','O5','P2','P4','Q4','R5','S5'], enemies: ['L4','M6','N3','Q5','Q6','R3','R4','R6'] } +] +test('own development retains the pressured real string and frontier geometry in every symmetry and color', () => { + for (const fixture of developmentFixtures) for (const rotation of [0,1,2,3]) for (const reflection of [false,true]) for (const swap of [false,true]) { + const point = p => transformPoint(p, rotation, reflection) + const own = swap ? 'B' : 'W'; const enemy = swap ? 'W' : 'B' + const result = run([...fixture.friends.map(p=>({point:point(p),color:own})),...fixture.enemies.map(p=>({point:point(p),color:enemy}))], point(fixture.move), {playerColor:own}) + const d = result.ownDevelopment.find(d=>d.kind==='own-development' && d.anchor===point(fixture.anchor)) + assert.ok(d, `${fixture.move}: rotation=${rotation}, reflection=${reflection}, swap=${swap}`) + assert.equal(result.primaryFriendlyGeometry.anchor, point(fixture.anchor)) + assert.equal(d.status, 'hypothesis') + assert.ok(d.pressurePoints.length >= 2) + assert.match(d.counterEvidence.join(' '), /没有评估棋块死活/) + assert.match(result.wording, /自身出头|沿边跳出|向中腹发展/) + if (fixture.move==='O6') { + const cap=result.relationCandidates.find(r=>r.kind==='cap') + assert.ok(cap) + assert.ok(cap.counterEvidence.some(e=>e.includes('自身发展竞争解释'))) + assert.equal(cap.enemyAnchor, point('O4')) + } else { + assert.equal(result.primaryFriendlyGeometry.kind, 'one-space-jump') + assert.equal(d.direction,'along-side') + assert.deepEqual(d.corridor,[point('S6')]) + assert.deepEqual(new Set(d.groupStones),new Set(['R5','S5'].map(point))) + assert.ok(result.geometry.some(g=>g.anchor===point('R5') && g.kind==='small-knight')) + } + } +}) +test('blocked corridors, distinct friendly strings and reduced pressure cannot select a guaranteed escape', () => { + const fixture=developmentFixtures[1] + const board=[...fixture.friends.map(w),...fixture.enemies.map(b)] + for (const stone of [w('S6'),b('S6')]) { + const result=run([...board,stone], 'S7', {playerColor:'W'}) + assert.ok(!result.ownDevelopment.some(d=>d.anchor==='S5')) + assert.ok(!result.primaryFriendlyGeometry || result.primaryFriendlyGeometry.kind!=='one-space-jump') + } + // R5 and S5 are no longer one real string when R5 is moved one column away. + const distinct=run([...board.filter(s=>s.point!=='R5'),w('Q7')], 'S7', {playerColor:'W'}) + assert.equal(distinct.primaryFriendlyGeometry,undefined) + const reduced=run(fixture.friends.map(w), 'S7', {playerColor:'W'}) + assert.ok(!reduced.ownDevelopment.some(d=>d.kind==='own-development')) + const inward=run(developmentFixtures[0].friends.map(w), 'O6', {playerColor:'W'}) + assert.equal(inward.ownDevelopment.length,0) +}) +test('multi-anchor side extension keeps enemy competition near the terminal point', () => { + const fixture=developmentFixtures[0] + const result=run([...fixture.friends.map(w),...fixture.enemies.map(b)], 'R8', {playerColor:'W'}) + const extension=result.ownDevelopment.find(d=>d.kind==='side-extension' && d.anchor==='R4') + assert.ok(extension) + assert.deepEqual(extension.corridor,['R5','R6','R7']) + assert.ok(extension.counterEvidence.some(e=>e.includes('Q9'))) + assert.match(extension.wording,/拆边/) + assert.equal(result.primaryFriendlyGeometry,undefined, 'independent Q5 and R3/R4 strings remain separate') + assert.match(formatMoveConceptsForPrompt(result),/自身发展\[side-extension\/hypothesis\]/) +}) diff --git a/tests/teacher-agent-runtime-contract.test.mjs b/tests/teacher-agent-runtime-contract.test.mjs index 2d9d13b..76143e8 100644 --- a/tests/teacher-agent-runtime-contract.test.mjs +++ b/tests/teacher-agent-runtime-contract.test.mjs @@ -126,7 +126,8 @@ test('Teacher prompt requires board, KataGo, and knowledge evidence without temp assert.match(agent, /调用知识库/) assert.match(agent, /匹配棋形、定式、死活、手筋/) assert.match(agent, /像老师讲棋/) - assert.match(agent, /相似匹配只能说“像某某型”/) + assert.match(agent, /知识检索的 exact\/strong 只表示相似度/) + assert.match(agent, /明确命名战术必须有独立读棋证据/) assert.match(agent, /常规定式少讲/) assert.match(agent, /分支列变化/) assert.match(agent, /中盘战详细讲目的和后续/) diff --git a/tests/teacher-analysis-reuse.test.mjs b/tests/teacher-analysis-reuse.test.mjs new file mode 100644 index 0000000..e35c06f --- /dev/null +++ b/tests/teacher-analysis-reuse.test.mjs @@ -0,0 +1,32 @@ +import assert from 'node:assert/strict' +import test from 'node:test' +import { loadConceptModule } from '../scripts/lib/concept_modules.mjs' +const { canReuseTeacherAnalysis } = await loadConceptModule('src/main/services/teacher/analysisReuse.ts') +const analysis = { gameId: 'game', moveNumber: 25, analysisQuality: { totalVisits: 95 }, playedMove: { visits: 1199 } } +const request = { gameId: 'game', moveNumber: 25 } + +test('explicit deeper requests cannot reuse shallow roots even with deep forced-played evidence', () => { + assert.equal(canReuseTeacherAnalysis(analysis, { ...request, requestedMaxVisits: 1200 }), false) + assert.equal(canReuseTeacherAnalysis(analysis, { ...request, requestedMaxVisits: 96 }), true) + assert.equal(canReuseTeacherAnalysis({ ...analysis, analysisQuality: { totalVisits: 1199 } }, { ...request, requestedMaxVisits: 1200 }), true) + assert.equal(canReuseTeacherAnalysis({ ...analysis, analysisQuality: undefined }, { ...request, requestedMaxVisits: 1200 }), false) + assert.equal(canReuseTeacherAnalysis(analysis, request), true) +}) + +test('reuse checks game, move and active trial branch in both directions', () => { + for (const input of [{ ...request, gameId: 'other' }, { ...request, moveNumber: 26 }, { ...request, trialBranchHash: 'trial' }]) { + assert.equal(canReuseTeacherAnalysis(analysis, input), false) + } + const trial = { ...analysis, trialContext: { active: true, branchHash: 'trial' } } + assert.equal(canReuseTeacherAnalysis(trial, request), false) + assert.equal(canReuseTeacherAnalysis(trial, { ...request, trialBranchHash: 'trial' }), true) + assert.equal(canReuseTeacherAnalysis(trial, { ...request, trialBranchHash: 'other' }), false) +}) + +test('invalid or unknown explicit search depth cannot manufacture reusable evidence', () => { + for (const totalVisits of [NaN, Infinity, -1, undefined]) { + assert.equal(canReuseTeacherAnalysis({ ...analysis, analysisQuality: { totalVisits } }, { ...request, requestedMaxVisits: 100 }), false) + } + for (const requestedMaxVisits of [NaN, Infinity, -1, 0]) assert.equal(canReuseTeacherAnalysis(analysis, { ...request, requestedMaxVisits }), false) + assert.equal(canReuseTeacherAnalysis(undefined, request), false) +}) diff --git a/tests/teacher-evidence-tool-gate.test.mjs b/tests/teacher-evidence-tool-gate.test.mjs new file mode 100644 index 0000000..492dba4 --- /dev/null +++ b/tests/teacher-evidence-tool-gate.test.mjs @@ -0,0 +1,50 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { hasInitialCurrentBoardImage, missingTeacherEvidenceTools } from '../src/main/services/teacher/evidenceToolGate.ts' +import { buildVisionEvidenceReport, buildVisionImageContentParts } from '../src/main/services/teacher/visionEvidence.ts' + +const png = 'data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAAEAAAABCAQAAAC1HAwCAAAAC0lEQVR42mP8/x8AAwMCAO+aSAAAAABJRU5ErkJggg==' +function fixture() { + const request = { mode: 'current-move', gameId: 'game', moveNumber: 12, boardImageDataUrl: png } + const report = buildVisionEvidenceReport(request) + const initialMessage = { role: 'user', content: buildVisionImageContentParts(request, report) } + return { intent: 'current-move', request, report, initialMessage, + successfulTools: new Set(['katago_analyzePosition', 'knowledge_matchPosition']) } +} +test('actual valid current-board attachment replaces only the redundant screenshot requirement', () => { + const input = fixture() + assert.equal(hasInitialCurrentBoardImage(input.request, input.report, input.initialMessage), true) + assert.deepEqual(missingTeacherEvidenceTools(input), []) + input.successfulTools.delete('katago_analyzePosition') + assert.deepEqual(missingTeacherEvidenceTools(input), ['katago_analyzePosition']) +}) +test('attached flag, invalid data, mismatched move, or unsent image never satisfy the tool gate', () => { + const input = fixture() + for (const change of [ + { initialMessage: { role: 'user', content: 'metadata only' } }, + { initialMessage: { role: 'user', content: [{ type: 'image_url', image_url: { url: `${png}different` } }] } }, + { request: { ...input.request, boardImageDataUrl: undefined } }, + { request: { ...input.request, boardImageDataUrl: 'data:image/png;base64,AA==' } }, + { request: { ...input.request, moveNumber: 13 } }, + { report: { ...input.report, images: input.report.images.map(image => ({ ...image, valid: false })) } }, + { report: { ...input.report, images: input.report.images.map(image => ({ ...image, moveNumber: 11 })) } }, + { report: { ...input.report, source: 'tool-capture' } } + ]) assert.deepEqual(missingTeacherEvidenceTools({ ...input, ...change }), ['board_captureTeachingImage']) +}) +test('trial attachment must identify the current branch endpoint', () => { + const input = fixture() + input.request.boardContext = 'trial' + input.request.trialBranch = { active: true, baseMoveNumber: 11, moves: [{ color: 'W', gtp: 'D4' }] } + assert.deepEqual(missingTeacherEvidenceTools(input), []) + input.request.trialBranch.baseMoveNumber = 10 + assert.deepEqual(missingTeacherEvidenceTools(input), ['board_captureTeachingImage']) +}) +test('game and range reviews retain screenshot tools even with a valid current-board attachment', () => { + const input = fixture() + for (const intent of ['game-review', 'move-range']) { + input.successfulTools = new Set(['sgf_readGameRecord', 'katago_analyzeGameBatch', 'katago_analyzeMoveRangeKeyMoves', 'knowledge_matchPosition']) + assert.deepEqual(missingTeacherEvidenceTools({ ...input, intent }), ['board_captureTeachingImage']) + input.successfulTools.add('board_captureTeachingImage') + assert.deepEqual(missingTeacherEvidenceTools({ ...input, intent }), []) + } +}) diff --git a/tests/teacher-trial-deepening.test.mjs b/tests/teacher-trial-deepening.test.mjs new file mode 100644 index 0000000..d9271e4 --- /dev/null +++ b/tests/teacher-trial-deepening.test.mjs @@ -0,0 +1,37 @@ +import assert from 'node:assert/strict' +import test from 'node:test' +import { readFileSync } from 'node:fs' +import ts from 'typescript' + +// Execute the production dispatcher with only the two engine transports replaced. +const source = readFileSync(new URL('../src/main/services/teacherAgent.ts', import.meta.url), 'utf8') +const ast = ts.createSourceFile('teacherAgent.ts', source, ts.ScriptTarget.Latest, true) +const dispatch = ast.statements.find(node => ts.isFunctionDeclaration(node) && node.name?.text === 'analyzeTeacherPosition') +assert.ok(dispatch) +const code = ts.transpileModule(`export ${dispatch.getText(ast)}`, { compilerOptions: { target: ts.ScriptTarget.ES2022, module: ts.ModuleKind.ES2022 } }).outputText +const prefix = `export const calls=[];export let branch='trial';export function setBranch(value){branch=value} const analyzePosition=async(...args)=>{calls.push(['main',args]);return {gameId:args[0],moveNumber:args[1]}};const analyzeTrialPositionWithProgress=async(input)=>{calls.push(['trial',input]);return {trialContext:{branchHash:branch}}};` +const { analyzeTeacherPosition, calls, setBranch } = await import(`data:text/javascript;base64,${Buffer.from(prefix + code).toString('base64')}`) +const trial = { active: true, baseMoveNumber: 9, moves: [{ color: 'B', gtp: 'D4', row: 15, col: 3 }], branchHash: 'trial' } + +test('deeper main-line analysis uses the requested budget and teacher cancellation group', async () => { + calls.length = 0 + await analyzeTeacherPosition({ id: 'run', request: {} }, 'game', 10, 1200) + assert.deepEqual(calls, [['main', ['game', 10, 1200, { runId: 'run', group: 'teacher' }]]]) +}) + +test('trial deepening calls the trial engine with the exact branch instead of same-number main line', async () => { + calls.length = 0; setBranch('trial') + await analyzeTeacherPosition({ id: 'run', request: { gameId: 'game', boardContext: 'trial', trialBranch: trial } }, 'game', 10, 1200) + assert.deepEqual(calls, [['trial', { gameId: 'game', baseMoveNumber: 9, trialMoves: trial.moves, maxVisits: 1200, runId: 'run', group: 'teacher' }]]) +}) + +test('wrong trial game, move, or returned branch is rejected without silently analyzing main line', async () => { + const state = { id: 'run', request: { gameId: 'game', boardContext: 'trial', trialBranch: trial } } + calls.length = 0 + await assert.rejects(analyzeTeacherPosition(state, 'other', 10, 1200), /试下/) + await assert.rejects(analyzeTeacherPosition(state, 'game', 9, 1200), /试下/) + assert.equal(calls.length, 0) + setBranch('wrong') + await assert.rejects(analyzeTeacherPosition(state, 'game', 10, 1200), /不同分支/) + assert.equal(calls.some(call => call[0] === 'main'), false) +}) diff --git a/tests/territory-judgement-contract.test.mjs b/tests/territory-judgement-contract.test.mjs index 35ebb86..cc01180 100644 --- a/tests/territory-judgement-contract.test.mjs +++ b/tests/territory-judgement-contract.test.mjs @@ -13,9 +13,10 @@ test('KataGo root ownership is preserved for positional judgement', () => { assert.match(types, /ownership\?: number\[\]/) assert.match(types, /ownershipStdev\?: number\[\]/) assert.match(katago, /rootInfo\?: \{/) - assert.match(katago, /rootInfo\.ownership/) - assert.match(katago, /rootInfo\.ownershipStdev/) - assert.match(katago, /ownership: Array\.isArray\(response\.rootInfo\.ownership\)/) + assert.match(katago, /readRootOwnership\(response, boardSize\)/) + const ownership = read('src/main/services/analysis/ownership.ts') + assert.match(ownership, /response\.ownership/) + assert.match(ownership, /response\.ownershipStdev/) }) test('renderer has a reusable territory judgement model and premium board overlay', () => { diff --git a/tests/tool-execution-history.test.mjs b/tests/tool-execution-history.test.mjs new file mode 100644 index 0000000..b27360d --- /dev/null +++ b/tests/tool-execution-history.test.mjs @@ -0,0 +1,48 @@ +import assert from 'node:assert/strict' +import test from 'node:test' +import { toolExecutionHistory } from '../src/main/services/llm/toolExecutionHistory.ts' + +const call = (id, name, args = {}) => ({ + id, + type: 'function', + function: { name, arguments: JSON.stringify(args) } +}) + +test('repair history preserves actual concept JSON and image with their tool call', () => { + const toolCall = call('knowledge-1', 'lookup_knowledge', { move: 'D4' }) + const actualConcepts = JSON.stringify({ newConcepts: [{ id: 'liberties', explanation: '实际知识结果' }] }) + const image = { role: 'user', content: [{ type: 'image_url', image_url: { url: 'data:image/png;base64,dGVzdA==' } }] } + const result = { ok: true, toolResult: actualConcepts, followupMessages: [image] } + const repairMessages = [{ role: 'user', content: '分析此步' }, ...toolExecutionHistory(toolCall, result)] + + assert.deepEqual(repairMessages.slice(1), [ + { role: 'assistant', content: '', tool_calls: [toolCall] }, + { role: 'tool', name: 'lookup_knowledge', tool_call_id: 'knowledge-1', content: actualConcepts }, + image + ]) + assert.deepEqual(JSON.parse(repairMessages[2].content).newConcepts, [{ id: 'liberties', explanation: '实际知识结果' }]) + assert.equal(repairMessages[3], image) + assert.deepEqual(result.followupMessages, [image]) +}) + +test('failed tool result remains verbatim and associated with the actual call', () => { + const toolCall = call('katago-failed', 'analyze_move', { move: 'Q16' }) + const actualFailure = 'KataGo 分析超时，未取得结果。' + assert.deepEqual(toolExecutionHistory(toolCall, { ok: false, toolResult: actualFailure, followupMessages: [] }), [ + { role: 'assistant', content: '', tool_calls: [toolCall] }, + { role: 'tool', name: 'analyze_move', tool_call_id: 'katago-failed', content: actualFailure } + ]) +}) + +test('multiple executions retain chronological call/result pairing for a new turn', () => { + const first = call('katago-1', 'analyze_move') + const second = call('knowledge-2', 'lookup_knowledge') + const history = [ + ...toolExecutionHistory(first, { ok: true, toolResult: '{"winrate":0.52}', followupMessages: [] }), + ...toolExecutionHistory(second, { ok: false, toolResult: '未匹配到知识', followupMessages: [] }) + ] + assert.deepEqual(history.map(m => m.role), ['assistant', 'tool', 'assistant', 'tool']) + assert.equal(history[0].tool_calls[0].id, history[1].tool_call_id) + assert.equal(history[2].tool_calls[0].id, history[3].tool_call_id) + assert.deepEqual(history.filter(m => m.role === 'tool').map(m => m.content), ['{"winrate":0.52}', '未匹配到知识']) +}) diff --git a/tests/vision-evidence-behavior.test.mjs b/tests/vision-evidence-behavior.test.mjs new file mode 100644 index 0000000..67e47c7 --- /dev/null +++ b/tests/vision-evidence-behavior.test.mjs @@ -0,0 +1,22 @@ +import test from 'node:test' +import assert from 'node:assert/strict' +import { verifyVisionEvidenceMarkdown } from '../src/main/services/teacher/visionEvidenceVerifier.ts' + +const attached = { required: true, attached: true } +test('successful captured image cannot be denied by alternate missing-image wording', () => { + for (const text of [ + '工具没有返回图像，所以我不能把棋盘图当作已核对的依据。', + '由于这次没取得截图，局部棋形判断只依据棋谱。', + '目前未能获取棋盘图像。', + '截图工具未返回图片。', + '本轮未附图，工具没有返回可查看截图。', + "The tool didn't return an image." + ]) { + assert.equal(verifyVisionEvidenceMarkdown(text, attached)[0]?.type, 'false-no-board-image-claim', text) + } +}) +test('image verification preserves uncertainty about the position and real missing-image reports', () => { + assert.deepEqual(verifyVisionEvidenceMarkdown('图中没有敌棋，不能凭小飞配置证明守角已经成功。', attached), []) + assert.deepEqual(verifyVisionEvidenceMarkdown('工具没有返回图像。', { required: false, attached: false }), []) + assert.equal(verifyVisionEvidenceMarkdown('暂时无法核对。', { required: true, attached: false })[0]?.type, 'required-image-missing') +})