diff --git a/CHANGELOG.md b/CHANGELOG.md index 09d0007..d5d0fec 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,15 @@ ### Added -- **Codex CLI now has the pre-write gate, which it never had.** `hooks/hooks-codex.json` wired only `SessionStart` and `UserPromptSubmit`, so on Codex a TaskFlow run had no gate, no evidence capture, and one reminder line per prompt — the flow was guidance there, not a lifecycle. `runtime.md` recorded this as a limitation of the host. It was not: Codex's own hooks documentation (read 2026-10-29) documents `PreToolUse` firing on `apply_patch`, `hookSpecificOutput.permissionDecision: "deny"`, and `additionalContext` on both tool events. The wiring now carries `PreToolUse` with matcher `apply_patch`, and the gate learned the shape that host sends. +- **A task's outside lookups are now recorded, and a claim to have made one is checked against the record.** `hooks/external-evidence` (`PostToolUse` on `WebSearch`/`WebFetch`) appends one `search|` or `fetch|` line to `/taskflow/external`. That store is deliberately not `taskflow/evidence`: the gate releases a stage only from `Skill`/`Agent|Task`/`mcp__*`, and a built-in tool must not be able to satisfy it — 2026-09-27's R12, unchanged. So the two predicates live in two files with two consumers, and the smoke suite asserts the separation by requiring that a task holding real lookups and no invocation is still refused for the missing stage. `task approve` then reads the external store for the opposite purpose: a `prd.md` answering `- Depends on external facts: yes` has to have every `- Source:` backed by a recorded lookup. Without that, the section is a form filled in from memory — and `achieved/2026-09-08-runtime-hooks` is the case in point, naming Anthropic's hooks reference and Codex's hooks guide in prose with no URL and no date, in a repository that has no way to tell whether either was opened. + +- **The question "does this rest on something outside the repository" is now asked at the three points where the answer can change.** `skills/taskflow/SKILL.md` puts it before Phase 1 converges, again at the Phase 3 spec decision — where defining an interface or a compatibility boundary is exactly when "only our own code" stops being true — and once more before the Plan is approved, against Steps that did not exist when the answer was first given. This is the half the field alone does not supply: every source this workflow points an Agent at is inside the repository, so nothing prompted the question, and a `no` written by default is indistinguishable from one reached by thinking. The `inspect` enumeration at `SKILL.md:54` is deliberately **not** extended — its seven listed sources are "open the repository and look", and adding a judgement question to that list would dress a decision up as an inspection. The smoke suite counts them to keep a ninth from being added quietly. + +- **A `no` has to point at where the work is settled, not describe it.** `- Why not:` must contain a backticked path or a `path:line`; "this is an internal change" is refused, because it is the sentence an unexamined `no` produces. The path is not resolved — a Plan may predate a move — so the check requires a pointer without claiming the target is still there. + +- **A `prd.md` now has to say whether its task's conclusions depend on a fact from outside this repository, and name what it found when they do.** `hooks/task approve` reads a new `## External Facts` section and refuses four shapes: no answer, an answer that is neither `no` nor `yes`, `no` without a `- Why not:` line, and `yes` without at least one `- Source:` carrying an absolute `http(s)` URL and a `- What it settled:` line naming the conclusion that source changed. Measured 2026-10-31 across 94 archived tasks, `## Reference Pointers` holds **382 bullets and not one is a URL**, 3 of 94 tasks have a `reference/` directory, and the only four third-party URLs in the whole archive sit in a single task. That number cannot prove TaskFlow suppresses search — most work here is internal and should cite nothing — which is why the rule records the judgement instead of counting the searches. `hooks/task promote` writes the section unanswered on purpose, so a new task is refused until someone answers it. + +- **Codex CLI now has the pre-write gate, which it never had.** `hooks/hooks-codex.json` wired only `SessionStart` and `UserPromptSubmit`, so on a Codex TaskFlow run had no gate, no evidence capture, and one reminder line per prompt — the flow was guidance there, not a lifecycle. `runtime.md` recorded this as a limitation of the host. It was not: Codex's own hooks documentation (read 2026-10-29) documents `PreToolUse` firing on `apply_patch`, `hookSpecificOutput.permissionDecision: "deny"`, and `additionalContext` on both tool events. The wiring now carries `PreToolUse` with matcher `apply_patch`, and the gate learned the shape that host sends. - **`capability-gate` reads a patch as well as a path.** Codex reports an edit as `tool_input.command` holding the patch text, with no `file_path` field at all, so the gate previously saw no input and exited 0 on every write. It now parses the four directives that name a target — `*** Update File:`, `*** Add File:`, `*** Delete File:`, `*** Move to:` — normalizes and joins each path exactly as it does `file_path`, and applies the same base/stage/task-directory rule to each. A move is `Update File: ` plus `Move to: `, so both ends are judged: the document really does leave one task and arrive at another, and reading only the destination leaves the departure unchecked. One patch naming several files is judged path by path, and any refusal ends the run, so a patch is all-or-nothing exactly as the host applies it. A patch that is non-empty but carries no recognizable directive is **refused**, because a gate that cannot read its host must not report success; an empty command is the one exception, since no edit happened. `hooks/task`'s approval decision moved to a new `hooks/approval-check` so the gate and `completion-gate` share one implementation rather than two. diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 75e48f2..4c77071 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -40,6 +40,10 @@ Everything above is about **isolation** — keeping a task's work off whatever b **A task's documents land with the task, inside its pull request.** Its PRD, Spec, and Plan; the verification record written into the Plan; and the archive that moves `TaskFlowDocs//` to `achieved/` — each is a commit on the task's branch, and they land in the task's own pull request before it is merged. They are part of what the task delivers, so they are reviewed with it. Do not push them to the base branch outside that pull request, and do not give them a pull request of their own. Either one splits the record from the change it describes, and the split is what strands a branch's commits when the pull request is merged without them. +**The archive is not an exception to that, and running `task complete` before the push is the normal order.** Nothing about the transaction needs the merge: `hooks/task complete` reads the working tree, moves the directory, and rewrites the Todo entry, and none of that consults `main`. So the sequence is — finish the work, `task complete`, commit the code, the documents and the archive together, push, open the pull request. A reviewer then sees the finished record beside the change, which is the reason the rule exists. + +Getting this backwards is a real failure and it looks harmless: archiving after the merge leaves the directory and Todo rewrite on no branch, so they must either be pushed to the base outside any pull request — which the paragraph above forbids — or given a second pull request, which the same paragraph forbids for the same reason. The teardown message `hooks/task complete` prints is about the worktree, not the archive: `git worktree remove` waits for the merge because removing a directory that still holds the only copy of anything is the damaging mistake. That condition says nothing about where the archive commit goes. + This holds whatever the author's push permission. Being able to push the base branch is not a reason to land a task's documents outside the task's pull request — they are not a separate change that happens to be small. A task that lands without a pull request at all, because its whole change is bookkeeping, takes its documents the same way as its change. **The landing claim is recorded in that pull request, not in the task's documents.** What the task did and how it landed — the commit it was pushed as, the CI run that verified it — belongs in the pull request body, where it is read with the change. Nothing is written back into `prd.md` or `plan.md` after the merge, because that would be a task document pushed to the base outside its own pull request, which is what the paragraph above forbids. A fact that exists only once the merge has happened therefore has no place in the task's documents, and needs none: what can still be checked afterwards is checked, not recorded. Write such a criterion in the Plan as a check — "verify against the merge topology" — and not as a promise to write the result back. diff --git a/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/plan.md b/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/plan.md new file mode 100644 index 0000000..4f51938 --- /dev/null +++ b/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/plan.md @@ -0,0 +1,141 @@ +# Plan — 外部事实：先判断，再取证 +> Task version: v1 +> Status: in_progress + +No spec required — small, self-contained task. + +## Reference Pointers + +- `hooks/task:1383-1391` — `approve` 现有的 `Skills / Tools Used` 非空检查；新的校验插在它后面，形态照抄。 +- `hooks/task:1497-1545` — `## Approval` 的写入是这段之后的第一处写；新校验必须在它之前。 +- `hooks/task:813-826` — `promote` 写 PRD 骨架的 `printf` 块。 +- `skills/taskflow/references/artifacts.md:25` — 「hook 按名匹配的小节」清单。 +- `skills/taskflow/references/artifacts.md:54-68` — PRD outline。 +- `hooks/smoke-test:1877-2070` — 既有 approve 夹具，新用例并入这里。 + +## Related Tasks + +- 来源：`TaskFlowDocs/todo.md` 的 `TF-20261031-61b973`（由 `TF-20261027-e8c6dc` 重新界定）。 +- 上游测量：`TF-20260919-90797c`（本任务不处理，见 PRD Out of Scope）。 +- 形态参照：`achieved/2026-09-27-capability-pre-write-gate/`（`approve` 先校验后写入的写法）。 + +## Skills / Tools Used + +- [PRD] `agent-skills:spec-driven-development` — purpose: 把一条被重新界定的 todo 收敛成可验收的需求；outcome: succeeded; incorporated: 把「测量行为变化」改成「记录判断」这个可检查的形状，并把混淆写进 Background 而不是藏进 Risks。 +- [Plan] `agent-skills:planning-and-task-breakdown` — purpose: 决定四处的先后与验证顺序；outcome: succeeded; incorporated: 先写校验再写骨架（骨架产生的字段要被校验接受），smoke 与文档放最后。 +- [Test] `Unaided — no capability applied to this phase; considered: testing strategy` — 断言形态沿用 `hooks/smoke-test:1877` 既有夹具，未引入外部能力。 + +## Preconditions + +- [x] 已读 `CONTRIBUTING.md`（分支/worktree）、`CODE_STYLE.md`（脚本风格）、`artifacts.md`（字段清单）。 +- [x] 隔离：工作树 `.worktrees/external-facts`，分支 `feat/external-facts`，基线 `main` @ `b4b80a6`。 +- [x] 已实测：94 个 achieved 任务的 `prd.md` 无一含 `External Facts`，字段名无冲突；`prd.md` 与 `plan.md` 成对存在（0 例外），所以不必先改 `approve` 的「prd.md 可选」行为。 + +## Approval + +- Status: approved +- Approved by: user +- Approved at: 2026-10-31 22:58 +0800 +- Approved version: v1 +- Approved scope: PRD / Plan + +## Steps + +### Step 1 — `approve` 校验 `## External Facts` + +- Goal: 让 `approve` 在写 `## Approval` 之前读 `prd.md` 的 `## External Facts`，把 R2/R3 的四种坏形态各自拒绝并报出具体原因。 +- Dependencies: 无。 +- Files: `hooks/task`。 +- Implementation checklist: + - [x] 在 `cap_skills` 非空检查之后、`approved_version` 之前插入一段，读 `$prd`。 + - [x] 缺文件或小节缺失 → 拒绝，消息给出两种合法写法的其中一行供照抄（A5）。 + - [x] 小节存在但正文为空 → 拒绝，消息与 `Skills / Tools Used` 的「missing or empty」同形（A8）。 + - [x] `- Depends on external facts:` 既不是 `no` 也不是 `yes` → 拒绝并列出两个合法值。 + - [x] `no` 且缺 `- Why not:` → 拒绝（A1 的反面）。 + - [x] `yes` 且零条 `- Source:` → 拒绝（A2）。 + - [x] 任一 `- Source:` 的值不以 `http://` 或 `https://` 起头 → 拒绝并说出「不是绝对 http(s) URL」（A3/A4）。 + - [x] 每条 `- Source:` 之后必须有 `- What it settled:` → 缺则拒绝。 + - [x] 所有失败都发生在任何 `## Approval` 写入之前，且 `fail` 前不 `mv` 任何临时文件。 +- Acceptance: PRD 的 A1–A5、A8。 +- Verification: 手工构造六种坏 `prd.md` 各跑一次，确认消息与退出码；再用真实仓库的一个任务跑通一次正向。 +- Rollback: 单段代码，`git revert` 即恢复。 +- Status: done + +### Step 2 — `promote` 的 PRD 骨架与 `artifacts.md` + +- Goal: 新任务的 `prd.md` 一出生就带着这个小节，且字段名与 Step 1 的匹配逐字一致。 +- Dependencies: Step 1（骨架的默认文本必须能被校验接受或至少给出可照抄的提示）。 +- Files: `hooks/task`、`skills/taskflow/references/artifacts.md`。 +- Implementation checklist: + - [x] `hooks/task` 的 PRD 骨架加 `## External Facts`，正文放一行 `- Depends on external facts: ` 与 `- Why not: `（空值，等作者填）。 + - [x] 骨架的默认值必须**不能**直接通过 Step 1 的校验 —— 空值就是「没回答」，与空的 `Skills / Tools Used` 同等对待。 + - [x] `artifacts.md` 的 PRD outline 加同一小节，位置与骨架一致。 + - [x] `artifacts.md:25` 的「hook 按名匹配的小节」清单里加 `## External Facts`，并补一条说明它的两个字段名由 `approve` 逐字读。 +- Acceptance: PRD 的 A7；`promote` 生成的新 `prd.md` 在不填该小节时 `approve` 拒绝。 +- Verification: 在夹具里 `promote` 一个新任务，读回 `prd.md` 确认小节存在；直接 `approve` 确认被拒且消息指向该小节。 +- Rollback: 两处 `printf` 与一行清单，`git revert`。 +- Status: done + +### Step 3 — smoke 断言与 SKILL.md + +- Goal: 让 Step 1 的每条拒绝在 CI 里被执行，且每条都有一个能被变异打红的具名断言；SKILL.md 说明这个字段是什么。 +- Dependencies: Step 1、Step 2。 +- Files: `hooks/smoke-test`、`skills/taskflow/SKILL.md`。 +- Implementation checklist: + - [x] 在既有 approve 夹具（`hooks/smoke-test:1877`）上，给 `ap_task/prd.md` 写入合法的 `## External Facts`，让原有正向用例继续通过。 + - [x] 新增 case：小节缺失 / 正文为空 / 值非 no|yes / `no` 缺 `Why not` / `yes` 缺 `Source` / `Source` 非 URL / 缺 `What it settled`，各一条具名断言。 + - [x] 每条断言都检查**消息片段**，不只检查退出码 —— 否则八条拒绝退化成一条。 + - [x] 变异验证：逐条破坏 Step 1 的分支，确认**对应**的断言变红，然后逐字节恢复（`diff -q`）。 + - [x] `SKILL.md:204` 的「`prd.md` must state」列表加一条；`SKILL.md:54` 的 `inspect` 八项**不动**（PRD Out of Scope）。 +- Acceptance: PRD 的 A6；变异 7 次各自红在对应用例，恢复后 `ALL SMOKE PASSED`。 +- Verification: `bash hooks/smoke-test`；七次变异的结果写进 `## Verification / Review`。 +- Rollback: 删除新增 case 与那一行 SKILL.md。 +- Status: done + +## Checkpoints + +- Step 1 后：`bash -n hooks/task` 通过；真实任务的 `prd.md` 填好两种形态各一次都能 `approve`。 +- Step 2 后：夹具里 `promote` 出的新任务，不填该小节时 `approve` 被拒。 +- Step 3 后：`bash hooks/smoke-test` → `ALL SMOKE PASSED`；七次变异各自红在对应用例。 + +## Verification / Review + +### 实测（2026-11-01） + +| 检查 | 结果 | +|---|---| +| `bash hooks/smoke-test` | `ALL SMOKE PASSED` | +| `bash -n hooks/task hooks/smoke-test` | 通过 | +| `hooks/task` 变异后恢复 | 六次变异后逐字节回到 pristine（`diff -q`） | + +### 六次变异（每次红在自己的具名断言上） + +| 变异 | smoke 输出 | +|---|---| +| `no` 不检查 `- Why not:` | `FAIL external facts message for no without a reason` | +| `yes` 允许零条 `- Source:` | `FAIL external facts message for yes without a source` | +| `- Source:` 不校验是否为绝对 http(s) URL | `FAIL external facts message for source is not a URL` | +| 不要求每条 `Source` 配 `- What it settled:` | `FAIL external facts message for source settled nothing` | +| 接受 `no` / `yes` 之外的值 | `FAIL external facts message for not no-or-yes` | +| 空答案（小节缺失或空）直接放行 | `FAIL external facts message for missing section` | + +### 实现过程中被自己抓到的两个缺陷 + +**一、一行死代码。** 第一版在 `## External Facts` 正文为空时先 `fail`，再进入 `case`。但空正文意味着 `ext_answer` 也是空，`case` 的 `""` 分支会打出**同一条消息** —— 那行 `fail` 永远不会是唯一的拒绝原因，删掉它不影响任何断言。删掉后 Step 3 的「missing section」与「empty section」两条用例仍然各自通过，证明它们走的是同一个 `""` 分支。 + +这是变异验证的价值所在：那行看起来像「更早、更明确地拒绝」，实际上是一条 `grep -v` 就能删掉、而测试不会有任何反应的代码。 + +**二、`prd.md` 自己成了新规则的第一个用户。** 本任务的 PRD 在 `approve` 之后才补上 `## External Facts`，因为批准时规则还不存在。补的时候必须真的回答，而不是写一句占位 —— 答案 `no` 的理由是「本任务的每条验收标准都由仓库内既有实现判定，无一处依赖外部规范」。 + +## Change Log + +- 2026-11-01 — Step 3 完成：smoke 新增 9 条 `## External Facts` 断言；SKILL.md 与 artifacts.md 同步；本任务 PRD 补上该小节。 + +## Follow-ups + +- **`no` 分支的比例与理由雷同度值得回看。** 落地一段时间后统计 `Depends on external facts: no` 在 `achieved/` 里的占比，若接近 100% 且 `Why not` 措辞雷同，说明字段退化成了橡皮图章（PRD Risks 第 1 条）。 +- **`TF-20260919-90797c` 的对照测量仍然悬空**，本任务按用户 2026-10-31 裁定不合并。 + +## Version History + +- v1 — planning. diff --git a/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/prd.md b/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/prd.md new file mode 100644 index 0000000..2def291 --- /dev/null +++ b/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/prd.md @@ -0,0 +1,84 @@ +# 外部事实：先判断，再取证 +> Task version: v1 +> Status: in_progress + +## Goal + +让每个任务在 `prd.md` 里回答一次「这个任务的结论是否依赖仓库外的事实」，并在答案为「是」时真的去检索、把来源记下来。答案不是靠自觉写散文，而是 `task approve` 读得出、能拒绝的一种形态。 + +## Background / Confirmed Facts + +**为什么现在做。** 2026-10-11 用户观察到「用了 TaskFlow 之后 Agent 似乎变得不喜欢做网络搜索」，要求先查是不是真的。当时立的 Todo `TF-20260919-90797c` 只覆盖测量，后又分出 `TF-20261027-e8c6dc`。2026-10-31 用户把它重新界定为：**「在判断需要外部信息、调研、外部信息确认等需求的时候，调用搜索」** —— 要的不是「多搜」，而是「该搜的时候真的搜」。测量因此从目的降为背景。 + +**三处机制，2026-10-31 逐条复核仍成立。** 原 Todo 引的是旧行号，位置已经漂移，结论没变： + +1. `skills/taskflow/SKILL.md:54` 的 `inspect` 枚举八项 —— repository、`index.md`、project rules、active and achieved tasks、tests、configuration、Git state、uncommitted changes —— **全部在仓库内或 Agent 自己的状态里，没有一项指向仓库外**。 +2. 「取外部事实」不是一个动作，只是 Phase 2 的一个产物描述。`skills/taskflow/SKILL.md:172` 把 `searching external sources` 挂在 Phase 2 的能力列，而 `SKILL.md:44` 和 `SKILL.md:219` 都写明 `reference/` 是 **optional**。于是模型里的因果关系是「因为要写 reference 笔记，所以去搜」，而不是「因为答案在仓库外，所以去搜」—— 不写笔记就不触发。 +3. `hooks/capability-evidence:48-50` 只认 `Skill` / `Agent|Task` / `mcp__*`，`WebSearch` / `WebFetch` 不被计入任何证据。这是 2026-09-27 的 R12 有意为之（`achieved/2026-09-27-capability-pre-write-gate/prd.md:61`），目的是不让内置工具满足能力门禁。后果是：**检索在 TaskFlow 里既不拦也不奖。** + +**测量数据，2026-10-31 在 94 个 achieved 任务上重跑：** + +| 指标 | 原 Todo（2026-10-27，91 任务） | 今天（94 任务） | +|---|---|---| +| `## Reference Pointers` 的 bullet 总数 | 365 | **382** | +| 其中含 URL 的 | 0 | **0** | +| 整个任务目录含 `reference/` 的 | 3 / 91 | **3 / 94** | +| 整个归档里第三方主机 URL 总数 | 4 | **4**，且全部集中在同一个任务（`2026-09-10-audit-hook-opportunities`） | +| 其余 URL | 71 条 | **71 条，全是 `github.com/hkwuks/TaskFlow` 自身** | + +**这个数字不能用来证明 TaskFlow 抑制了搜索。** 本仓库的绝大多数任务是内在工作：改自己的 bash hook、改自己的 `SKILL.md`、修自己的 smoke 套件。这类任务本来就应当 0 个外部 URL。原 Todo 自己写了这一句，但它的 Next action 跳过了它直接给方案。**所以本任务不承诺「外部检索次数会上升」，也不做对照实验** —— 用户 2026-10-31 裁定：接受这个混淆，只写明。 + +**能检查的只能是「问题被问了并记录」，不是「行为被改变了」。** 这是本任务验收标准的形状，也是它和「加一句散文」的根本区别：散文无法被检查，字段可以。 + +## Requirements + +- **R1 — 一份文档承担这个判断，且它必须是 PRD。** 判断属于需求阶段（决定要不要查），不属于计划阶段（决定怎么查）。因此字段只加在 `prd.md`。 +- **R2 — 字段是一个顶层小节 `## External Facts`，内容是下面两种形态之一**，第一行由 `hooks/task approve` 逐字匹配： + - `- Depends on external facts: no` —— 下一行必须是 `- Why not: <一句话>`。 + - `- Depends on external facts: yes` —— 后面必须跟至少一条 `- Source: `，每条带 `- What it settled: <这句话改变了哪个结论>`。 +- **R3 — 答案为 `yes` 时，`Source` 必须是绝对 `http` / `https` URL。** 不是仓库内路径、不是「我读了很多代码」、不是 `mcp__` 工具名。判断标准是这句话**能否在仓库里被证伪** —— 不能，才是外部事实。`http` 合法（允许内网文档站），`ftp` / `file` / `git@` 不合法。 +- **R4 — `approve` 在写 `## Approval` 之前校验 R2/R3，失败即拒绝并给出可执行的修法。** 与现有 `Skills / Tools Used` 的校验同一位置、同一形态：先于任何写入，一次只报一条具体原因。 +- **R5 — 骨架与文档同步。** `hooks/task promote` 写出的 PRD 骨架、`skills/taskflow/references/artifacts.md` 的 PRD outline 与「hook 匹配的字段清单」三处都要出现该小节，且字段名逐字一致。 +- **R6 — 不引入门禁以外的行为变化。** 不改 `capability-evidence` 的谓词（R12 的理由仍成立），不新增 hook，不联网，不改 `hooks/completion-gate`。 +- **R7 — 已归档的 94 个任务不回填，且 `approve` 不在 `achieved/` 上运行。** 与 `done`-only 规则那次同处理。 + +## Acceptance Criteria + +- **A1** 一个 `prd.md` 含 `## External Facts` + `- Depends on external facts: no` + `- Why not: …` 的任务可以 `approve` 通过。 +- **A2** 同一文档把答案改成 `yes` 却不写任何 `- Source:` 行时，`approve` 退出非 0 并点名「yes 需要至少一条 Source」。 +- **A3** 答案是 `yes` 且 `- Source:` 的值是 `hooks/task` 这类仓库内路径时被拒绝，消息说明「不是 URL」。 +- **A4** `- Source:` 写成 `ftp://` 或 `git@…` 时被拒绝；写成 `http://` 时通过。 +- **A5** 整个小节缺失时 `approve` 拒绝，消息同时给出两种合法写法中的一行，让人照抄即可。 +- **A6** 上述四条由 `hooks/smoke-test` 在既有 `task approve` 夹具上覆盖，且每条都有一个能被变异打红的具名断言。 +- **A7** `skills/taskflow/references/artifacts.md` 的「Section headings a hook matches by name」清单里出现 `## External Facts`，`## Items` 之外的位置不变。 +- **A8** 一个 `prd.md` 只有 `## External Facts` 标题、正文为空时被拒绝（与 `Skills / Tools Used` 现有「missing or empty」同形），而不是当成 `no`。 + +## External Facts + +- Depends on external facts: no +- Why not: 这条规则本身就是判断标准 —— 「结论能否被仓库外的事实证伪」。本任务的每一条要求、每一个验收标准都由仓库内的既有实现（`hooks/task`、`hooks/smoke-test`、`artifacts.md`、`SKILL.md`）判定，无一处依赖外部规范、版本或第三方文档。三项能力调用均为本机可执行，不涉及远端。 + +## In Scope + +- `hooks/task` 的 `approve`：新增一段校验，位置在 `cap_skills` 检查之后、任何 `## Approval` 写入之前。 +- `hooks/task` 的 `promote`：PRD 骨架增加该小节。 +- `hooks/smoke-test`：在既有 approve 夹具上新增用例。 +- `skills/taskflow/references/artifacts.md`：PRD outline 与字段清单。 +- `skills/taskflow/SKILL.md`：在 Phase 1 说明该字段的含义。 + +## Out of Scope + +- **不改 `SKILL.md:54` 的 `inspect` 八项枚举。** 加第九项「外部事实」会让一个判断问题伪装成一个检查动作，而这八项是「打开仓库就能看」的清单，性质不同。机制 (1) 的修补方式是让 `## External Facts` 每个任务都必须回答，而不是往枚举里塞一条。 +- **不改 `capability-evidence` 的谓词。** R12 的理由（内置工具不得满足能力门禁）仍然成立，让 `WebSearch` 进入证据池会把门禁变成一个「搜一下就能过」的东西。 +- **不做对照实验，不承诺外部检索频率上升。** 见 Background 的混淆说明。 +- **不检查 URL 可达性。** 与 `hooks/release-check` 的 `$schema` 断言同一理由：联网检查会在 DNS 抖动时变红，却仍在远端搬家时是错的。 +- **不改 Phase 2 的可选性。** `reference/` 仍然 optional；它记录的是研究成果，`## External Facts` 记录的是判断与出处，两者不合并。 + +## Risks / Deferred Items + +- **`no` 分支可能被当成橡皮图章。** 一句 `- Why not: 这是仓库内部重构` 就能过。这是本任务接受的代价：字段保证「这个问题必须被显式回答」，不保证答案诚实 —— 而散文连前者都不保证。若将来发现 `no` 的比例接近 100% 且理由雷同，再考虑加反向抽查。 +- **`TF-20260919-90797c` 的对照测量仍然悬空。** 它测的是「委派 subagent 是否省 token」，与外部检索是两条独立线索，2026-10-31 用户裁定不合并。 + +## Open Questions + +- 无。三个实现层面的岔口（产出形态、落点、对照）已于 2026-10-31 由用户裁定。 diff --git a/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/version.md b/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/version.md new file mode 100644 index 0000000..74211fa --- /dev/null +++ b/TaskFlowDocs/achieved/2026-10-31-external-facts/old/v1/version.md @@ -0,0 +1,28 @@ +# Archived Version v1 — Plan — 外部事实：先判断，再取证 +> Task: 2026-10-31-external-facts +> Version: v1 +> Status: superseded +> Archived: 2026-11-01 20:05 +0800 +> Superseded by: v2 +> Archive mode: file + +## Change Summary + +v1 交付的是一个**可检查的字段**：`prd.md` 必须回答「本任务的结论是否依赖仓库外的事实」，`approve` 读它并拒绝四种坏形态。它在 2026-11-01 实现完毕，CI 全绿（PR #108）。 + +用户随后指出：**要解决的不是「没有记录」，而是「模型在当前框架下倾向于不搜索」。** 这两件事之间隔着一步 —— v1 的字段记录判断，但不改变行为；一个不搜索的模型写一句 `no` 加个像样的理由就通过了，而且看起来完全合规。v1 的 PRD 自己在 Risks 里写了这一点（「`no` 分支可能被当成橡皮图章」），但把它当成可接受的代价，而不是本任务要解决的问题。 + +v2 因此改变了承诺： + +- **加一个真实的停顿点。** 现有框架里没有一个时刻会问「这件事的答案在仓库外吗」—— `inspect` 八项全在仓库内，`summarize-state` 注入的全是内部结论，`prompt-reminder` 每回合只说任务状态。v2 在 Phase 1、Phase 3 收敛前和 Plan 定稿前各插一个问句。 +- **让检索变成可见事件。** v1 明确把 `WebSearch`/`WebFetch` 排除在 `capability-evidence` 之外（沿用 R12）。v2 把它们接进来，但**能力隔离**：检索记录与能力记录分开存放、分别消费，`TaskFlow` 的门禁仍只认 `Skill`/`Agent`/`mcp__*`。 +- **无外部证据则拦。** 这是 v1 没有的强度。一个任务若声明依赖外部事实却拿不出检索记录，`approve` 拒绝。 + +被 v1 接受、v2 不再接受的东西：一个空口无凭的 `no`。 + +## Archived Materials +- prd.md +- plan.md + +## Restore +Read the archived v1 core docs from a new temporary restore root. Do not overwrite the current task root. diff --git a/TaskFlowDocs/achieved/2026-10-31-external-facts/plan.md b/TaskFlowDocs/achieved/2026-10-31-external-facts/plan.md new file mode 100644 index 0000000..bebbcaa --- /dev/null +++ b/TaskFlowDocs/achieved/2026-10-31-external-facts/plan.md @@ -0,0 +1,153 @@ +# Plan — 外部事实：先判断，再取证 +> Task version: v2 +> Status: completed + +No spec required — small, self-contained task. + +## Reference Pointers + +- `hooks/capability-evidence:44-52` — 谓词边界；新 hook 要复用它的定位逻辑但**不共享它的取值范围**。 +- `hooks/hooks.json` — `PostToolUse` 现有两条接线，新的一条加在其后。 +- `hooks/task:1396-1460` — v1 写的 `## External Facts` 校验；R3/R4 在 `yes` 分支上扩展。 +- `hooks/smoke-test:1877` — approve 夹具；隔离断言 A3 加在这里。 +- `skills/taskflow/SKILL.md:54`、`:204`、`:217`、`:246` — 三处停顿点的落点。 +- `skills/taskflow/references/runtime.md:44-51` — 事件表，新行加在此。 + +## Related Tasks + +- 来源：`TaskFlowDocs/todo.md` 的 `TF-20261031-61b973`（v1 `TF-20261027-e8c6dc` 重述后重定界）。 +- 上游测量：`TF-20260919-90797c`（不合并，见 PRD Out of Scope）。 +- 形态参照：`achieved/2026-10-29-midturn-presence/`（`PostToolUse` 能不能向模型说话，那次已测）。 +- **v1 已实现的部分**（PR #108，未合并）：`## External Facts` 字段、`approve` 的四类拒绝、`promote` 骨架、9 条 smoke 断言。v2 在其上扩展，不重做。 + +## Skills / Tools Used + +- [PRD] `agent-skills:spec-driven-development` — purpose: 把「记录判断」重写成「矫正倾向」这个新承诺，并划清与 R12 的边界；outcome: succeeded; incorporated: 三处停顿点 + 两套存储的隔离判据（A3 的反向证明），以及「无外部证据则拦」只在 `yes` 分支触发的剩余漏洞写明而非掩盖。 +- [PRD] `get-api-docs` — purpose: 查 `PostToolUse` 是否为 `WebSearch`/`WebFetch` 投递事件，这是 R2 能否成立的前提；outcome: succeeded; incorporated: 官方文档写明匹配值同 `PreToolUse` 且明确列出这两个工具，`WebSearch.tool_input` 的字段为 `query`/`allowed_domains`/`blocked_domains`（见 PRD 的 `## External Facts`）。 +- [Plan] Unaided — no capability applied to this phase; considered: work breakdown and task decomposition +- [Test] `Unaided — no capability applied to this phase; considered: testing strategy` — 断言形态沿用既有夹具，未引入外部能力。 + +## Preconditions + +- [x] 已读 `CONTRIBUTING.md`、`CODE_STYLE.md`、`artifacts.md`。 +- [x] 隔离：工作树 `.worktrees/external-facts`，分支 `feat/external-facts`，v1 的提交 `96044df` 已在其中，未合并（PR #108 保持 open）。 +- [x] **宿主事实已查官方文档**（2026-11-01，非记忆）：`PostToolUse` 匹配值同 `PreToolUse`，后者明确列出 `WebFetch`/`WebSearch`；`WebSearch.tool_input` 有 `query`/`allowed_domains`/`blocked_domains`。 +- [x] 已确认 `hooks/summarize-state` 与 `prompt-reminder` 的现有输出形状，三处停顿点落点明确。 + +## Approval + +- Status: approved +- Approved by: user +- Approved at: 2026-11-01 20:09 +0800 +- Approved version: v2 +- Approved scope: PRD / Plan + +## Steps + +### Step 1 — 外部证据存储与记录器 + +- Goal: 让 `WebSearch`/`WebFetch` 成为可观察事件，且与能力证据**在代码上就不共用一条路径**。 +- Dependencies: 无。 +- Files: `hooks/external-evidence`（新建）、`hooks/hooks.json`、`hooks/README.md`（钩子清单，仓库规则要求每个 hook 在两处出现）。 +- Implementation checklist: + - [x] 新建 `hooks/external-evidence`，extensionless bash，读 `hook_event_name tool_name cwd tool_input.query tool_input.url`。 + - [x] 谓词只有 `WebSearch|WebFetch` 两个值 —— **不 import、不 fall through 到 `capability-evidence` 的任何分支**。 + - [x] 写到 `/taskflow/external`，行格式 `search|` 或 `fetch|`，字段里的 `|`/换行按 `capability-evidence:83` 同样方式清洗。 + - [x] 在 `hooks/smoke-test` 之外的位置也复述存储路径：`task approve` 读它，所以路径常量必须一致（写进注释并让 Step 2 引用）。 + - [x] `hooks.json` 加 `PostToolUse` matcher `WebSearch|WebFetch`。 + - [x] `hooks/README.md` 的清单加一行。 +- Acceptance: PRD 的 A1、A2。 +- Verification: 喂两个真实形状的事件 JSON，检查 `external` 文件内容；喂 `Bash` 事件，确认不写。 +- Rollback: 删新文件 + 去掉接线两行。 +- Status: done + +### Step 2 — `approve` 消费外部证据，且证明它不能顶替能力记录 + +- Goal: `yes` 的 `- Source:` 必须有检索记录背书；同时 A3 反向证明隔离成立。 +- Dependencies: Step 1（存储与字段格式）。 +- Files: `hooks/task`、`hooks/smoke-test`。 +- Implementation checklist: + - [x] 在 v1 的 `yes` 分支里，逐条 `- Source:` 去 `taskflow/external` 找匹配记录（域名包含，或 query 包含该 URL 的 host）。找到记为有据；找不到收集起来。 + - [x] 有任何一条没找到 → `fail`，消息点名是哪一条，并说明「声明依赖外部事实，但本任务版本没有任何检索记录」。 + - [x] `no` 分支：理由必须含一个仓库内可指认的东西（反引号路径，或 `path:line` 形态）——判据写死成可检查的形状，不用自然语言判断。 + - [x] **A3 的反向断言先写**：一个只有 `external` 记录、`released` 为空的夹具，`approve` 必须仍以 `stage [PRD] was never released` 拒绝。这条在 Step 1 之后就能写，先让它红着，Step 2 完成后确认它仍红（证明隔离），然后固化。 + - [x] 记录读取失败（文件不存在、行损坏）一律按「没有记录」处理并拒绝，不 fail-open。 +- Acceptance: PRD 的 A3、A4、A5、A6。 +- Verification: 四类夹具各跑一次；A3 在 Step 1 前后各跑一次，两次都必须拒绝且理由不同。 +- Rollback: 单段代码，`git revert`。 +- Status: done + +### Step 3 — 三处停顿点与文档 + +- Goal: 让「答案在仓库外吗」这个问题在它可能被推翻的三个时刻各出现一次，并把存储写进文档。 +- Dependencies: Step 1、Step 2。 +- Files: `skills/taskflow/SKILL.md`、`skills/taskflow/references/artifacts.md`、`skills/taskflow/references/runtime.md`、`CHANGELOG.md`、`hooks/smoke-test`。 +- Implementation checklist: + - [x] Phase 1 收敛段（`SKILL.md:217` 附近）加一句可执行的问句，并说明答 `yes` 时要去检索、检索会被记录。 + - [x] Phase 3 决定 Spec 之前加同一问句的复查形态（「Phase 1 答过的那次判断，在定义契约时还成立吗」）。 + - [x] Plan 定稿前加第三次复查，与 `## Approval` 相邻。 + - [x] **`SKILL.md:54` 的 `inspect` 八项不动。** + - [x] `artifacts.md` 补 `taskflow/external` 的路径、行格式、它与 `evidence` 的区别（一句话：一个释放 stage，一个证明查过）。 + - [x] `runtime.md` 事件表加一行：`External-evidence capture (never releases a stage)`。 + - [x] `CHANGELOG.md` 在 v1 那条之外补 v2 的说明，**明确写v1 的局限**（记录判断但不改变行为）。 + - [x] smoke 断言：三处问句存在于 SKILL.md（各一条具名断言），`inspect` 八项仍完整。 +- Acceptance: PRD 的 A7、A8。 +- Verification: `bash hooks/smoke-test` → `ALL SMOKE PASSED`；变异破坏每一处问句与每一条 approve 分支，各自红在对应用例。 +- Rollback: 文档改动 `git revert`。 +- Status: done + +## Checkpoints + +- Step 1 后：两个真实形状的事件各写一行，`Bash` 事件不写；`external` 与 `evidence` 是两个文件。 +- Step 2 后：A3 三次都拒（Step 1 前、Step 1 后、Step 2 后），A4/A6 各拒，A5 过。 +- Step 3 后：`ALL SMOKE PASSED`；变异各自红在具名用例上。 + +## Verification / Review + +### 实测（2026-11-01，v2） + +| 检查 | 结果 | +|---|---| +| `bash hooks/smoke-test` | `ALL SMOKE PASSED`（新增 external capture、隔离、yes/no 背书、三处停顿点共约 20 条断言） | +| `bash hooks/release-check .` | `STATUS: pass` | +| `bash -n` 全部改动脚本 | 通过 | +| `git diff --check` | clean | +| `hooks/task` / `skills/taskflow/SKILL.md` 变异后恢复 | 逐字节（`diff -q`） | + +### 变异验证 + +| 变异 | 红在哪 | +|---|---| +| `approve` 不查 `- Source:` 是否有检索记录 | `FAIL the unbacked source was not named` | +| 有未背书来源时不 fail | `FAIL the unbacked source was not named` | +| `no` 不检查理由是否指向仓库内 | `FAIL external facts message for no reason names nothing` | +| `inspect` 枚举加第 8 项（真正的第 54 行） | `FAIL the inspect enumeration changed size` | +| `inspect` 枚举加项（错改到 frontmatter 第 3 行） | 未变红 —— **变异打错位置**，断言本身是对的 | + +### 实现过程中被抓到的三个缺陷 + +**一、`|` 会伪造字段边界。** 第一版只 `tr -d '\r\n'`，`a|b` 记成 `search|a|b` —— 一个记录里多了一个分隔符。断言比对整行而不只是行数才抓到。第二版对整个 record 跑 `tr`，把分隔符本身也换成了空格，每条记录变成 `search claude code hooks`。第三版改成只清洗分隔符之后的值。 + +**二、A3 夹具搞坏了共享 store。** 隔离断言第一次写在共享 store 上，而前面的用例已经把它推到 `released` 4 / `evidence` 0，`approve` 因 store 不一致拒绝 —— 理由不是我要测的那个。改用独立 worktree 后才测到真正的理由。 + +**三、`inspect` 断言的变异打错了位置。** `tests, configuration` 在 `SKILL.md` 出现两次（frontmatter 与第 54 行），我用字符串替换改中了前者，smoke 通过，我一度以为断言没在测东西。真正改第 54 行后它立刻变红。**教训：变异没变红时，先确认变异落到了要测的那一处。** + +### 一个仍未消除的漏洞 + +`no` 是绕得过这一整套的：R4 的拦截只在 `yes` 分支触发。理由必须指向仓库内某处，但那只是把撒谎的成本抬高一点。要彻底堵死就得要求每个任务都产出外部证据，而那会让大多数内部任务造假。本任务选择写明它，而不是用一条让所有任务说谎的规则去堵（PRD Risks 第 1 条）。 + +## Change Log + +- 2026-11-01 — v1 归档至 `old/v1/`。用户重述问题为「模型倾向于不搜索」，v1 的字段记录判断但不改变行为。v2 改变承诺：加三处停顿点、接 `WebSearch`/`WebFetch` 为可见事件（与能力证据隔离）、`yes` 无检索记录则拦。 + +## Follow-ups + +- **落地后核对 `PostToolUse` 对 `WebSearch` 的真实投递形态。** 本版本依据官方文档的匹配说明，未在真实会话中观测过。第一次真实使用时看 `taskflow/external` 是否有行；没有则说明匹配值不同，需要按实际返回值修正（写进 `runtime.md` 的「documented, not yet measured」一栏）。 +- **`no` 仍是可被铁了心的模型利用的漏洞。** 见 PRD Risks 第 1 条。若将来发现 `no` 占比接近 100% 且理由雷同，再考虑更强的手段。 +- **外部证据存储无限增长**，同 `taskflow/evidence`，先不处理。 +- **`TF-20260919-90797c` 的对照测量仍然悬空**，本任务按用户 2026-10-31 裁定不合并。 + +## Version History + +- v1 — 交付可检查的 `## External Facts` 字段；已实现（PR #108），未合并。 +- v2 — 用户 2026-11-01 重述：要解决的是倾向，不是记录。加停顿点、接检索事件、无证据则拦。 diff --git a/TaskFlowDocs/achieved/2026-10-31-external-facts/prd.md b/TaskFlowDocs/achieved/2026-10-31-external-facts/prd.md new file mode 100644 index 0000000..cf0b4fc --- /dev/null +++ b/TaskFlowDocs/achieved/2026-10-31-external-facts/prd.md @@ -0,0 +1,95 @@ +# 外部事实：先判断，再取证 +> Task version: v2 +> Status: completed + +## Goal + +让 TaskFlow 在「这件事的答案在仓库外」时真的去看，而不是把判断记录成 `no` 就过去。v1 造了一个字段，它记录判断但不改变行为 —— 一个不搜索的模型写一句 `no` 加个像样的理由就能通过。v2 要解决的是产生那句 `no` 的倾向本身。 + +## Background / Confirmed Facts + +**用户 2026-11-01 重述问题**：「不是说只要求 prd 必须包含外部事实，而是说在当前的 taskflow 框架下模型会倾向于不搜索解决问题，因此可能需要一些机制来矫正这个问题。」 + +**倾向是被框架塑造的，这一点可以指到具体位置。** 每一处都在把注意力向内拉： + +| 位置 | 现状 | +|---|---| +| `skills/taskflow/SKILL.md:54` | `inspect` 枚举八项 —— repository、`index.md`、project rules、active and achieved tasks、tests、configuration、Git state、uncommitted changes —— **没有一项指向仓库外** | +| `hooks/summarize-state` | 会话开场注入的全是内部结论与内部路径 | +| `hooks/prompt-reminder` | 每回合只说「当前任务是什么、这条指令算不算变更」，不说「这件事的答案在哪」 | +| `skills/taskflow/SKILL.md:172` | `searching external sources` 只挂在 Phase 2 的能力列，而 `SKILL.md:44`/`219` 写明 `reference/` 是 optional —— 因果被写成「因为要写笔记所以去搜」 | +| `hooks/capability-evidence:48-50` | 只认 `Skill`/`Agent|Task`/`mcp__*`；`WebSearch`/`WebFetch` **完全不可见** | + +结果是「答案在仓库外」这个判断**没有任何一个时刻被触发**。加字段不会触发它。 + +**但能力是有的，而且有实证。** `achieved/2026-09-08-runtime-hooks/plan.md` 的 `## Reference Pointers` 写着： + +``` +- Claude Code hooks reference (docs.anthropic.com) and Codex Hooks guide + (learn.chatgpt.com/docs/hooks, plus openai/codex repo schemas) — verified + event names, config locations, and output contract. +``` + +那次模型**确实去查了**。缺的不是能力，是让这件事可检查、可追溯：没有 URL，没有日期，`learn.chatgpt.com/docs/hooks` 今天还成不成立无从判断。 + +**测量（2026-10-31，94 个 achieved 任务）**：`## Reference Pointers` 共 **382 条 bullet，含 URL 的 0 条**；有 `reference/` 的 **3/94**；整个归档第三方 URL 共 **4 条**，全在一个任务里；其余 71 条全是 `github.com/hkwuks/TaskFlow` 自身。 + +**这个数字不能证明 TaskFlow 抑制了搜索** —— 本仓库多数任务是内在工作，本来就不该引用外部。它证明的是另一件事：**这个框架里没有任何地方能区分「查过了」和「没想到要查」**。v2 修的是这个可区分性。 + +**宿主事实（2026-11-01 查 Claude Code hooks 官方文档，非记忆）**：`PostToolUse` 的匹配取值为「Matches on tool name, same values as PreToolUse」，而 `PreToolUse` 的匹配范围明确列出 `WebFetch` 与 `WebSearch`。`WebSearch` 的 `tool_input` 字段为 `query`、`allowed_domains`、`blocked_domains`。因此接上这两个工具是可行的，不是假设。 + +## Requirements + +- **R1 — 三个停顿点。** 在三个收敛动作之前各问一次「这一条依赖仓库外的事实吗」：Phase 1 收敛成 `prd.md` 之前、Phase 3 决定 Spec 之前、Plan 定稿之前。判断在 Phase 1 定，但**问题要在它可能被推翻的每一处重问** —— 一个 Phase 1 答 `no` 的任务，在 Phase 3 才可能第一次意识到自己在定义一个新的接口契约。 +- **R2 — 检索变成可见事件，且与能力完全隔离。** 新增一个记录器把 `WebSearch`/`WebFetch` 记成外部证据。**它与 `taskflow/evidence` 是两套存储、两条消费路径**：`capability-gate` 与 `task approve` 的 stage 对账继续只认 `Skill`/`Agent`/`mcp__*`，检索记录**永远不能**释放 stage、**永远不能**满足 `[PRD]`/`[Plan]`/`[Test]`（R12 不变）。隔离的判据是可检查的：一个只有检索记录的任务，`approve` 必须仍因缺 stage 被拒。 +- **R3 — `## External Facts` 从「记录判断」升级为「记录判断 + 证据」。** `yes` 的每条 `- Source:` 必须能在外部证据存储里找到对应的检索记录（同一任务版本、URL 域名或 query 匹配）。没有记录 = 拒绝。 +- **R4 — 无外部证据则拦。** 一个任务声明 `yes` 却拿不出检索记录时 `approve` 拒绝（R3 的推论）。一个任务声明 `no` 时不拦，但 `no` 的理由现在要面对一个问题：「你能指出这个结论在仓库内的哪一处被判定吗」—— 答不出的 `no` 就是 v1 的那个橡皮图章。 +- **R5 — 不引入联网检查。** 记录器只读宿主已经投递的事件，不发起任何网络请求。可达性仍然不校验（与 `$schema` 断言同一理由）。 +- **R6 — 不改变 Phase 2 的 optional 性质。** `reference/` 仍是可选产物；`## External Facts` 与外部证据记录都不是「必须写笔记」。 + +## Acceptance Criteria + +- **A1** `WebSearch` 的 `PostToolUse` 事件被记为一条外部证据，记录含 `query` 与该事件的 cwd 所指向的任务。 +- **A2** `WebFetch` 的同一路径可记，`tool_input.url` 被保留。 +- **A3 — 隔离的反向证明。** 一个任务只有外部证据记录、没有任何能力调用记录时，`approve` 仍以「stage [PRD] was never released」拒绝。这条**必须**失败，才能证明检索记录不能顶替能力记录。 +- **A4** 一个任务声明 `yes`，写了 `- Source:`，但外部证据存储里没有对应记录 → `approve` 拒绝并说出缺的是哪一条。 +- **A5** 同上，但有对应记录 → `approve` 通过。 +- **A6** 声明 `no` 且理由指向仓库内某处 → `approve` 通过；理由为空或只有一句泛泛而谈 → 拒绝。 +- **A7** `SKILL.md` 的三处停顿点各有一句可执行的问句，且 `SKILL.md:54` 的 `inspect` 八项**不变**（加一项会让判断伪装成检查动作）。 +- **A8** 全部断言在 `hooks/smoke-test` 里，每条都能被变异打红。 + +## External Facts + +- Depends on external facts: yes +- Source: https://docs.claude.com/en/docs/claude-code/hooks +- What it settled: R2 的前提 —— `PostToolUse` 是否为 `WebSearch`/`WebFetch` 投递事件，以及 `WebSearch.tool_input` 有哪些字段。查前只知道 R12 把这两个工具排除在能力谓词外，不知道它们是否可被观察到；若不可观察，R2 整条（把检索变成可见事件）就没有落点，整个 v2 要改成纯停顿点方案。文档明写「`PostToolUse` … Matches on tool name, same values as `PreToolUse`」，而 `PreToolUse` 的匹配范围明确列出 `WebFetch`、`WebSearch`，并给出 `query`/`allowed_domains`/`blocked_domains`。R2 因此成立，方案不变。 + +## In Scope + +- `hooks/external-evidence` — 新 hook，`PostToolUse` on `WebSearch|WebFetch`，写 `/taskflow/external`。 +- `hooks/hooks.json` — 新增一条 `PostToolUse` 接线。 +- `hooks/task` — `approve` 的 R3/R4 校验；`promote` 骨架措辞更新。 +- `hooks/smoke-test` — A1–A8。 +- `skills/taskflow/SKILL.md` — 三处停顿点；Phase 1 `prd.md` 段落更新。 +- `skills/taskflow/references/artifacts.md` — 字段清单与存储说明。 +- `skills/taskflow/references/runtime.md` — 事件表新增一行。 +- `CHANGELOG.md`。 + +## Out of Scope + +- **不改 `capability-evidence` 的谓词。** 它是 R12 的执行者，v2 不碰它。外部证据走另一个 hook、另一个文件、另一条消费路径 —— 隔离靠的是不同的代码，不是同一个 case 语句里的另一个分支。 +- **不改 `SKILL.md:54` 的 `inspect` 八项枚举。** 理由同 v1。 +- **不做对照实验。** 仍然无法从数据单独分离「TaskFlow 造成的」与「这类任务本就不需要」。 +- **不检查 URL 可达性。** +- **不回填 94 个已归档任务。** +- **不为其他宿主的 `WebSearch` 接线。** Codex/Trae/CodeBuddy 的 web 工具名不同且未测；本版本只接 Claude Code，`runtime.md` 记录这个限制。 + +## Risks / Deferred Items + +- **「无外部证据则拦」只在声明 `yes` 时触发。** 一个铁了心答 `no` 的模型仍然过得去 —— 这是真实的剩余漏洞。R4 的第二半（理由必须指向仓库内的判定处）只是把橡皮图章的代价抬高一点，不是消除它。**能彻底消除它的办法是要求每个任务都产出外部证据，而那会把大多数内部任务变成造假**。本任务选择接受这个漏洞并写明，而不是用一个会让所有任务说谎的规则去堵它。 +- **外部证据存储会无限增长。** 每个 `WebSearch` 一行，一个长任务可能有几十行。与 `taskflow/evidence` 同样的问题，同样先不处理。 +- **`PostToolUse` 对 `WebSearch` 的实际投递形态尚未在真实会话中测量** —— 依据是官方文档的匹配说明，不是一次观测。落地后第一次真实使用时应当核对（写进 Follow-ups）。 + +## Open Questions + +- 无。三个岔口（矫正点、强度、#108 去留）已于 2026-11-01 由用户裁定。 diff --git a/TaskFlowDocs/todo.md b/TaskFlowDocs/todo.md index 2b16e60..ec121e2 100644 --- a/TaskFlowDocs/todo.md +++ b/TaskFlowDocs/todo.md @@ -1112,3 +1112,11 @@ has an issue number or URL. - Goal: Keep TaskFlow present between the steps of one implementation run, so a long turn cannot leave the approved Plan behind while nothing in the workflow speaks up again. - Task: `TaskFlowDocs/achieved/2026-10-29-midturn-presence/` - Next action: None — completed and archived. + +## Make a task record whether its conclusion depends on a fact from outside this re + +- ID: TF-20261031-61b973 +- Status: done +- Goal: Make a task record whether its conclusion depends on a fact from outside this repository, and require a real search when it does — so external evidence is either gathered deliberately or explicitly ruled out, never skipped by default +- Task: `TaskFlowDocs/achieved/2026-10-31-external-facts/` +- Next action: None — completed and archived. diff --git a/hooks/README.md b/hooks/README.md index b0233ca..b39ef68 100644 --- a/hooks/README.md +++ b/hooks/README.md @@ -20,6 +20,7 @@ taskflow/hooks/ ├── summarize-state # derives the state summary (shared logic) ├── repository-docs-context # syncs index metadata and derives routes ├── capability-evidence # records real capability invocations (Claude Code only) +├── external-evidence # records WebSearch/WebFetch lookups, in a store of their own (Claude Code only) ├── capability-gate # refuses a stage document's first body write (Claude Code only) ├── json-field # shared one-level JSON reader for the two hooks above ├── task-id # the one rule for what is a legal task name; used by task/archive/reopen diff --git a/hooks/external-evidence b/hooks/external-evidence new file mode 100644 index 0000000..75f5fdd --- /dev/null +++ b/hooks/external-evidence @@ -0,0 +1,114 @@ +#!/usr/bin/env bash +# external-evidence — PostToolUse hook: record that this task looked something up +# outside the repository. +# +# The problem this exists for: nothing in TaskFlow can tell "I checked" apart +# from "it did not occur to me to check". Measured 2026-10-31 over 94 archived +# tasks, `## Reference Pointers` holds 382 bullets and not one is a URL. That +# does not prove the framework suppresses search -- most work here is internal -- +# but it does mean a task that never looked and a task that looked and found +# nothing both leave the same trace: none. +# +# So this records the looking. It is deliberately NOT part of the capability +# evidence store, and the separation is the whole design: +# +# taskflow/evidence -- what released a stage. `capability-gate` and +# `task approve` read it to decide whether a stage +# document may be written. Only Skill, Agent|Task, and +# mcp__* go in it, so a gate cannot be satisfied by +# running a built-in tool. +# taskflow/external -- what this file writes. It releases nothing, it never +# will, and `capability-evidence` is not modified to +# feed it. `task approve` reads it for the opposite +# purpose: to check that a task claiming outside facts +# actually went and got some. +# +# Keeping them in one file with a `kind` column would have been fewer lines and +# would have put the two meanings back together the first time someone wrote a +# query that looked like a capability name. Two files, two predicates, two +# consumers. +# +# What this does NOT do, deliberately: +# - it never fetches anything. It reads the event the host already delivered; +# a reachability check would redden on a DNS blip and still be wrong the +# moment the remote moved. +# - it writes nothing when the event names another tool. `Bash` running curl +# is not a recorded lookup: the host does not say what the command did, and +# guessing from the command text is the keyword classifier this repository +# rejects everywhere else. +# - every error path exits 0. A hook that blocks a search because its own +# bookkeeping failed would be worse than the silence it replaces. +# +# Extensionless, like every other TaskFlow hook, so Claude Code Windows +# auto-detection never interferes. + +set -euo pipefail +LC_ALL=C +export LC_ALL + +HERE="$(cd "$(dirname "$0")" && pwd)" + +event_json="$(cat 2>/dev/null || true)" +[ -n "$event_json" ] || exit 0 + +fields="$(printf '%s\n' "$event_json" | bash "$HERE/json-field" \ + hook_event_name tool_name cwd tool_input.query tool_input.url 2>/dev/null)" || exit 0 + +event_name="$(printf '%s\n' "$fields" | sed -n '1p')" +tool_name="$(printf '%s\n' "$fields" | sed -n '2p')" +event_cwd="$(printf '%s\n' "$fields" | sed -n '3p')" +lookup_query="$(printf '%s\n' "$fields" | sed -n '4p')" +lookup_url="$(printf '%s\n' "$fields" | sed -n '5p')" + +[ "$event_name" = "PostToolUse" ] || exit 0 + +# Two tool names, both built in. This is the predicate, and it is a tool class +# of exactly the kind `capability-evidence` refuses to count -- which is why the +# two predicates live in two files rather than in one `case` with a shared +# fall-through. +case "$tool_name" in + WebSearch) record="search|$lookup_query" ;; + WebFetch) record="fetch|$lookup_url" ;; + *) exit 0 ;; +esac +[ -n "${record#*|}" ] || exit 0 + +root="${event_cwd:-$PWD}" + +git_dir="$(git -C "$root" rev-parse --absolute-git-dir 2>/dev/null)" || exit 0 +[ -n "$git_dir" ] || exit 0 +worktree="$(git -C "$root" rev-parse --show-toplevel 2>/dev/null)" || exit 0 +[ -n "$worktree" ] || exit 0 + +# No-op outside a repository running TaskFlow. The hook is installed +# machine-wide, so every unrelated project reaches this script. +docs="$worktree/TaskFlowDocs" +[ -d "$docs" ] || exit 0 +active=0 +for dir in "$docs"/*/; do + [ -d "$dir" ] || continue + name="${dir%/}" + name="${name##*/}" + case "$name" in achieved|repository-docs) continue ;; esac + if [ -f "$dir/plan.md" ]; then active=1; break; fi +done +[ "$active" = 1 ] || exit 0 + +# `|` would forge a field boundary and a newline would forge a line, so both are +# replaced with a space inside the value -- never in the separator, which is why +# this cleans the two halves rather than the joined string. The assertion in the +# smoke suite compares the whole line rather than the count, which is what caught +# both earlier versions: `tr -d '\r\n'` left the pipe in place so `a|b` recorded +# with a forged separator, and running `tr` over the whole record ate the +# separator itself so every line came out space-separated. +# The cap keeps one pathological query from bloating a file every approve reads. +ee_kind="${record%%|*}" +ee_value="${record#*|}" +ee_value="$(printf '%s' "$ee_value" | tr '|\r\n' ' ' | cut -c1-400)" +record="$ee_kind|$ee_value" +[ -n "$ee_value" ] || exit 0 + +store="$git_dir/taskflow" +mkdir -p "$store" 2>/dev/null || exit 0 +printf '%s\n' "$record" >> "$store/external" 2>/dev/null || exit 0 +exit 0 diff --git a/hooks/hooks.json b/hooks/hooks.json index 572650f..ab0687c 100644 --- a/hooks/hooks.json +++ b/hooks/hooks.json @@ -56,6 +56,16 @@ "shell": "bash" } ] + }, + { + "matcher": "WebSearch|WebFetch", + "hooks": [ + { + "type": "command", + "command": "\"${CLAUDE_PLUGIN_ROOT}/hooks/run-hook.cmd\" external-evidence", + "shell": "bash" + } + ] } ] } diff --git a/hooks/smoke-test b/hooks/smoke-test index ec46fe6..bb55181 100755 --- a/hooks/smoke-test +++ b/hooks/smoke-test @@ -1910,6 +1910,22 @@ bash "$HERE/task" intake "Approve smoke goal." "local fixture" --root "$ap" >/de ap_id=$(awk '/^- ID: TF-/{print $3; exit}' "$ap/TaskFlowDocs/todo.md") bash "$HERE/task" promote "$ap_id" 2026-09-09-approve-smoke small --root "$ap" >/dev/null ap_task="$ap/TaskFlowDocs/2026-09-09-approve-smoke" +# Every case below is about stage reconciliation, so the PRD's `## External +# Facts` answer is filled in once, here, rather than repeated in each fixture. +# `promote` leaves it unanswered on purpose -- an unanswered skeleton must be +# refused -- and its own section below covers that. +set_external_facts() { # set_external_facts + local prd="$1"; shift + TASKFLOW_EXT_BODY="$1" awk ' + BEGIN { body = ENVIRON["TASKFLOW_EXT_BODY"]; done = 0 } + /^## External Facts$/ { print; printf "%s\n", body; done = 1; next } + done && /^## / { done = 0 } + done { next } + { print } + ' "$prd" > "$prd.tmp" && mv "$prd.tmp" "$prd" +} +set_external_facts "$ap_task/prd.md" '- Depends on external facts: no +- Why not: the fixture settles everything in `hooks/task`.' # Missing Approval section: refuse, leave bytes alone. ap_bad="$ap/TaskFlowDocs/2026-09-09-approve-bad" mkdir -p "$ap_bad" @@ -2090,6 +2106,8 @@ ap_id2=$(awk '/^- ID: TF-/{id=$3} END{print id}' "$ap_large_root/TaskFlowDocs/to bash "$HERE/task" promote "$ap_id2" 2026-09-09-approve-large small --root "$ap_large_root" >/dev/null ap_large="$ap_large_root/TaskFlowDocs/2026-09-09-approve-large" printf '# Spec\n> Task version: v1\n> Status: planning\n' > "$ap_large/spec.md" +set_external_facts "$ap_large/prd.md" '- Depends on external facts: no +- Why not: the fixture settles everything in `hooks/task`.' # The large task is its own worktree, so its records are its own: one task per # store is the production shape, and it is what keeps this fixture from being # satisfied by the small task's releases. @@ -2158,6 +2176,127 @@ printf '%s\n' "$approval_fields" | grep -qx -- '- Approved scope: PRD / Spec / P || { echo "FAIL approve scope with spec"; printf '%s\n' "$approval_fields"; exit 1; } # approve must not rewrite > Status: on the docs grep -q '^> Status: in_progress$' "$ap_task/plan.md" || { echo "FAIL approve changed plan doc status"; exit 1; } +# `## External Facts` is a second refusable answer on the same document, and it +# is checked before approval is written for the same reason the stage lines are: +# a refusal that has already touched `## Approval` leaves the Plan saying it was +# approved. Each of the seven refusals has its own message, so each case reads +# the message and not only the exit code -- seven branches that all said +# "invalid" would be one branch with seven names. +ef_probe="$ap/TaskFlowDocs/2026-09-09-approve-extfacts" +mkdir -p "$ef_probe" +printf '# Plan\n> Task version: v1\n> Status: planning\n## Approval\n- Status: requested\n## Skills / Tools Used\n- [PRD] x\n- [Plan] x\n- [Test] x\n' > "$ef_probe/plan.md" +ef_case() { # ef_case + local name="$1" want="$2" body="$3" + { printf '# PRD\n> Task version: v1\n> Status: planning\n' + [ -n "$body" ] && printf '%s\n' "$body"; } > "$ef_probe/prd.md" + if bash "$HERE/task" approve 2026-09-09-approve-extfacts --root "$ap" >/dev/null 2>"$ap/ef.err"; then + echo "FAIL external facts accepted: $name"; exit 1 + fi + grep -qF -- "$want" "$ap/ef.err" \ + || { echo "FAIL external facts message for $name"; cat "$ap/ef.err"; exit 1; } +} +ef_case "missing section" "has no '## External Facts' answer" '' +ef_case "empty section" "has no '## External Facts' answer" '## External Facts' +ef_case "not no-or-yes" "'Depends on external facts: maybe' is not an answer" '## External Facts +- Depends on external facts: maybe' +ef_case "no without a reason" "needs a '- Why not:' line" '## External Facts +- Depends on external facts: no' +ef_case "yes without a source" "needs at least one '- Source:' line" '## External Facts +- Depends on external facts: yes' +ef_case "source is not a URL" "is not an absolute http(s) URL: hooks/task" '## External Facts +- Depends on external facts: yes +- Source: hooks/task +- What it settled: x' +ef_case "source settled nothing" "Each source has to say which conclusion it changed" '## External Facts +- Depends on external facts: yes +- Source: https://example.invalid/a' +# A `no` with its reason and a `yes` with one recorded source are both clean. +# They are checked by where the refusal moves to, not by exit code: the probe +# has no stage releases, so a clean answer is refused one layer later. +for ef_ok in '## External Facts +- Depends on external facts: no +- Why not: every conclusion is a path in this repository' '## External Facts +- Depends on external facts: yes +- Source: http://intranet.example.invalid/doc +- What it settled: the retention default'; do + printf '# PRD\n> Task version: v1\n> Status: planning\n%s\n' "$ef_ok" > "$ef_probe/prd.md" + if bash "$HERE/task" approve 2026-09-09-approve-extfacts --root "$ap" >/dev/null 2>"$ap/ef.err"; then + echo "FAIL external facts: a clean answer was accepted without stage releases"; exit 1 + fi + grep -q 'External Facts' "$ap/ef.err" \ + && { echo "FAIL external facts refused a clean answer"; cat "$ap/ef.err"; exit 1; } +done +# `promote` leaves the section unanswered, and an unanswered skeleton is refused. +# This is the half that makes the field mandatory rather than merely available. +ap_new="$ap/TaskFlowDocs/2026-09-09-approve-skeleton" +bash "$HERE/task" intake "Approve skeleton goal." "local fixture" --root "$ap" >/dev/null +bash "$HERE/task" promote "$(awk '/^- ID: TF-/{id=$3} END{print id}' "$ap/TaskFlowDocs/todo.md")" \ + 2026-09-09-approve-skeleton small --root "$ap" >/dev/null +grep -qx -- '- Depends on external facts:' "$ap_new/prd.md" \ + || { echo "FAIL promote did not write an unanswered External Facts skeleton"; sed -n '/^## External Facts/,/^## /p' "$ap_new/prd.md"; exit 1; } +set_skills_body "$ap_new/plan.md" \ + '- [PRD] Unaided — no capability applied to this phase; considered: requirements elicitation' \ + '- [Plan] Unaided — no capability applied to this phase; considered: work breakdown' \ + '- [Test] Unaided — no capability applied to this phase; considered: testing strategy' +if bash "$HERE/task" approve 2026-09-09-approve-skeleton --root "$ap" >/dev/null 2>"$ap/ef.err"; then + echo "FAIL an unanswered External Facts skeleton was approved"; exit 1 +fi +grep -q "has no '## External Facts' answer" "$ap/ef.err" \ + || { echo "FAIL skeleton refusal message"; cat "$ap/ef.err"; exit 1; } +# External lookups cannot release a stage, and this is the assertion that says so. +# A task with two real lookups and no capability record must still be refused, and +# must be refused for the stage reason -- not because `approve` noticed the +# external store at all. If a later change ever teaches the gate to count a lookup +# this keeps failing until the two stores are separated again. +ap_ex="$ap/TaskFlowDocs/2026-09-09-approve-external" +mkdir -p "$ap_ex" +printf '# Plan\n> Task version: v1\n> Status: planning\n## Approval\n- Status: requested\n## Skills / Tools Used\n- [PRD] x\n- [Plan] x\n- [Test] x\n' > "$ap_ex/plan.md" +printf '# PRD\n> Task version: v1\n> Status: planning\n## External Facts\n- Depends on external facts: yes\n- Source: https://docs.claude.com/en/docs/claude-code/hooks\n- What it settled: the hook event names\n' > "$ap_ex/prd.md" +# Its own store, not the shared one the cases above have been mutating: an empty +# `evidence` beside a `released` holding three sequences is an out-of-step store, +# and `approve` would refuse for that reason instead of the one under test. +ap_ex_root="$(mktemp -d)" +cp -a "$ap/TaskFlowDocs" "$ap_ex_root/TaskFlowDocs" +ap_ex_git="$(git -C "$ap_ex_root" init -q && git -C "$ap_ex_root" rev-parse --absolute-git-dir)/taskflow" +mkdir -p "$ap_ex_git" +printf 'search|claude code hooks\nfetch|https://docs.claude.com/en/docs/claude-code/hooks\n' > "$ap_ex_git/external" +: > "$ap_ex_git/evidence" +: > "$ap_ex_git/released" +# Capability store intact but holding nothing for this task: the isolation claim is +# that the lookups above do not stand in for an invocation. +if bash "$HERE/task" approve 2026-09-09-approve-external --root "$ap_ex_root" >/dev/null 2>"$ap/ex.err"; then + echo "FAIL an external lookup released a capability stage"; cat "$ap/ex.err"; exit 1 +fi +grep -q 'stage \[PRD\] was never released by the pre-write gate' "$ap/ex.err" \ + || { echo "FAIL external refusal was for the wrong reason"; cat "$ap/ex.err"; exit 1; } +rm -rf "$ap_ex_root" + +# A `yes` needs a lookup behind it. Two stores, two claims: the first `yes` names +# a host nothing recorded and must be refused naming that host; the second is +# backed by a real `external` line and must get past this check. +ef_case "no reason names nothing" "names nothing a reader can open" '## External Facts +- Depends on external facts: no +- Why not: this is an internal change' + +ap_yes="$ap/TaskFlowDocs/2026-09-09-approve-yes" +mkdir -p "$ap_yes" +printf '# Plan\n> Task version: v1\n> Status: planning\n## Approval\n- Status: requested\n## Skills / Tools Used\n- [PRD] x\n- [Plan] x\n- [Test] x\n' > "$ap_yes/plan.md" +printf 'fetch|https://docs.claude.com/en/docs/claude-code/hooks\n' > "$ap_store/external" +printf '# PRD\n> Task version: v1\n> Status: planning\n## External Facts\n- Depends on external facts: yes\n- Source: https://example.invalid/never-fetched\n- What it settled: something\n' > "$ap/TaskFlowDocs/2026-09-09-approve-yes/prd.md" +if bash "$HERE/task" approve 2026-09-09-approve-yes --root "$ap" >/dev/null 2>"$ap/yes.err"; then + echo "FAIL a source with no recorded lookup was accepted"; cat "$ap/yes.err"; exit 1 +fi +grep -q 'no lookup backs: https://example.invalid/never-fetched' "$ap/yes.err" \ + || { echo "FAIL the unbacked source was not named"; cat "$ap/yes.err"; exit 1; } +# Backed: the same claim with a host the `external` store does carry gets past the +# lookup check (and is refused one layer later, for the stage releases it has none of). +printf '# PRD\n> Task version: v1\n> Status: planning\n## External Facts\n- Depends on external facts: yes\n- Source: https://docs.claude.com/en/docs/claude-code/hooks\n- What it settled: something\n' > "$ap/TaskFlowDocs/2026-09-09-approve-yes/prd.md" +if bash "$HERE/task" approve 2026-09-09-approve-yes --root "$ap" >/dev/null 2>"$ap/yes.err"; then + echo "FAIL a backed source was accepted with no stage releases"; exit 1 +fi +grep -q 'no lookup backs' "$ap/yes.err" \ + && { echo "FAIL a backed source was refused for an unbacked lookup"; cat "$ap/yes.err"; exit 1; } + # The stage gate's wording is pinned behaviourally above: every branch # ("missing stage [PRD]", "was never released by the pre-write gate", the two # reconciliation mismatches) was just read back out of a real rejection. The three @@ -2587,6 +2726,58 @@ grep -q -F -- 'TaskFlow is entered before the work' "$repo/hooks/capability-gate || { echo "FAIL the gate denial does not state the order"; exit 1; } echo ok +echo "== the archive lands in the task's own pull request, not after the merge ==" +# The ordering rule is prose, so what can be pinned is that the prose says the +# ordering -- three documents state it, and the failure it prevents (an archive +# with no branch to live on) is silent until someone tries to land it. +for arc_doc in "$repo/CONTRIBUTING.md" "$skill/SKILL.md"; do + grep -q 'archiving after the merge\|Archiving after the merge' "$arc_doc" \ + || { echo "FAIL $arc_doc does not warn against archiving after the merge"; exit 1; } +done +grep -q 'The archive is not an exception' "$repo/CONTRIBUTING.md" \ + || { echo "FAIL CONTRIBUTING does not say the archive rides in the same pull request"; exit 1; } +grep -q 'Run it before the merge, not after' "$skill/SKILL.md" \ + || { echo "FAIL the Skill does not state when complete runs"; exit 1; } +# The confusion this rule exists to end: `complete` prints a teardown line that +# waits for the merge, and reading that as "archive waits for the merge" is what +# produced a second pull request carrying only bookkeeping. Both documents have to +# separate the two, or the next reader makes the same inference. +grep -q 'about the \*\*worktree\*\*' "$skill/SKILL.md" \ + || { echo "FAIL the Skill does not separate teardown from archiving"; exit 1; } +echo ok + +echo "== the external-facts question is asked where it can change, and nowhere it cannot ==" +# The field alone does not fix the bias it was added for: every source this file's +# earlier sections point an Agent at is inside the repository, so nothing prompted +# the question in the first place. These assertions pin the three places the +# question is asked -- and pin that the `inspect` enumeration was NOT extended, +# because adding a judgement question to a list of "open the repo and look" items +# would dress a decision up as an inspection. +ef_skill="$skill/SKILL.md" +for ef_where in 'Before this phase converges, ask where the answer comes from' \ + 'Re-ask the external-facts question here, because this is where it can change' \ + 'Before requesting approval, read `## External Facts` one last time against the Plan'; do + grep -q -F -- "$ef_where" "$ef_skill" \ + || { echo "FAIL the external-facts pause point is missing: $ef_where"; exit 1; } +done +# The enumeration is counted rather than quoted: a ninth entry is the change this +# task decided against, and a silent reword is the other way it could drift. Seven +# items follow the sentence's own "inspect the repository, " -- that lead is the +# eighth source, so the count below is 8 only when the lead is included. The `, and ` +# before the last item is the sentence's, not a separate entry. +ef_inspect="$(sed -n 's/^After the request passes the applicability gate, first record or update its Todo item\. Then inspect the repository, \(.*\)\. Separate findings.*/\1/p' "$ef_skill")" +[ -n "$ef_inspect" ] || { echo "FAIL the inspect enumeration is gone or was reworded"; exit 1; } +ef_count="$(printf '%s' "$ef_inspect" | sed 's/, and /, /' | tr ',' '\n' | sed 's/^ *//; s/ *$//' | grep -c .)" +[ "$ef_count" -eq 7 ] \ + || { echo "FAIL the inspect enumeration changed size (expected 7 listed plus the sentence's own 'repository'): $ef_inspect"; exit 1; } +# `artifacts.md` and `runtime.md` have to say where a lookup is recorded and that +# it releases nothing, or the next reader re-derives the two stores from scratch. +grep -q 'taskflow/external' "$skill/references/artifacts.md" \ + || { echo "FAIL artifacts.md does not name the external store"; exit 1; } +grep -q 'external-evidence' "$skill/references/runtime.md" \ + || { echo "FAIL runtime.md does not name the external-evidence hook"; exit 1; } +echo ok + echo "== artifact language rule is documented without contradicting the parsers ==" # The rule splits one document into prose and machine-read structure. If it ever # names a structural line as translatable, the hooks stop finding it, so pin both @@ -2869,6 +3060,63 @@ printf '{not-json}' | bash "$ce" || { echo "FAIL capability-evidence failed on b || { echo "FAIL the cwd fallback did not record"; exit 1; } echo ok +echo "== external capture records lookups, in a store of its own ==" +# The point of this section is the separation, not the recording: a lookup has to +# be visible without ever becoming something that can release a stage. So every +# assertion here reads `external`, and the last one reads `evidence` to prove it +# was never written. +ee="$HERE/external-evidence" +ee_root="$tmp/external-evidence" +mkdir -p "$ee_root/active/TaskFlowDocs/2026-11-01-lookup" \ + "$ee_root/plain" \ + "$ee_root/achieved/TaskFlowDocs/achieved/2026-01-01-old" +for ee_repo in active plain achieved; do git -C "$ee_root/$ee_repo" init -q; done +printf 'x\n' > "$ee_root/active/TaskFlowDocs/2026-11-01-lookup/plan.md" +printf 'x\n' > "$ee_root/achieved/TaskFlowDocs/achieved/2026-01-01-old/plan.md" +ee_git="$(git -C "$ee_root/active" rev-parse --absolute-git-dir)/taskflow" +ee_store="$ee_git/external" +ee_ev="$ee_git/evidence" +ee_fire() { # ee_fire + printf '%s' "$2" | bash "$ee" || { echo "FAIL external-evidence exited nonzero"; exit 1; } +} +ee_lines() { # ee_lines + [ -f "$1" ] && wc -l < "$1" | tr -d ' ' || printf '0' +} +ee_fire active "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"WebSearch\",\"cwd\":\"$ee_root/active\",\"tool_input\":{\"query\":\"claude code hooks reference\"}}" +ee_fire active "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"WebFetch\",\"cwd\":\"$ee_root/active\",\"tool_input\":{\"url\":\"https://docs.claude.com/en/docs/claude-code/hooks\"}}" +[ "$(ee_lines "$ee_store")" -eq 2 ] || { echo "FAIL expected 2 external records, got $(ee_lines "$ee_store")"; exit 1; } +[ "$(sed -n '1p' "$ee_store")" = "search|claude code hooks reference" ] \ + || { echo "FAIL WebSearch record shape"; cat "$ee_store"; exit 1; } +[ "$(sed -n '2p' "$ee_store")" = "fetch|https://docs.claude.com/en/docs/claude-code/hooks" ] \ + || { echo "FAIL WebFetch record shape"; cat "$ee_store"; exit 1; } +# A built-in that is not one of the two, a non-PostToolUse event, and a tool name +# hidden inside another field's text. The last one is why `json-field` parses +# rather than pattern-matches: a regex would read `WebSearch` out of prose. +ee_fire active "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"Bash\",\"cwd\":\"$ee_root/active\",\"tool_input\":{\"command\":\"curl https://example.invalid\"}}" +ee_fire active "{\"hook_event_name\":\"PreToolUse\",\"tool_name\":\"WebSearch\",\"cwd\":\"$ee_root/active\",\"tool_input\":{\"query\":\"x\"}}" +ee_fire active "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"Bash\",\"cwd\":\"$ee_root/active\",\"tool_input\":{\"command\":\"echo \\\"tool_name\\\":\\\"WebSearch\\\"\"}}" +[ "$(ee_lines "$ee_store")" -eq 2 ] || { echo "FAIL a non-web tool added an external record"; cat "$ee_store"; exit 1; } +# The separation in one assertion: after two real lookups, the capability store +# this hook must never touch still does not exist. +[ ! -e "$ee_ev" ] || { echo "FAIL external-evidence wrote the capability store"; cat "$ee_ev"; exit 1; } +# No-op outside a repository running TaskFlow, same as capability-evidence. +ee_fire plain "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"WebSearch\",\"cwd\":\"$ee_root/plain\",\"tool_input\":{\"query\":\"x\"}}" +[ ! -e "$(git -C "$ee_root/plain" rev-parse --absolute-git-dir)/taskflow/external" ] \ + || { echo "FAIL a non-TaskFlow repository got an external store"; exit 1; } +ee_fire achieved "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"WebSearch\",\"cwd\":\"$ee_root/achieved\",\"tool_input\":{\"query\":\"x\"}}" +[ ! -e "$(git -C "$ee_root/achieved" rev-parse --absolute-git-dir)/taskflow/external" ] \ + || { echo "FAIL an achieved-only repository got an external store"; exit 1; } +# A lookup with no query/url records nothing rather than a bare `search|`. +ee_fire active "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"WebSearch\",\"cwd\":\"$ee_root/active\",\"tool_input\":{}}" +[ "$(ee_lines "$ee_store")" -eq 2 ] || { echo "FAIL an empty WebSearch added a record"; cat "$ee_store"; exit 1; } +# `|` and newlines cannot reach the record, or one lookup would forge two lines. +ee_fire active "{\"hook_event_name\":\"PostToolUse\",\"tool_name\":\"WebSearch\",\"cwd\":\"$ee_root/active\",\"tool_input\":{\"query\":\"a|b\"}}" +[ "$(ee_lines "$ee_store")" -eq 3 ] || { echo "FAIL a pipe split one lookup into two records"; cat "$ee_store"; exit 1; } +grep -q '^search|a b$' "$ee_store" || { echo "FAIL the pipe was not stripped"; tail -1 "$ee_store"; exit 1; } +# Bad input is a silent no-op, never a failed tool call. +printf '{not-json}' | bash "$ee" || { echo "FAIL external-evidence failed on bad JSON"; exit 1; } +echo ok + echo "== the pre-write gate releases a stage only against real evidence ==" cg="$HERE/capability-gate" cg_root="$tmp/capability-gate" @@ -3245,6 +3493,11 @@ xt_approve() { # xt_approve -> "|" } printf '# Plan\n> Task version: v1\n> Status: planning\n' > "$xt_first/plan.md" printf '# Plan\n> Task version: v1\n> Status: planning\n' > "$xt_second/plan.md" +# Both PRDs answer `## External Facts` the same way, because what this section +# measures is which task's release row `approve` reads, not the PRD's content. +for xt_prd in "$xt_first/prd.md" "$xt_second/prd.md"; do + printf '# PRD\n> Task version: v1\n> Status: planning\n## External Facts\n- Depends on external facts: no\n- Why not: everything is settled in `hooks/task`.\n' > "$xt_prd" +done printf 'invoke|a:prd\ninvoke|b:prd\n' > "$xt_store/evidence" cg_verdict "$xt_first/prd.md" allow cg_verdict "$xt_second/prd.md" allow @@ -3477,7 +3730,7 @@ echo "== approve refuses a store it cannot reconcile stages against ==" # `approve OK (user, v1)`; deleting that one row made the same document fail. # Whether a task could be approved therefore depended on an unsupported release. rc_plan="$gds/repo/$rc_task/plan.md" -printf '# PRD\n> Task version: v1\n> Status: planning\n' > "$gds/repo/$rc_task/prd.md" +printf '# PRD\n> Task version: v1\n> Status: planning\n## External Facts\n- Depends on external facts: no\n- Why not: everything is settled in `hooks/task`.\n' > "$gds/repo/$rc_task/prd.md" cat > "$rc_plan" <<'RC_PLAN' # Plan > Task version: v1 @@ -3668,7 +3921,7 @@ ug="$tmp/unaided-gate" mkdir -p "$ug/TaskFlowDocs/2026-09-27-declared" git -C "$ug" init -q printf '# Plan\n\n> Task version: v1\n> Status: planning\n\n## Approval\n\n- Status: requested\n- Approved by: pending\n- Approved at: pending\n- Approved version: pending\n- Approved scope: pending\n\n## Skills / Tools Used\n\n- [PRD] Unaided — no capability applied to this phase; considered: requirements elicitation and framing\n- [Plan] Unaided — no capability applied to this phase; considered: work breakdown and task decomposition\n- [Test] Unaided — no capability applied to this phase; considered: testing strategy\n\n## Steps\n\n### Step 1 — work\n\n- Status: pending\n' > "$ug/TaskFlowDocs/2026-09-27-declared/plan.md" -printf '# PRD\n\n> Task version: v1\n> Status: planning\n' > "$ug/TaskFlowDocs/2026-09-27-declared/prd.md" +printf '# PRD\n\n> Task version: v1\n> Status: planning\n\n## External Facts\n\n- Depends on external facts: no\n- Why not: everything is settled in `hooks/task`.\n' > "$ug/TaskFlowDocs/2026-09-27-declared/prd.md" printf '# Todo\n\n## Items\n\n## Declared\n\n- ID: TF-20260927-aaaaaa\n- Status: promoted\n- Priority: normal\n- Owner: Claude\n- Source: smoke fixture\n- Added: 2026-09-27\n- Updated: 2026-09-27\n- Goal: declared fixture\n- Task: `TaskFlowDocs/2026-09-27-declared/`\n- Next action: Approve.\n' > "$ug/TaskFlowDocs/todo.md" ug_store="$(git -C "$ug" rev-parse --absolute-git-dir)/taskflow" ug_name=2026-09-27-declared @@ -3730,7 +3983,7 @@ uc="$tmp/unaided-conflict" mkdir -p "$uc/TaskFlowDocs/2026-09-27-conflict" git -C "$uc" init -q printf '# Plan\n\n> Task version: v1\n> Status: planning\n\n## Approval\n\n- Status: requested\n- Approved by: pending\n- Approved at: pending\n- Approved version: pending\n- Approved scope: pending\n\n## Skills / Tools Used\n\n- [PRD] Unaided — no capability applied to this phase; considered: requirements elicitation and framing\n- [Plan] Unaided — no capability applied to this phase; considered: work breakdown and task decomposition\n- [Test] Unaided — no capability applied to this phase; considered: testing strategy\n\n## Steps\n\n### Step 1 — work\n\n- Status: pending\n' > "$uc/TaskFlowDocs/2026-09-27-conflict/plan.md" -printf '# PRD\n\n> Task version: v1\n> Status: planning\n' > "$uc/TaskFlowDocs/2026-09-27-conflict/prd.md" +printf '# PRD\n\n> Task version: v1\n> Status: planning\n\n## External Facts\n\n- Depends on external facts: no\n- Why not: everything is settled in `hooks/task`.\n' > "$uc/TaskFlowDocs/2026-09-27-conflict/prd.md" printf '# Todo\n\n## Items\n\n## Conflict\n\n- ID: TF-20260927-bbbbbb\n- Status: promoted\n- Priority: normal\n- Owner: Claude\n- Source: smoke fixture\n- Added: 2026-09-27\n- Updated: 2026-09-27\n- Goal: conflict fixture\n- Task: `TaskFlowDocs/2026-09-27-conflict/`\n- Next action: Approve.\n' > "$uc/TaskFlowDocs/todo.md" uc_store="$(git -C "$uc" rev-parse --absolute-git-dir)/taskflow" uc_name=2026-09-27-conflict @@ -6008,6 +6261,7 @@ tp="$tmp/test-phase" mkdir -p "$tp/TaskFlowDocs/2026-01-01-tp" git -C "$tp" init -q tp_plan="$tp/TaskFlowDocs/2026-01-01-tp/plan.md" +printf '# PRD — Test phase fixture\n> Task version: v1\n> Status: planning\n\n## External Facts\n\n- Depends on external facts: no\n- Why not: everything is settled in `hooks/task`.\n' > "$tp/TaskFlowDocs/2026-01-01-tp/prd.md" tp_store="$(git -C "$tp" rev-parse --absolute-git-dir)/taskflow" tp_name=2026-01-01-tp tp_version() { # tp_version -- rewrite the Plan's Task version diff --git a/hooks/task b/hooks/task index 6af1a37..bac218a 100755 --- a/hooks/task +++ b/hooks/task @@ -816,6 +816,11 @@ case "$command" in ## Acceptance Criteria +## External Facts + +- Depends on external facts: +- Why not: + ## In Scope ## Out of Scope @@ -1388,6 +1393,125 @@ case "$command" in if [ -z "$(printf '%s' "$cap_skills" | tr -d '[:space:]')" ]; then fail "approve blocked: Skills / Tools Used is missing or empty" fi + # `## External Facts` is the one place a task says whether its conclusion + # depends on a fact from outside this repository, and names what it found if + # so. It exists because prose could not be checked: measured 2026-10-31 over + # 94 archived tasks, `## Reference Pointers` holds 382 bullets and not one + # is a URL, and 3 of 94 tasks have a `reference/` directory -- while nothing + # in that measurement can say whether any of them should have searched or + # none of them needed to, because most work here is internal. So the field + # does not demand a search: it demands that the question was answered once, + # in a form a reader can check. `no` must carry its reason for the same + # reason -- an unexplained `no` is the default the field exists to displace. + # + # Only `prd.md` carries it. Whether evidence is gathered at all is a + # requirements question; the Plan's job is how, not whether. + ext_msg="approve blocked: prd.md has no '## External Facts' answer, so it cannot be seen whether this task's conclusions rest on anything outside the repository. Add the section with one of: + - Depends on external facts: no + - Why not: +or + - Depends on external facts: yes + - Source: + - What it settled: " + ext_body="$(awk ' + /^## External Facts$/ { p = 1; next } + p && /^## / { exit } + p { print } + ' "$prd" 2>/dev/null)" || ext_body="" + # No separate "is the section there" test: an absent section and an empty + # one reach the `""` arm below with the same message, which is what they + # are -- both say nothing. A line that checked for absence first could only + # ever fire on a document the arm below would refuse anyway. + ext_answer="$(printf '%s\n' "$ext_body" | sed -n 's/^- Depends on external facts:[[:space:]]*\(..*\)[[:space:]]*$/\1/p' | head -1)" + case "$ext_answer" in + no) + printf '%s\n' "$ext_body" | grep -qE '^- Why not:[[:space:]]*[^[:space:]]' \ + || fail "approve blocked: 'Depends on external facts: no' needs a '- Why not:' line. An unexplained no is the default this field exists to displace. Say what puts the conclusion inside this repository." + # A reason is prose, so the only checkable part is whether it points at + # something a reader can open. A backticked path or a `path:line` is that + # pointer; "this is an internal refactor" is not, and it is exactly the + # sentence an unexamined `no` produces. This does not verify the path + # exists -- a Plan may predate a move, and a false MISSING is worse than a + # loose reason -- it only requires that the reason names one. + ext_why="$(printf '%s\n' "$ext_body" | sed -n 's/^- Why not:[[:space:]]*//p' | head -1)" + printf '%s\n' "$ext_why" | grep -qE '`[^`]+`|[A-Za-z0-9_./-]+\.[A-Za-z]+:[0-9]+' \ + || fail "approve blocked: the '- Why not:' reason names nothing a reader can open. Point at where the conclusion is settled inside this repository — a backticked path such as \`hooks/task\` or \`skills/taskflow/SKILL.md:54\`, or a path with a line number. 'It is an internal change' is the sentence an unexamined no produces, and this field exists to stop that sentence being the default." + ;; + yes) + # One pass over the section, because the three defects have three + # different remedies and a reader who is told "invalid" has to guess. + ext_bad="$(printf '%s\n' "$ext_body" | awk ' + { line = $0; sub(/\r$/, "", line) } + /^- Source:/ { + src++ + v = line; sub(/^- Source:[[:space:]]*/, "", v); sub(/[[:space:]]+$/, "", v) + if (v !~ /^https?:\/\/[^[:space:]]+$/) badurl = (badurl == "" ? v : badurl) + next + } + /^- What it settled:/ { + v = line; sub(/^- What it settled:[[:space:]]*/, "", v); sub(/[[:space:]]+$/, "", v) + if (v != "") settled++ + next + } + END { + if (src == 0) { print "none"; exit } + if (badurl != "") { print "url\t" badurl; exit } + if (settled < src) { print "settled\t" src "\t" settled; exit } + }')" || ext_bad="" + case "$ext_bad" in + "") ;; + none) + fail "approve blocked: 'Depends on external facts: yes' needs at least one '- Source:' line. If nothing outside the repository settles a conclusion, answer no and give the reason." + ;; + url*) + fail "approve blocked: a '- Source:' is not an absolute http(s) URL: $(printf '%s' "$ext_bad" | cut -f2). A repository path, a tool name, or a sentence is not an external source." + ;; + settled*) + fail "approve blocked: $(printf '%s' "$ext_bad" | cut -f2) '- Source:' line(s) but $(printf '%s' "$ext_bad" | cut -f3) '- What it settled:'. Each source has to say which conclusion it changed; a source that changed nothing did not need fetching." + ;; + esac + # Every `- Source:` has to be backed by a recorded lookup. Without this the + # section is a form a task fills in from memory, which is the failure the + # whole task exists to prevent: measured 2026-11-01, `achieved/` holds a + # Plan whose `## Reference Pointers` names Anthropic's hooks reference and + # Codex's hooks guide in prose, with no URL and no date, and nothing in the + # repository can tell whether either was ever opened. + # + # The store is `taskflow/external`, NOT `taskflow/evidence`. `external` is + # written by `hooks/external-evidence` from `WebSearch`/`WebFetch`, the two + # built-in tools `capability-evidence` deliberately refuses to count, so a + # lookup can never release a stage. This check reads it for the opposite + # purpose and reads nothing else from it. + ext_store="$cap_git_dir/taskflow/external" + ext_unbacked="" + while IFS= read -r ext_src; do + [ -n "$ext_src" ] || continue + ext_host="$(printf '%s' "$ext_src" | sed -n 's|^https\{0,1\}://\([^/]*\).*|\1|p')" + ext_seen=0 + if [ -f "$ext_store" ] && [ -n "$ext_host" ]; then + while IFS= read -r ext_line; do + case "$ext_line" in + *"$ext_host"*|*"$ext_src"*) ext_seen=1; break ;; + esac + done < "$ext_store" + fi + [ "$ext_seen" = 1 ] || ext_unbacked="$ext_unbacked${ext_unbacked:+, }$ext_src" + done <` with a `- What it settled:` line saying which conclusion that source changed. The question is whether a conclusion could be falsified by something outside this repository — a version, a specification, someone else's documented behaviour. Code that is only in this repository, and facts only this repository can settle, belong in `## Reference Pointers` instead. Answering `yes` is a commitment to go and look, not a note that looking might be useful: the phase table's `searching external sources` is what that answer invokes, and a `yes` with no source behind it is refused. Most tasks here are internal and answer `no`; the field exists so that answer is a decision rather than a default. + Record which repository documents were inspected and which apply. If remote PR-rule discovery was attempted, record its sources, result, and incorporated conclusions without storing sensitive payloads. +**Before this phase converges, ask where the answer comes from.** A requirement earned from memory is indistinguishable, in the document, from one earned by looking — and everything this phase is told to inspect is inside the repository, so nothing here prompts the question. Ask it once, explicitly: *which of these conclusions would a reader have to check against something outside this repository* — a version, a specification, another project's documented behaviour, a price, a limit. When the answer is none, say so in `## External Facts` and point at where the reasoning is settled inside the repository; when it is not none, go and look before writing the criterion, and the lookup is recorded for you. A `yes` with no lookup behind it is refused by `task approve`, which is the whole point: the field records a decision, and the record makes the decision checkable. Do not answer `no` because the work feels internal — that is the default this question exists to displace. + Expose assumptions and turn vague requests into testable criteria using the approach that best fits the task. Do not silently decide product, compatibility, or risk questions owned by the user. **When the goal or the approach is not yet settled, diverge before you converge.** Writing requirements is a converging act: it picks one reading of the request and turns it into criteria, which quietly discards the readings not yet on the table. So when the request leaves the target or the way to reach it undetermined, or more than one workable direction exists, this phase opens with the class above — `divergent exploration` — before `prd.md` is drafted. The condition is the state of the request, so it is answerable from what you have: a request whose goal and approach are both fixed converges directly, and one that is still an idea does not. What the class produces is candidate directions and the trade-offs between them, for the user to choose from; only the chosen ones become requirements, and the rest are recorded as out-of-scope or deferred so the discarded ground is visible rather than lost. As with every class in the table, this names what to look for and never what to use — availability, fit, and the decision to proceed unaided remain yours, and an unaided PRD is a legitimate outcome recorded as one. @@ -224,6 +229,8 @@ Agent suggestions may mark evidence as candidate. Only the Primary Agent or user ### 3. Design — Spec decision +**Re-ask the external-facts question here, because this is where it can change.** Phase 1 answered it against the request; this phase answers it against the contract. Defining an interface, an error semantic, or a compatibility boundary is exactly when a "we are only changing our own code" reading stops being true — a status code another service must accept, a field a plugin manifest must declare, a version an operator must install. If Phase 1 answered `no` and this phase turns out to depend on something outside, say so rather than proceeding: the answer changes, `prd.md` changes, and any lookup made now is recorded like any other. + Classify the task before planning: - Large: multiple modules/files, architecture/API/data-flow/compatibility/error-contract changes, multiple sessions, or roughly more than 30 minutes. `spec.md` is required. @@ -256,6 +263,8 @@ A `## Change Log` in `plan.md` records work revisions — wording or approach ch Include checkpoints after meaningful groups of steps. Record risks, deviations, verification results, review findings, and unresolved follow-ups. `Skills / Tools Used` is required: stage-tagged invoke or Unaided lines as each document starts (see the capability section); do not leave the section blank. `task approve` rejects missing stages and unqualified stage Unaided lines. +**Before requesting approval, read `## External Facts` one last time against the Plan.** The Steps were not written yet when Phase 1 answered, and they are where a task most often discovers it needs to know something the repository does not contain — a tool's actual behaviour, a format another project defines, a limit that is not documented here. A `yes` must have its lookups already done and recorded, because `task approve` refuses one that does not; a `no` must still name where the work is settled inside the repository. If this pass changes the answer, change `prd.md` and note it in the change log rather than approving a document whose facts section no longer describes the work. + Record approval as: ```markdown @@ -308,6 +317,8 @@ The archive transaction below is the single definition; `artifacts.md` and `vers Run it as one command: `hooks/task complete --user-accepted`. It checks the gates, marks the documents `completed`, and hands over to `hooks/archive`, which moves the directory and closes the Todo entry. Doing the steps by hand is what the collapse looked like — the same three gates live in `complete`, and a hand edit of two `> Status:` lines followed by `hooks/archive` used to skip all of them. +**Run it before the merge, not after.** The transaction reads the working tree and never consults the base branch, so nothing about it waits for a pull request to land. The order is: finish and verify, `hooks/task complete`, then commit the code, the documents and the archive together on the task's branch, push, and open the pull request. Archiving after the merge is the mistake to avoid, because the moved directory and the rewritten Todo entry then exist on no branch — and the only ways to land them are the two `CONTRIBUTING.md` forbids: a push to the base outside any pull request, or a second pull request carrying nothing but bookkeeping. The teardown line `complete` prints is about the **worktree**: `git worktree remove` waits for the merge, and that condition is about not deleting a directory that may still hold the only copy of something. It is not a statement about when to archive. + Before that command, confirm all acceptance criteria pass, verification is recorded, the current Task version is consistent across existing core documents, unresolved items are explicit follow-ups, and no other agent is writing core documents. What the command performs: mark the task `completed`, move the entire directory, update the linked Todo item's `Task:` path to `TaskFlowDocs/achieved//`, close it, then verify the active path is absent, the achieved path exists, and the achieved root PRD and Plan both say `completed`. If any part fails, record the blocker and do not claim archival completion. **`hooks/archive` refuses an unfinished task on its own.** It requires the Plan's `## Approval` to say `approved` for its current `> Task version:` with an approver other than `pending`, no unchecked `- [ ]` item, and every Step block closed — a Step is closed when it says `- Status: done`, and nothing else counts. That last one is a whitelist, and deliberately: it used to name the three words that meant unfinished, so any word it did not know (`completed`, a misspelling, an empty value, a missing line) read as finished and archived a Plan that was not. `hooks/task progress` has only ever accepted `pending|in_progress|done|blocked`, so `done` is the one word an author can write that means closed, and the two now read the same rule from both sides. The three gates are the same ones `complete` applies, applied again where the move happens, because `archive` is also runnable by hand. All of them are checked before the directory moves, so a refusal leaves the task active, the Todo untouched, and `achieved/` without it. diff --git a/skills/taskflow/references/artifacts.md b/skills/taskflow/references/artifacts.md index 80f8374..b58ba4a 100644 --- a/skills/taskflow/references/artifacts.md +++ b/skills/taskflow/references/artifacts.md @@ -22,7 +22,8 @@ The rule binds the document, not whoever drafts it: prose that arrives from a ca Hooks parse task documents by exact text, so the lines they match stay English verbatim: -- Section headings a hook matches by name: `## Approval`, `## Skills / Tools Used`, `## Verification / Review`, `## Change Log`, `## Items`, `## Removed`, `## Active / Resumable`, `## Closed / Reference Only`, `## Reference Pointers` +- Section headings a hook matches by name: `## Approval`, `## Skills / Tools Used`, `## Verification / Review`, `## Change Log`, `## Items`, `## Removed`, `## Active / Resumable`, `## Closed / Reference Only`, `## Reference Pointers`, `## External Facts` +- The two answers inside `## External Facts`, which `hooks/task approve` reads at column 0: `- Depends on external facts: no` with a following `- Why not: `, or `- Depends on external facts: yes` with at least one `- Source: ` and one `- What it settled: ` per source. The field names and the two words `no` / `yes` are read literally; the reasons and conclusions are prose and follow the user's language. Two further checks ride on them: a `no`'s reason must contain a backticked path or a `path:line`, so the reason points at where the conclusion is settled rather than describing it in the abstract; and a `yes`'s every `- Source:` must be backed by a matching line in `/taskflow/external`, which `hooks/external-evidence` writes from `WebSearch`/`WebFetch`. That store and `taskflow/evidence` are separate on purpose — `evidence` releases a stage and only `Skill`/`Agent`/`mcp__*` may enter it, while `external` releases nothing and exists so a lookup can be told apart from a lookup that never happened. - The reinforce bullet inside `## Reference Pointers`: `- [Reinforce] \`\` — `, matched at column 0. The marker, the backticks and the `—` separator are text a hook reads; translating or re-indenting any of them turns the declaration off without an error. The reason after the separator is prose and follows the user's language. - The `## Todo Record` heading: not matched by any hook, but it is written by `hooks/archive` and must stay at column 0 in `plan.md` — the same rule that keeps it out of a Todo entry, where a column-0 `## ` line is a section boundary to every hook that splits the file. - Entry headings a hook matches by name: `### Step N — `, `### SN — ` @@ -61,6 +62,7 @@ TaskFlowDocs/ ## Background / Confirmed Facts ## Requirements ## Acceptance Criteria +## External Facts ## In Scope ## Out of Scope ## Risks / Deferred Items diff --git a/skills/taskflow/references/runtime.md b/skills/taskflow/references/runtime.md index 3dda637..bf1ecc4 100644 --- a/skills/taskflow/references/runtime.md +++ b/skills/taskflow/references/runtime.md @@ -49,9 +49,12 @@ An environment running any other agent host needs no hooks: it continues with th | Optional permission guardrail (never approves core writes) | `PermissionRequest` | `PermissionRequest` | `PreToolUse` | `tools/pre-execute` | `PreToolUse` | | Pre-write capability gate on a stage document (deny only) | `PreToolUse` (`Write`, `Edit`) → `hookSpecificOutput.permissionDecision: deny` | `PreToolUse` (`apply_patch`) → `hookSpecificOutput.permissionDecision: deny`, path read out of the patch text — documented, not yet measured on a real Codex | `PreToolUse` (`Write|Edit`) → `hookSpecificOutput.permissionDecision: deny` | not wired — `write`/`edit` are lowercase and the case arms are not | `PreToolUse` (`Write|Edit`) → `hookSpecificOutput.permissionDecision: deny` | | Capability-call capture (feeds the gate) | `PostToolUse` (`Skill`, `Agent|Task`, `mcp__.*`) | not wired | `PostToolUse` (`Skill`, `Agent|Task`, `mcp__.*`) | not wired — `skill` is lowercase too | `PostToolUse` (`Skill`, `mcp__.*`) — Trae has no `Agent`/`Task` tool | +| External-lookup capture (never releases a stage) | `PostToolUse` (`WebSearch`, `WebFetch`) | not wired | not wired | not wired — no equivalent web tool wired | not wired | Host event names and output fields are version-sensitive; check each host's current hooks reference when wiring a new release. The gate's contract was measured against **Claude Code 2.1.282** on 2026-09-27 rather than assumed: `Skill` fires both `PreToolUse` and `PostToolUse`, its `tool_input.skill` carries the capability name, and `PreToolUse` fires for `Edit`. That measurement covers `Skill` only — that `Agent`/`Task` and `mcp__.*` reach `PostToolUse` the same way is inferred from the same tool-call lifecycle, not measured. The Trae column is documented, not measured — see **Trae specifics** below. The CodeBuddy columns come from the packaged CLI rather than from a running session, and the Codex and dsh columns from Codex's own hook documentation (read 2026-10-29) and measured payloads respectively — **CodeBuddy specifics**, **Codex specifics** and **dsh specifics** below each state which standing they hold. +**External-lookup capture is documented, not yet measured.** Claude Code's hooks reference (read 2026-11-01) states that `PostToolUse` matches on tool name with the same values as `PreToolUse`, and that `PreToolUse` matches any tool name except `EndConversation`, naming `WebFetch` and `WebSearch` among the built-ins; the same reference gives `WebSearch`'s `tool_input` as `query`, `allowed_domains`, `blocked_domains`. `hooks/external-evidence` reads `tool_input.query` for a search and `tool_input.url` for a fetch. That the events arrive in that shape has not been observed in a running session — the first real task to record a lookup is where it gets checked, and this paragraph moves to *measured* when one has. + The gate is wired wherever the host delivers **one of the two input shapes `capability-gate` now reads**: the written file's path under `tool_input.file_path` (Claude Code, CodeBuddy, Trae), or the patch text under `tool_input.command` with `tool_name` of `apply_patch` (Codex). A host that reports an edit some other way must not be wired until the gate learns that shape — the hook would find no input, exit 0, and allow every write, while the repository and this document both claimed a gate was present. A silent no-op gate is worse than an absent one, which is why a host is wired only against a shape the script can actually read. The same applies to a host whose tool names miss the matcher arm: the matcher never fires, so the script is not even reached. dsh is that case, and **dsh specifics** below states it. The recorder's predicate is the tool class (`Skill`, `Agent`/`Task`, `mcp__*`), so it travels to any host that reports those names. Trae has no `Agent`/`Task` tool, so that one evidence path exists only where the host does. @@ -342,6 +345,7 @@ repo-root/ ├── merge-todo # the driver: merges todo.md by entry ├── repository-docs-context # syncs index metadata + derives routes ├── capability-evidence # records real invocations in the worktree store + ├── external-evidence # records WebSearch/WebFetch lookups in a separate store ├── capability-gate # refuses a stage document's first body write ├── json-field # shared one-level JSON reader ├── task-id # the one rule for a legal task name