feat(repo): record public_at and guidance:rev history for leak checks

dokimi/changelog.d/20261101_120000_antonwolfy_replay-leak-record.md

@@ -0,0 +1,16 @@
+### Added
+
+- **Replay tasks and `guidance:rev:` arms record what a leak check needs** —
+  `repo tasks replay` writes `public_at` into each `task.json`: the earliest
+  `createdAt` of a gold thread's first comment, left out (with a note) when
+  a gold thread has no time with an offset. A `guidance:rev:<rev>` cell's
+  `meta.json` `arm_rev` adds `guidance_latest` (the last commit to touch each
+  guidance file at `<rev>`, or a path a guidance link resolves through),
+  `guidance_commits` (the commits that touch a guidance path outside
+  `eval_dir` or a path a link
+  resolves through, a merge when its own result changes one, newest first by committer
+  date, at most 200) and `guidance_commits_truncated`, each commit with its
+  author and committer date (a UTC date as `Z`), so a reader can see whether
+  a guidance commit came after a task's review was public. A `guidance:rev:`
+  cell whose source is a shallow clone fails its preparation: the history
+  needs the full one (`git fetch --unshallow`).

dokimi/docs/repo-evaluation-contract.md

@@ -391,7 +391,27 @@ commit and is not in the workspace, because step 3 starts a new history; a
 grader diffs the agent's changes from `start_sha`. With the arm come
 `arm_files`, the files it deleted, wrote or changed; for `guidance:rev:` an
 `arm_rev` object (`rev` as given, its `sha`, and `removed`: the base's guidance
-files with no counterpart at `<rev>`); for `patch:` an `arm_patch` object (the
+files with no counterpart at `<rev>`), which also records the guidance history
+of `<rev>` (`dokimi/repos/guidance_history.py`): `guidance_latest`, for
+each guidance file the arm wrote, the commit reachable from `<rev>` that last
+touched it, or for a link it or any path its resolution goes through (each
+intermediate link and the file it resolves to), by committer date;
+`guidance_commits`, the commits reachable from `<rev>` that touch a path the
+guidance globs match outside `eval_dir` (which the arm leaves out, as it does
+the files) or a path a guidance link resolves through, newest first by committer
+date, at most 200, each with its `subject`; and `guidance_commits_truncated`,
+`true` when that cap cut the list. Each commit has its `sha`, `authored` and
+`committed` (strict ISO 8601, `%aI` and `%cI`; a UTC date is written `Z`,
+whichever form the git that read the history prints). The history needs the whole of it: a shallow
+git dir (a local checkout cloned with `--depth`) fails the cell's preparation
+(`git fetch --unshallow`), as its history stops at the boundary commit. A merge commit counts for a
+path when its result differs there from every parent: a conflict resolution,
+an edit made in the merge, or a clean merge of its parents' edits to
+different parts of one file; a merge that takes the path as one parent has
+it does not, and the commits it merged count. A reader can compare `committed` with a replay task's
+`public_at` (§4.6) to see whether a guidance commit came after the task's
+answer was public; the author date alone cannot show it, as an amend or a
+rebase keeps it. For `patch:` an `arm_patch` object (the
 file's absolute `path` and full `sha256`, `added`: the files the patch added
 that neither the guidance globs nor `.mcp.json` match, and `mcp_servers`: the servers
 the patch added to the `.mcp.json` at the root, read before the agent runs). `args_if_present` lists the profile's
@@ -759,8 +779,14 @@ by a reviewer: it has no line and is never gold.
   `score: {metric: recall, max: 1.0}`, `review_patch`, `gold`, `window`
   (`--window`, default 5) and the prompt (`--prompt-file` replaces the default,
   which asks for the findings JSON of §4.5). For the reader it adds
-  `source_pr`, `source_url`, `reviewed_commit`, `merge_commit` and
-  `left_out_files`.
+  `source_pr`, `source_url`, `reviewed_commit`, `merge_commit`,
+  `left_out_files` and `public_at`: the earliest `createdAt` of a gold
+  thread's first comment, as the threads file gives it, the time the task's
+  answer was public. It is absent when any gold thread's `createdAt` is
+  missing, not ISO 8601 or has no offset (`Z` is one), as the earliest time
+  is then unknown; the build's notes name those threads. Against the
+  `committed` dates a `guidance:rev:` arm records (§3), it shows whether a
+  guidance commit came after it.
 - `gold.json` holds `{id, file, line, kind, comment, addressed, reply, url}`
   per thread. `id` is `pr<number>-t<n>`, `n` being the thread's position among
   all the PR's threads, so ids stay the same when a thread is dropped. `kind`

dokimi/dokimi/repos/guidance_history.py

@@ -0,0 +1,201 @@
+# Copyright (C) 2026 Intel Corporation
+# SPDX-License-Identifier: Apache-2.0
+
+"""The guidance history of a ``guidance:rev:<rev>`` arm, for its ``meta.json`` ``arm_rev`` (contract §3).
+
+A replay task's answer is public from its ``public_at`` (§4.6); a guidance commit made after it may
+encode that review. :func:`arm_rev_history` records what a reader needs to check that: the commit
+that last touched each guidance file at the revision, and the newest commits that touch a guidance
+path, each with its author and committer date. The flag reads the committer date: ``git commit
+--amend`` and a rebase keep the author date (https://github.com/intel/dokimi/issues/290). A path
+under the manifest's ``eval_dir`` is not guidance, as the arm installs none of them.
+
+The history is one ``git log -z --raw`` over the revision, filtered with :func:`~.manifest.match_glob`.
+``-z`` ends every field with a NUL, and each file the commit touched is a ``:``-led status record
+followed by its one path, so a subject or a path, whatever it holds, cannot pass for the other.
+Git cuts a subject at a NUL (``git commit`` refuses one), so a subject is one field. The output is
+split on NUL as bytes and each field then decoded as UTF-8 (``--encoding=UTF-8``; a byte that is not
+UTF-8 becomes U+FFFD, as :mod:`dokimi.process` decodes), so a ``\\r`` in a subject or a path stays one.
+A date is git's ``%aI`` / ``%cI`` with a UTC offset written ``Z``, which git 2.51 prints and older git
+(2.43) prints as ``+00:00``. The history is read once per git dir, revision, globs, ``eval_dir`` and
+links in a process, also when cells of the arm prepare at once: every cell asks the same. The read
+covers every commit reachable from the revision and every path each one touched, so its cost grows
+with the history: for 42,000 commits, about a second of ``git log`` (some 20-30 MB) and a fraction of
+one to parse it, the globs matched once per distinct path; the other cells of the arm wait. A key of the
+repository's own config that changes what the log prints (as checked with git 2.51) is overridden by
+a flag (:data:`_PINNED`). A shallow git dir is refused: its history stops at the boundary commit, so
+it would name a later commit as the last to touch a file.
+
+The arm installs a guidance link (``AGENTS.md -> docs/current -> GUIDE.md``) as the bytes of the file
+it resolves to, so a commit that touches any path of its chain, an intermediate link or that file,
+counts for the link's path, whether or not a glob matches that path.
+
+A merge counts when its own result changes a guidance path against every parent
+(``--diff-merges=dense-combined``): a conflict resolution, an edit made in the merge, or a clean
+merge that joins its parents' edits to different parts of one file. A merge that takes a file as
+one parent has it does not: the commits it merged count.
+"""
+
+from __future__ import annotations
+
+import re
+import threading
+from collections.abc import Iterator
+from dataclasses import dataclass
+from datetime import datetime
+from functools import cache, lru_cache
+from pathlib import Path
+from typing import Any
+
+from dokimi import process
+
+from .errors import PrepError
+from .manifest import _is_within, match_glob
+from .source import SourceError, is_shallow, isolated_git_env
+
+#: At most this many commits in ``guidance_commits``; ``guidance_latest`` is not capped
+GUIDANCE_COMMITS_CAP = 200
+_GIT_TIMEOUT_S = 1800.0
+_FORMAT = "--format=%H%x00%aI%x00%cI%x00%s"  # with -z, a NUL ends the subject too
+_COMMIT_ID = re.compile(r"[0-9a-f]{40}(?:[0-9a-f]{24})?")  # SHA-1 or SHA-256
+#: Each overrides a key of the repository's config that changes the output (git 2.51)
+_PINNED = (
+    "--no-show-signature",  # log.showSignature: verification text outside the fields
+    "--root",  # log.showRoot=false: the root commit with no files
+    "--encoding=UTF-8",  # i18n.logOutputEncoding: a subject re-encoded, in UTF-16 with NULs
+    "--ignore-submodules=none",  # diff.ignoreSubmodules=all: no commit that adds or moves a gitlink
+)
+
+_Links = tuple[tuple[str, tuple[str, ...]], ...]  # (link path, the paths it resolves through), sorted
+
+
+@dataclass(frozen=True)
+class _Commit:
+    sha: str
+    authored: str
+    committed: str
+    subject: str
+    paths: tuple[str, ...]  # the guidance paths it touched
+
+    def dates(self) -> dict[str, str]:
+        return {"sha": self.sha, "authored": self.authored, "committed": self.committed}
+
+
+def arm_rev_history(git_dir: Path, globs: tuple[str, ...], eval_dir: str, sha: str, files: list[str],
+                    links: dict[str, list[str]]) -> dict[str, Any]:
+    """``guidance_latest`` for each of ``files`` (the guidance at ``sha``; ``links`` maps those that
+    are links to every path they resolve through, the file they resolve to last), ``guidance_commits``
+    (newest first by committer date, at most :data:`GUIDANCE_COMMITS_CAP`) and
+    ``guidance_commits_truncated``. A path under ``eval_dir`` is not guidance: the arm installs none."""
+    commits = _guidance_commits(git_dir, sha, globs, eval_dir, tuple(sorted((k, tuple(v)) for k, v in links.items())))
+    latest: dict[str, dict[str, str]] = {}
+    for c in commits:  # newest first: the first commit naming a file is its latest
+        for path in c.paths:
+            latest.setdefault(path, c.dates())
+    return {"guidance_latest": {p: latest[p] for p in sorted(files) if p in latest},
+            "guidance_commits": [{**c.dates(), "subject": c.subject} for c in commits[:GUIDANCE_COMMITS_CAP]],
+            "guidance_commits_truncated": len(commits) > GUIDANCE_COMMITS_CAP}
+
+
+_key_locks: dict[tuple[Any, ...], threading.Lock] = {}
+_key_locks_guard = threading.Lock()
+
+
+def _guidance_commits(git_dir: Path, sha: str, globs: tuple[str, ...], eval_dir: str,
+                      links: _Links) -> tuple[_Commit, ...]:
+    """:func:`_read_guidance_commits`, one thread per key at a time: a cell that asks while another
+    cell reads the same history waits for that read and takes its answer from the cache."""
+    key = (git_dir, sha, globs, eval_dir, links)
+    with _key_locks_guard:
+        lock = _key_locks.setdefault(key, threading.Lock())
+    with lock:
+        return _read_guidance_commits(*key)
+
+
+@lru_cache(maxsize=64)
+def _read_guidance_commits(git_dir: Path, sha: str, globs: tuple[str, ...], eval_dir: str,
+                           links: _Links) -> tuple[_Commit, ...]:
+    """The commits reachable from ``sha`` that touch a path ``globs`` match outside ``eval_dir`` or a
+    path a link of ``links`` resolves through, newest first by committer date."""
+    linked: dict[str, list[str]] = {}
+    for link, chain in links:
+        for path in chain:
+            linked.setdefault(path, []).append(link)
+    _refuse_shallow(git_dir)
+    found = []
+    # once per distinct path: a path recurs in many commits, and match_glob builds its pattern on every call
+    counts_for = cache(lambda p: _counts_for(p, globs, eval_dir, linked))
+    for (commit, authored, committed, subject), touched in _parse_log(_git_log(git_dir, sha)):
+        paths = tuple(dict.fromkeys(g for p in touched for g in counts_for(p)))
+        if paths:
+            found.append(_Commit(commit, authored, committed, subject, paths))
+    # stable: commits with one committer date keep git's order
+    return tuple(sorted(found, key=lambda c: datetime.fromisoformat(c.committed), reverse=True))
+
+
+def _counts_for(path: str, globs: tuple[str, ...], eval_dir: str, linked: dict[str, list[str]]) -> list[str]:
+    """The guidance paths a change to ``path`` counts for: itself when a glob matches it outside ``eval_dir``
+    (as :func:`~.workspace._guidance_at` lists them), and each link through it (no chain reaches into
+    ``eval_dir``: :func:`~.workspace._link_chain` refuses one)."""
+    own = not _is_within(path, eval_dir) and any(match_glob(g, path) for g in globs)
+    return ([path] if own else []) + linked.get(path, [])
+
+
+def _parse_log(out: bytes) -> list[tuple[tuple[str, str, str, str], list[str]]]:
+    """Each commit of :func:`_git_log`'s output as ``((sha, authored, committed, subject), paths)``.
+    A field that starts with ``:`` (after the newline that ends a header) is a status record, and the
+    field after it is its path; a path is never read as a header. Any other non-empty field starts a
+    header of four fields; an empty one ends a merge's header, and the output. Each field is decoded
+    on its own, after the split: no ``\\r`` is turned into a newline."""
+    fields = (field.decode("utf-8", "replace") for field in out.split(b"\0"))
+    commits: list[tuple[tuple[str, str, str, str], list[str]]] = []
+    for field in fields:
+        if field.lstrip("\n").startswith(":") and commits:
+            commits[-1][1].append(next(fields, ""))
+        elif field:
+            commits.append((_header(field, fields), []))
+    return commits
+
+
+def _header(sha: str, fields: Iterator[str]) -> tuple[str, str, str, str]:
+    authored, committed, subject = next(fields, ""), next(fields, ""), next(fields, "")
+    if not _COMMIT_ID.fullmatch(sha):
+        raise PrepError(f"git log: a commit starts with {sha[:60]!r}, not a commit id")
+    return sha, _utc_as_z(authored), _utc_as_z(committed), subject
+
+
+def _utc_as_z(date: str) -> str:
+    """A strict ISO 8601 date of git's with a UTC offset as ``Z``, whichever form the git that printed it uses."""
+    return date.removesuffix("+00:00") + "Z" if date.endswith("+00:00") else date
+
+
+def _refuse_shallow(git_dir: Path) -> None:
+    try:  # in the log's own environment: a caller's GIT_DIR would answer for another repository
+        shallow = is_shallow(git_dir, _env())
+    except SourceError as exc:
+        raise PrepError(str(exc)) from exc
+    if shallow:
+        raise PrepError(f"{git_dir} is a shallow clone: the guidance history of a guidance:rev: arm needs its full "
+                        "history (git fetch --unshallow)")
+
+
+def _git_log(git_dir: Path, sha: str) -> bytes:
+    """Every commit reachable from ``sha`` with its dates, subject and the paths it touched; a merge
+    with the paths its result changes against every parent."""
+    return _git(git_dir, f"log {sha[:12]}", "log", "-z", "--raw", "--no-abbrev", "--no-renames",
+                "--diff-merges=dense-combined", *_PINNED, _FORMAT, sha, "--")
+
+
+def _git(git_dir: Path, what: str, *args: str) -> bytes:
+    """The stdout of ``git -C git_dir *args`` as bytes, with no system or global config; ``what`` names it in an error."""
+    done = process.git("-C", str(git_dir), *args, timeout=_GIT_TIMEOUT_S, env=_env())
+    if done.timed_out:
+        raise PrepError(f"git {what}: timed out after {_GIT_TIMEOUT_S:.0f} seconds")
+    if done.returncode:
+        raise PrepError(f"git {what} exited {done.returncode}: {process.git_failure(done)}")
+    return done.stdout_bytes
+
+
+def _env() -> process.Env:
+    """git's environment for the history: :func:`~.source.isolated_git_env`, as the arm's own reads have it."""
+    return process.Env.only((), isolated_git_env())

dokimi/dokimi/repos/replay/build.py

@@ -24,6 +24,7 @@
 import json
 from collections import Counter
 from dataclasses import dataclass, field
+from datetime import datetime
 from pathlib import Path
 from typing import Any
 
@@ -118,7 +119,8 @@ def _make(h: History, manifest: RepoManifest, pr: PullRequest, opts: Options,
     gold = _gold(h, manifest, pr, threads, rev, opts.window, notes)
     if not gold:
         raise _Skip("no gold thread is left")
-    return _task(pr, base, rev, leave_out, opts), gold, _mbox(pr, rev, h.patch(base, rev, leave_out))
+    task = _task(pr, base, rev, leave_out, opts, _public_at(threads, {g["id"] for g in gold}, notes))
+    return task, gold, _mbox(pr, rev, h.patch(base, rev, leave_out))
 
 
 def _needed(pr: PullRequest) -> list[str]:
@@ -204,7 +206,28 @@ def _where(t: Thread) -> str:
     return f"{t.path}:{t.line}" if t.line else (t.path or "no path")
 
 
-def _task(pr: PullRequest, base: str, rev: str, leave_out: list[str], opts: Options) -> dict[str, Any]:
+def _public_at(threads: list[Thread], gold: set[str], notes: list[str]) -> str | None:
+    """The earliest first-comment ``createdAt`` of the threads whose id is in ``gold``, as the input gives it.
+    ``None``, with a note naming them, when any of them has no usable time: missing, not ISO 8601 or with no
+    offset (GitHub's ``Z`` is one, UTC). The minimum of the others could be later than the real one."""
+    times, unusable = [], []
+    for t in (t for t in threads if t.id in gold):
+        try:
+            at = datetime.fromisoformat(t.created)
+        except ValueError:
+            at = None
+        if at is None or at.tzinfo is None:  # a time with no offset is not comparable with one that has it
+            unusable.append(t.id)
+        else:
+            times.append((at, t.created))
+    if unusable:
+        notes.append(f"no public_at: no first-comment createdAt with an offset on {', '.join(unusable)}")
+        return None
+    return min(times)[1] if times else None
+
+
+def _task(pr: PullRequest, base: str, rev: str, leave_out: list[str], opts: Options,
+          public: str | None) -> dict[str, Any]:
     task: dict[str, Any] = {
         "id": task_id(pr), "family": "review", "base": base, "grader": "dokimi:review_replay",
         "score": {"metric": "recall", "max": 1.0}, "review_patch": PATCH, "gold": GOLD, "window": opts.window,
@@ -214,6 +237,8 @@ def _task(pr: PullRequest, base: str, rev: str, leave_out: list[str], opts: Opti
         task["source_url"] = pr.url
     if leave_out:
         task["left_out_files"] = leave_out
+    if public:
+        task["public_at"] = public  # when the gold's answer was public (contract §4.6)
     return task