Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
d8eda45
feat(repo): dokimi_config.v1 schema and repo_config loader
napetrov Oct 31, 2026
ac28b88
fix(repo): normalise shared globs, refuse ancestor suites, escape ski…
napetrov Oct 31, 2026
4a64bda
fix(repo): a file another suite claims still moves its own skill
napetrov Oct 31, 2026
5d70511
Merge branch 'main' of https://github.com/intel/dokimi into pa-323-fe…
napetrov Oct 31, 2026
911d072
Address review: a symlink root digests as its link
napetrov Oct 31, 2026
43e0998
Address review: manifest brackets, drive paths, malformed globs
napetrov Oct 31, 2026
d2a5cea
Keep a symlinked skill directory's content in skill_sha
napetrov Oct 31, 2026
72eb3bf
fix(repo): nested skills, evals-only suites, store false, budget numbers
napetrov Oct 31, 2026
bf015b4
fix(repo): exact budget hint; budget text that is no number is a conf…
napetrov Oct 31, 2026
81b24fc
fix(repo): a nan or inf budget is refused; symlinked evals hash their…
napetrov Oct 31, 2026
1a07894
fix(repo): follow symlinked eval directories; refuse an evals dir tha…
napetrov Oct 31, 2026
2b2c04f
fix(repo): a suite claims whole directories of its skill, not single …
napetrov Oct 31, 2026
c8d18fd
fix(repo): a suite symlink also counts by its target text
napetrov Oct 31, 2026
031b7ff
fix(repo): refuse a link into the skill, an id clash, and a huge budg…
napetrov Oct 31, 2026
479e791
fix(repo): literal config paths, link-aware impact, case-sensitive globs
napetrov Nov 1, 2026
b4504b0
Thread: PRRT_kwDOVBg7g86yVRTx: already fixed in 72eb3bf0
napetrov Nov 1, 2026
c80ce22
fix(config): shared config at its link and target, suite directories,…
napetrov Nov 1, 2026
1208af2
Merge remote-tracking branch 'intel/main' into pr323-land1031
napetrov Nov 1, 2026
5a23ae0
fix(config): claim a suite after its directory is deleted, a link on …
napetrov Nov 1, 2026
1cb3efe
fix(config): portable skill_sha and claims through chained links
napetrov Nov 1, 2026
8f23573
fix(config): keep ".." after links, pin digest order and mode lookup
napetrov Nov 1, 2026
f4d6cc7
Keep .. after a link in an absolute link target when finding hops
antonwolfy Nov 1, 2026
b109fc3
Claim links on the way to a wildcard suite that matches nothing
antonwolfy Nov 1, 2026
90422e1
Order directory children by name in the suite walk
antonwolfy Nov 1, 2026
809282b
Keep a trailing slash and a root self-link in link_text
antonwolfy Nov 1, 2026
94f43ac
fix(config): name suites by their resolved spelling, claim wildcard p…
napetrov Nov 1, 2026
8c71859
Merge intel/main into feat/dokimi-config
napetrov Nov 1, 2026
54ffa03
Let the Windows extended-path digest test run where symlinks work
antonwolfy Nov 1, 2026
48c78a8
Merge remote-tracking branch 'intel/main' into pr323-land1031
napetrov Nov 2, 2026
7b121d8
fix(config): address the round-8 review of the link and suite walks
napetrov Nov 2, 2026
a7fdab1
Merge remote-tracking branch 'intel/main' into pr323-land1031
napetrov Nov 2, 2026
ef9a0b1
fix(config): keep the glob and digest walks of a suite or skill complete
napetrov Nov 2, 2026
4b18921
fix(config): a terminal ** matches directories only, not FIFOs or soc…
napetrov Nov 2, 2026
dbce8d4
Hash a manifest under the suite directory once
antonwolfy Nov 2, 2026
171da87
Skip the two git-index mode tests when git is missing
antonwolfy Nov 2, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions dokimi/changelog.d/20261030_180000_napetrov_feat-repo-config.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
<!--
Copyright (C) 2026 Intel Corporation
SPDX-License-Identifier: Apache-2.0
-->

### Added

- **`dokimi_config.v1`, the repository config schema and loader (library only, no CLI yet)** — a
repository's `.github/dokimi/config.yaml` (or `.dokimi/config.yaml`) declares its subjects (skills, and
the repository itself with `kind: repo`), where each subject's evals are (beside the skill by default,
or a central glob such as `evaluation/harbor/tasks/onetbb-*`), the plan defaults, `shared` paths for
the smoke set, and the budget and baseline store. The loader `dokimi.repo_config` finds the file in
the order explicit path, `.github/dokimi/config.yaml`, `.dokimi/config.yaml` under the repository root,
validates it against the schema, and reports each subject's `skill_sha` (the tree digest of the skill
directory without its evals and without any nested skill directory, which is its own subject) and
`suite_hash`. A skill's suite claims whole directories of its skill (its evals, or a directory such as
`tests/**`), and those directories leave `skill_sha`. A suite that picks an individual file of its skill
(`SKILL.md`, a script, a reference, one eval file) is refused at load, naming the subject and the file. A link in a suite that resolves into its skill is refused too. An explicit subject whose id is taken by a default skill at another directory is refused instead of silently replacing it. `baseline.store: false`
reads as `off`, and a budget number that YAML reads as text (`1e3`) is refused with the plain
decimal or the `1.0e+3` spelling, and a `.nan` or `.inf` budget is refused. A suite directory is
hashed file by file, following symlinks; a link also counts by its target text, so an edit to a shared eval, or a retarget of its link, moves `suite_hash`. Nothing reads the file yet: the commands that use it come later,
and existing `.agent-eval/` lookups are unchanged.
1 change: 1 addition & 0 deletions dokimi/docs/reference/cli-interface.json
Original file line number Diff line number Diff line change
Expand Up @@ -13143,6 +13143,7 @@
"'answers'",
"'arms'",
"'cli_routes'",
"'dokimi_config'",
"'effective_instructions'",
"'eval'",
"'eval_coverage'",
Expand Down
6 changes: 4 additions & 2 deletions dokimi/dokimi/artifacts.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,9 @@
``result`` (one status per planned cell), ``repo_budget`` (what a repository puts
into an agent's context), ``effective_instructions`` (the effective-instructions/v0
receipt of a reach probe), ``need_graph`` and ``eval_coverage`` (a frozen need graph and an
eval set's coverage of it) and ``task_inspect`` (which eval-task format each
path under a tree is). This module stamps each with a
eval set's coverage of it), ``task_inspect`` (which eval-task format each
path under a tree is) and ``dokimi_config`` (a repository's subjects and where
their evals are). This module stamps each with a
``schema_version`` and validates it against the JSON Schema in
``dokimi/schemas/`` on save and load, so fixtures and baselines stay
forward-compatible and self-describing.
Expand Down Expand Up @@ -63,6 +64,7 @@
"task_inspect": "task_inspect.v1",
"redteam": "redteam.v1",
"cli_routes": "cli_routes.v1",
"dokimi_config": "dokimi_config.v1",
"need_graph": "need_graph.v1",
"eval_coverage": "eval_coverage.v1",
}
Expand Down
23 changes: 3 additions & 20 deletions dokimi/dokimi/commands/repo_run.py
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@

from dokimi.repos.harness_profile import DELEGATION_MODES
from dokimi.exit_codes import EXIT_USAGE
from dokimi.utils import tree_digest


def _context(args: argparse.Namespace, out: Path, pin: bool = False):
Expand Down Expand Up @@ -59,14 +60,14 @@ def _context(args: argparse.Namespace, out: Path, pin: bool = False):
sha = getattr(tree, "sha", None)
if sha is None: # a working tree: tag by what is built, or an edited Dockerfile reuses a stale image
ctx_dir = export_build_context(tree, manifest, out / "_eval" / "build-context")
tag = "worktree-" + context_digest(ctx_dir)[:12]
tag = "worktree-" + tree_digest(ctx_dir)[:12]
else:
tag = sha[:12]
image = f"dokimi/{slug(manifest.id).lower()}:{tag}"
elif not runner.image_exists(image):
raise ValueError(f"--image {image} does not exist")
if pin:
stamp = {"tree": tree.label, "eval_dir": context_digest(staged), "image": image}
stamp = {"tree": tree.label, "eval_dir": tree_digest(staged), "image": image}
if _uses_builtin_graders(staged, manifest): # else no score depends on them: an upgrade keeps the batch
stamp["graders"] = graders_digest()
_pin_batch(out, stamp)
Expand Down Expand Up @@ -95,24 +96,6 @@ def _context(args: argparse.Namespace, out: Path, pin: bool = False):
return ctx, tree


def context_digest(root: Path) -> str:
"""sha256 over every path, mode bit and byte of a build context (a link by its target)."""
import hashlib
import os

h = hashlib.sha256()
for path in sorted(root.rglob("*")):
rel = path.relative_to(root).as_posix()
if path.is_symlink():
h.update(f"L {rel}\0{os.readlink(path)}\0".encode())
elif path.is_file():
h.update(f"F {rel}\0{int(os.access(path, os.X_OK))}\0".encode())
h.update(hashlib.sha256(path.read_bytes()).digest())
elif path.is_dir():
h.update(f"D {rel}\0".encode())
return h.hexdigest()


def _require_grade(content: Path, manifest: RepoManifest, ids: list[str] | None) -> None:
"""Refuse the selected tasks when one needs ``entry_points.grade`` and the manifest has none."""
from dokimi.repos.manifest import ManifestError, TaskSummary, grade_problems
Expand Down
Loading
Loading