Repository navigation
Conversation
…gent The agent's cwd was a `dokimi-run-*` temp dir, its skill plugin sat at `/opt/abskill`, and the prompt opened with a "Solve this coding task…" wrapper. None of that is in the stock harness. The workspace is now a plain mkdtemp, the plugin is mounted at `/opt/claude-plugins`, and the prompt is the task instruction behind the treatment notice only. Skills still load through `--plugin-dir`: a skill in `$CLAUDE_CONFIG_DIR/skills` is not offered under `--setting-sources ""`, so the `~/.claude/skills` location is not used. A test runs payload_audit over the arms' argv and cwd. The docker_solver line baseline drops from 899 to 897. Signed-off-by: Nikolay Petrov <nikolay.a.petrov@intel.com>
|
✅ OpenCodeReview: Review complete: 0 finding(s) across 10 selected item(s). Review details
|
There was a problem hiding this comment.
🟡 Changes recommended
The argv audit test omits the executable path, missing a remaining dokimi marker and overstating its whole-command audit guarantee.
1 open issue
What changed in this PR
1 new finding.
This PR reduces harness-identifying text in dokimi’s Docker Claude runs, helping evaluations better reflect ordinary agent use.
Changes:
- Uses generic temporary workspace names and renames the container plugin directory.
- Removes the harness-written prompt wrapper, retaining the task instruction and optional treatment notice.
- Updates regression tests, documentation, and the growth baseline.
| File | Description |
|---|---|
dokimi/tests/test_payload_audit.py |
Adds argv and workspace marker checks. |
dokimi/tests/test_container_agent.py |
Updates plugin-path and prompt assertions. |
dokimi/tests/test_cleanup_timeouts.py |
Updates the plugin-removal timeout matcher. |
dokimi/tests/test_agent_sandbox.py |
Adjusts treatment-notice assertions. |
dokimi/scripts/growth_baseline.json |
Lowers the solver’s file-size baseline. |
dokimi/dokimi/harnesses/trigger_probe.py |
Updates the workspace-name comment. |
dokimi/dokimi/harnesses/docker_solver.py |
Renames the plugin directory and removes prompt framing. |
dokimi/dokimi/harnesses/agent_sandbox.py |
Uses default temporary directory names. |
dokimi/docs/coding-harnesses.md |
Documents the updated workspace and plugin paths. |
dokimi/changelog.d/20261101_120000_napetrov_fidelity-arm-leaks.md |
Records changes and prompt-comparability implications. |
Review effort: Balanced
| argv = h._claude_args(task, treatment, _CONTAINER_PLUGIN, "haiku", out) | ||
| assert _CONTAINER_PLUGIN in argv and "abskill" not in " ".join(argv) | ||
| assert audit_payload(argv, markers, allowed=allowed) == [] |
antonwolfy
left a comment
There was a problem hiding this comment.
Review at 2592e75. The prompt, plugin-path and workspace-name changes are right, and the updated assertions pin them. One thing is worth fixing before merge: the new audit test and the changelog claim more than they check.
What I ran, on a Linux review host at 2592e75:
make ci-local --with-docker: 35 passed, 0 failed, 7 skipped; 12002 tests passed; changed-line coverage 100%.- Mutation run on the new code with the four touched test files: restoring the
dokimi-run-prefix,/opt/abskill, the "Solve this coding task" wrapper, an addedTask:prefix or suffix, and a dropped or always-on notice are all caught. Two mutants pass: renaming the-hostarm'splugindirectory todokimi-abskill, andmkdtemp(prefix="project-")(harmless).
Checked, no issues
- Prompt:
TREATMENT_NOTICEends in a blank line, so notice plus instruction keeps its separator. Both arms are pinned withprompt == task.instruction. - Stale names: no remaining
/opt/abskill, no code that sweepsdokimi-run-*temp dirs or strips the old wrapper, and nothing readsprompt.txtin the package. - Changelog fragment:
### Changed, licence header, non-comparability stated.
A smaller point on wording: the changelog says the docker-claude arms' cwd changed. The temp tree applies to the -host variants only. The in-container arm runs with docker exec -w /app and never had a dokimi-run-* cwd.
| h = DockerSolverHarness(harness_id="docker-claude", mode="claude", skill_notice=notice) | ||
| argv = h._claude_args(task, treatment, _CONTAINER_PLUGIN, "haiku", out) | ||
| assert _CONTAINER_PLUGIN in argv and "abskill" not in " ".join(argv) | ||
| assert audit_payload(argv, markers, allowed=allowed) == [] |
There was a problem hiding this comment.
I agree with Copilot here; the repro is below.
The audit covers the arguments after the executable, not the command the agent runs, and the executable still says dokimi.
_claude_args() returns the argv after the executable. _solve_claude_in_container() runs [AGENT_BIN, *args] (docker_solver.py:448), and AGENT_BIN is /opt/dokimi-agent/claude (container_agent.py:80). Audited with the same default markers:
audit_payload(["/opt/dokimi-agent/claude", "-p", "x"]) -> [Leak(marker="dokimi", path="$[0]")]
The process's argv[0] and the /opt/dokimi-agent/ directory are visible inside the container, so the PR's claim that the arms carry no marker (description and changelog: "asserts the arms' argv and cwd carry no marker") does not hold, and the test can't say so either way.
Suggestion: either audit [AGENT_BIN, *argv] and rename the in-container directory (AGENT_BIN and FILES_DIR) to something neutral such as /opt/claude-bin, or narrow the test comment and the changelog sentence to "the arguments after the executable" and say the /opt/dokimi-agent path is still visible. Also cover the -host arm: its plugin dir is workspace.root / "plugin", and renaming it to dokimi-abskill passes the whole suite.

Summary
Skills still load through
--plugin-dir: a skill in$CLAUDE_CONFIG_DIR/skillsis not offered under the launch's
--setting-sources ""(measured, see thecomment in
docker_solver.py), so moving skills to~/.claude/skillsis not done here.Evidence
dokimi-run-/abskill/Solve this coding task.appear in the argv and cwd of the docker-claude arms.After:
tests/test_payload_audit.py::test_docker_claude_arms_send_no_harness_artifactsrunsaudit_payloadover the argv (both notice settings) and the workspace cwd with the default markers, and finds nothing. The prompt assertions intest_container_agent.pyandtest_agent_sandbox.pynow check the raw instruction.pytest -n 6).scripts/check_growth.py --base intel/main:docker_solver.py899 -> 897, baseline ratcheted, no growth.Merge Danger
Door: two-way. The prompt and paths revert with a single commit; no data or config is migrated.
Blast Radius: docker-claude and docker-claude-host arms only. Runs recorded before this change used the old prompt text and paths, so their prompt and
prompt.txtare not comparable with new runs. The changelog fragment says so.🤖 Generated with Claude Code