Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions HARNESS_STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@

A harness may have several independent interventions and PRs. A historical `AWAITING_EXTERNAL` batch state **remains historical** even when a separate publication ledger later confirms an upstream merge. All external submission/acceptance claims below come from the linked owning source; `SUBMITTED_UNVERIFIED_LIVE` means **no current PR-state claim**.

**Scope:** 153 harnesses · 167 interventions · 38 unique external PR URLs · 25 assessment batches · 31 monitored PR targets.
**Scope:** 153 harnesses · 168 interventions · 39 unique external PR URLs · 25 assessment batches · 32 monitored PR targets.

## Harness overview

Expand Down Expand Up @@ -94,7 +94,7 @@ A harness may have several independent interventions and PRs. A historical `AWAI
| [markmhendrickson/ateles](https://github.com/markmhendrickson/ateles) | 1 | 0 | 0 | 0 | 1 |
| [martian56/rook](https://github.com/martian56/rook) | 1 | 0 | 0 | 0 | 0 |
| [me7ko-dev/genesis-agent](https://github.com/me7ko-dev/genesis-agent) | 1 | 0 | 0 | 0 | 0 |
| [microsoft/fara](https://github.com/microsoft/fara) | 1 | 1 | 0 | 0 | 0 |
| [microsoft/fara](https://github.com/microsoft/fara) | 2 | 1 | 0 | 0 | 0 |
| [mila-iqia/babyai](https://github.com/mila-iqia/babyai) | 1 | 1 | 1 | 0 | 0 |
| [mnbresearch/crewdeck](https://github.com/mnbresearch/crewdeck) | 2 | 1 | 0 | 0 | 0 |
| [murilo-bassan/durable-agent-runtime](https://github.com/murilo-bassan/durable-agent-runtime) | 2 | 1 | 0 | 0 | 0 |
Expand Down Expand Up @@ -259,6 +259,7 @@ A harness may have several independent interventions and PRs. A historical `AWAI
| martian56/rook | [assessment:rook-dispatch-approval](data/assessment-candidate-batches/2026-10-22-02.json) | S1 | CLOSED_NO_CHANGE | NONE | — | assessment-upstream-2026-10-22-02 | No code change justified |
| me7ko-dev/genesis-agent | [assessment:genesis-agent-multiprocess-skill-commit](data/assessment-candidate-batches/2026-11-09-18.json) | S2 | CLOSED_NO_CHANGE | NONE | — | assessment-upstream-2026-11-09-18 | No code change justified |
| microsoft/fara | [assessment:fara-native-checkpoint-integrity](data/assessment-candidate-batches/2026-10-23-11.json) | S1 | AWAITING_EXTERNAL | NOT_SUBMITTED | — | assessment-upstream-2026-10-23-11 | Refresh source; check external publication grant |
| microsoft/fara | [monitor:microsoft/fara#96](data/pr-maintenance/targets.json) | — | NOT_TRACKED | WATCHLIST_ONLY | [#96](https://github.com/microsoft/fara/pull/96) | monitor_only | Read PR maintenance inbox; no technical inference |
| mila-iqia/babyai | [assessment:babyai-task-evaluation-input-contract](data/assessment-candidate-batches/2026-10-23-09.json) | S1 | AWAITING_EXTERNAL | SUBMITTED_UNVERIFIED_LIVE | [#128](https://github.com/mila-iqia/babyai/pull/128) | assessment-upstream-2026-10-23-09 | Check current maintainer/CI outcome |
| mnbresearch/crewdeck | [assessment:crewdeck-s2-manual-claim-admission](data/assessment-candidate-batches/2026-11-06-12.json) | S2 | AWAITING_EXTERNAL | NOT_SUBMITTED | — | assessment-upstream-2026-11-06-12 | Refresh source; check external publication grant |
| mnbresearch/crewdeck | [monitor:mnbresearch/crewdeck#1](data/pr-maintenance/targets.json) | — | NOT_TRACKED | WATCHLIST_ONLY | [#1](https://github.com/mnbresearch/crewdeck/pull/1) | monitor_only | Read PR maintenance inbox; no technical inference |
Expand Down
14 changes: 14 additions & 0 deletions data/external-pr-publication-readbacks/2026-11-11.json
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,20 @@
"publication_state": "OPEN_READY",
"approved_exact_title_body": true,
"approval_boundary": "Owner standing external PR publication mandate effective 2026-11-11; authenticated xLagerFeuer host POST and matching title/body (one trailing LF normalized), three-file head/base GitHub readback; no individual PR approval"
},
{
"id": "fara-native-checkpoint-integrity",
"repository": "microsoft/fara",
"public_fork": "opensiro-contributions/fara",
"fork_local_pr_url": "https://github.com/opensiro-contributions/fara/pull/1",
"pr_url": "https://github.com/microsoft/fara/pull/96",
"published_at": "2026-11-11T23:58:47Z",
"base_sha": "a675d6d61c41c47ae87bacefeab22caad18e3e84",
"head_sha": "5b2499026322e25ac6d6182778914171a85f56b6",
"pr_commits": 1,
"publication_state": "OPEN_READY",
"approved_exact_title_body": true,
"approval_boundary": "Standing external-PR mandate ACTIVE; authenticated xLagerFeuer dev-alpha-2 GitHub REST POST; unchanged fork-local #1 title/body; external #96 open, non-draft, two-file exact head/base/provider readback. Python 3/3 targeted native tests PASS; browser/model suite not run; no CLA/DCO-person attestation."
}
]
}
15 changes: 10 additions & 5 deletions data/pr-maintenance/targets.json
Original file line number Diff line number Diff line change
Expand Up @@ -76,11 +76,6 @@
"url": "https://github.com/cbrock84/headcount/pull/47",
"monitoring": "verify_404_only"
},
{
"repository": "chumingjun/dsh-harness-one",
"number": 173,
"url": "https://github.com/chumingjun/dsh-harness-one/pull/173"
},
{
"repository": "chankov/agent-fleet",
"number": 43,
Expand All @@ -91,6 +86,11 @@
"number": 38,
"url": "https://github.com/chekusu/wanman/pull/38"
},
{
"repository": "chumingjun/dsh-harness-one",
"number": 173,
"url": "https://github.com/chumingjun/dsh-harness-one/pull/173"
},
{
"repository": "cordfuse/politik",
"number": 33,
Expand All @@ -116,6 +116,11 @@
"number": 205,
"url": "https://github.com/lbdremy/phax/pull/205"
},
{
"repository": "microsoft/fara",
"number": 96,
"url": "https://github.com/microsoft/fara/pull/96"
},
{
"repository": "mila-iqia/babyai",
"number": 128,
Expand Down
14 changes: 13 additions & 1 deletion docs/PR_MAINTENANCE.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ This is a construction/operating contract. The read-only detector and alert tran

The watcher uses Python standard library only and authenticates using the GitHub Actions token. Each observation is a live **GET** from the GitHub repository that owns it; unexpected API errors, missing identities, unknown states and truncated pagination **fail closed**. Third-party message **bodies are deliberately not stored, executed, or trusted as instructions**: only event IDs, author, status and GitHub URLs are used. Humans/agents inspect the exact review on the external source before repairing.

Current pinned targets are **31** reviewed external PR identities for contributor `xLagerFeuer` (18 previously retained, seven 2026-11-10 publication readbacks, [KnowWork #4](https://github.com/knowflow-ai/KnowWork/pull/4), [Runloop #1](https://github.com/murilo-bassan/durable-agent-runtime/pull/1), [CodeAct #26](https://github.com/xingyaoww/code-act/pull/26), [Clouds-Coder #50](https://github.com/FonaTech/Clouds-Coder/pull/50), [yoagent #450](https://github.com/yologdev/yoagent/pull/450), and [phax #205](https://github.com/lbdremy/phax/pull/205) on 2026-11-11), including Headcount #47 (previously verified but returning GitHub 404 on 2026-11-05), closed-unmerged StarNet #56, and six earlier Corrective PRs: [Workflow One #173](https://github.com/chumingjun/dsh-harness-one/pull/173) (merged 2026-10-29), [Reef/Polyp #19](https://github.com/Coral-Bricks-AI/reef/pull/19), [Spartan #1](https://github.com/CorticalComputer/Spartan/pull/1), [BESSER #205](https://github.com/BESSER-PEARL/BESSER-Agentic-Framework/pull/205), [BabyAI #128](https://github.com/mila-iqia/babyai/pull/128), and [Phoenix #1](https://github.com/Phoenixrr2113/agent-harness/pull/1). This list is a versioned scope snapshot and is **not** dynamically widened by an unreviewed search result. Adding/removing targets requires a reviewable manifest change and explicit checkpoint-migration handling. Closed/merged PRs remain in the snapshot to preserve lifecycle evidence until a deliberate archival change.
Current pinned targets are **32** reviewed external PR identities for contributor `xLagerFeuer` (18 previously retained, seven 2026-11-10 publication readbacks, [KnowWork #4](https://github.com/knowflow-ai/KnowWork/pull/4), [Runloop #1](https://github.com/murilo-bassan/durable-agent-runtime/pull/1), [CodeAct #26](https://github.com/xingyaoww/code-act/pull/26), [Clouds-Coder #50](https://github.com/FonaTech/Clouds-Coder/pull/50), [yoagent #450](https://github.com/yologdev/yoagent/pull/450), and [phax #205](https://github.com/lbdremy/phax/pull/205) and [Fara #96](https://github.com/microsoft/fara/pull/96) on 2026-11-11), including Headcount #47 (previously verified but returning GitHub 404 on 2026-11-05), closed-unmerged StarNet #56, and six earlier Corrective PRs: [Workflow One #173](https://github.com/chumingjun/dsh-harness-one/pull/173) (merged 2026-10-29), [Reef/Polyp #19](https://github.com/Coral-Bricks-AI/reef/pull/19), [Spartan #1](https://github.com/CorticalComputer/Spartan/pull/1), [BESSER #205](https://github.com/BESSER-PEARL/BESSER-Agentic-Framework/pull/205), [BabyAI #128](https://github.com/mila-iqia/babyai/pull/128), and [Phoenix #1](https://github.com/Phoenixrr2113/agent-harness/pull/1). This list is a versioned scope snapshot and is **not** dynamically widened by an unreviewed search result. Adding/removing targets requires a reviewable manifest change and explicit checkpoint-migration handling. Closed/merged PRs remain in the snapshot to preserve lifecycle evidence until a deliberate archival change.

**Reviewed availability-only exception (Upstream #139 / PR #140):** Headcount #47 is retained as the eighteenth identity, with the sole allowlisted `monitoring: verify_404_only` manifest mode. Every run MUST GET that exact source and confirm HTTP 404. It may then report **17 fully observed / 1 explicitly UNVERIFIABLE of 18**, continue reading the other 17, and store a distinct *availability marker* instead of stale PR fields. The first transition emits one inbox `SOURCE_UNVERIFIABLE` alert, never a fabricated close/merge/review. Recovery (HTTP 200), a different status, any additional unavailable URL, or a malformed source **fails closed** pending source review and removal of the exception. The state is never silently `OPEN` or `CLOSED_UNMERGED`. This is a reviewed change to monitoring *coverage*, not a removal of the historical cohort, or permission to claim an 18-of-18 event snapshot. A returned count of 17 is prominently labelled in the observer and inbox output.

Expand Down Expand Up @@ -71,3 +71,15 @@ For the Headcount transition, the existing checkpoint key is **preserved** and r
**Checkpoint migration for yoagent #450 (2026-11-11, standing publication mandate):** The external [yoagent #450](https://github.com/yologdev/yoagent/pull/450) was submitted Open/Ready from [fork-local #1](https://github.com/opensiro-contributions/yoagent/pull/1) and read back at head `b06e768ad53ab01c4e1f26dca4af156652712684`. Native Rust fmt, clippy, rustdoc and full Linux all-features suite PASS (1,937 passed, 17 ignored), with the initial GASP file-permission failure attributed to a root-container capability after a successful capability-restricted rerun; other OS CI remains unverified. This exact allowlisted addition permits 29 → 30 checkpoint migration without manufacturing historic events. The external maintainers retain sole merge authority; no adoption or product benefit is claimed.

**Checkpoint migration for phax #205 (2026-11-11, standing publication mandate):** External [phax #205](https://github.com/lbdremy/phax/pull/205) was submitted Open/Ready from [fork-local #1](https://github.com/opensiro-contributions/phax/pull/1) and read back at head `45e90eaad89f7503ef7dbaed1d24a41c0418e3d5` against original `main@75afd70adae7adebe7f750c49942abbad308e774`. The exact source-matched Node 24 first-party `pnpm check:full` exited 0 (429 unit and 238 integration test files, typecheck/lint/format/architecture/knip/model-catalog all PASS); `pnpm build` and `pnpm site:build` also PASS. GitHub confirmed three changed files and disclosure v3. This exact allowlisted addition permits 30 to 31 maintenance targets without replaying historical events. There is no third-party acceptance, deployment or measured product adoption claim.

**Checkpoint migration for Fara #96 (2026-11-11 UTC, standing publication mandate):**
External [Fara #96](https://github.com/microsoft/fara/pull/96) was
published Open/Ready from [fork-local #1](https://github.com/opensiro-contributions/fara/pull/1)
after exact GitHub readback of original base
a675d6d61c41c47ae87bacefeab22caad18e3e84 and fork head
5b2499026322e25ac6d6182778914171a85f56b6, two changed files and
first-party Python targeted regression 3/3 PASS. Full browser/model tests
were not executed. This exact allowlisted addition permits a 31 to 32
maintenance-target migration without replaying historical comments or
inventing maintainer activity. Publication is not third-party acceptance,
measured downstream benefit or a VSM regrade.
2 changes: 2 additions & 0 deletions scripts/pr_maintenance_watch.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@
"https://github.com/FonaTech/Clouds-Coder/pull/50",
# yoagent standing-mandate publication, native all-features validated on 2026-11-11.
"https://github.com/yologdev/yoagent/pull/450",
# Fara focused 3/3 native tests and exact external #96 readback 2026-11-11.
"https://github.com/microsoft/fara/pull/96",
# phax source-native full-gate PASS and standing publication 2026-11-11.
"https://github.com/lbdremy/phax/pull/205",
}
Expand Down
6 changes: 3 additions & 3 deletions tests/test_portfolio_reconcile.py
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,9 @@ def pages(self, path):
class PortfolioReconcileTests(unittest.TestCase):
def test_frozen_source_union_exact_size_and_identity(self):
entries = portfolio.cohort()
self.assertEqual(len(entries), 38)
self.assertEqual(sum("watchlist" in row["groups"] for row in entries), 31)
self.assertEqual(len({row["url"] for row in entries}), 38)
self.assertEqual(len(entries), 39)
self.assertEqual(sum("watchlist" in row["groups"] for row in entries), 32)
self.assertEqual(len({row["url"] for row in entries}), 39)
self.assertEqual(len([row for row in entries if "casebook" in row["groups"]]), 9)
self.assertEqual(len([row for row in entries if "registry" in row["groups"]]), 1)
self.assertEqual(
Expand Down
10 changes: 7 additions & 3 deletions tests/test_pr_maintenance_watch.py
Original file line number Diff line number Diff line change
Expand Up @@ -145,10 +145,10 @@ def write_targets(self, root):
"targets": [{"repository": "Example/project", "number": 3, "url": URL}]}))
return p

def test_exact_manifest_has_thirty_one_targets_and_no_duplicates(self):
def test_exact_manifest_has_thirty_two_targets_and_no_duplicates(self):
data = watch.validate_targets()
self.assertEqual(len(data["targets"]), 31)
self.assertEqual(len({t["url"] for t in data["targets"]}), 31)
self.assertEqual(len(data["targets"]), 32)
self.assertEqual(len({t["url"] for t in data["targets"]}), 32)
self.assertIn("https://github.com/knowflow-ai/KnowWork/pull/4",
{t["url"] for t in data["targets"]})
self.assertIn("https://github.com/knowflow-ai/KnowWork/pull/4",
Expand All @@ -165,6 +165,10 @@ def test_exact_manifest_has_thirty_one_targets_and_no_duplicates(self):
self.assertIn("https://github.com/yologdev/yoagent/pull/450", watch.APPROVED_CHECKPOINT_ADDITIONS)
self.assertIn("https://github.com/FonaTech/Clouds-Coder/pull/50", {t["url"] for t in data["targets"]})
self.assertIn("https://github.com/FonaTech/Clouds-Coder/pull/50", watch.APPROVED_CHECKPOINT_ADDITIONS)
self.assertIn("https://github.com/microsoft/fara/pull/96",
{t["url"] for t in data["targets"]})
self.assertIn("https://github.com/microsoft/fara/pull/96",
watch.APPROVED_CHECKPOINT_ADDITIONS)
self.assertIn("https://github.com/lbdremy/phax/pull/205", {t["url"] for t in data["targets"]})
self.assertIn("https://github.com/lbdremy/phax/pull/205", watch.APPROVED_CHECKPOINT_ADDITIONS)
self.assertIn("https://github.com/cbrock84/headcount/pull/47",
Expand Down
2 changes: 1 addition & 1 deletion tests/test_resolved_pr_acceptance.py
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ def test_multiple_bot_report_comments_fail_before_write(self):
self.assertFalse(api.writes)

def test_only_reviewed_watchlist_and_no_unreviewed_expansion(self):
self.assertEqual(len(metric.validate_targets()["targets"]), 31)
self.assertEqual(len(metric.validate_targets()["targets"]), 32)
self.assertEqual(metric.INBOX_REPO, "opensiro/upstream")
self.assertEqual(metric.INBOX_NUMBER, 125)

Expand Down
4 changes: 2 additions & 2 deletions tests/test_verified_external_outcomes.py
Original file line number Diff line number Diff line change
Expand Up @@ -87,12 +87,12 @@ class VerifiedExternalOutcomeTests(unittest.TestCase):
def test_owner_ledgers_cover_exact_portfolio_with_one_replacement_chain(self):
items, blocked = outcomes.cohort()
self.assertEqual((len(items), sum(len(x["pr_urls"]) for x in items), blocked),
(37, 38, 21))
(38, 39, 21))
self.assertEqual(len([x for x in items if x["kind"] == "improvement"]), 2)
self.assertEqual(len([x for x in items if x["kind"] == "corrective"]), 13)
self.assertEqual(len([x for x in items if x["kind"] == "casebook"]), 8)
self.assertEqual(len([x for x in items if x["kind"] == "registry"]), 1)
self.assertEqual(len([x for x in items if x["kind"] == "publication"]), 13)
self.assertEqual(len([x for x in items if x["kind"] == "publication"]), 14)
self.assertIn("publication:knowwork-sandbox-start-race",
{x["id"] for x in items})
self.assertIn("publication:runloop-budget-finalization-idempotency",
Expand Down
Loading