Skip to content

feat(harbor): add pi agent baselines and audit dev34 results - #54

Merged
minixalpha merged 14 commits into
mainfrom
benchmark/pi-deepseek-flash-dev34-20261024
Oct 30, 2026
Merged

minixalpha merged 14 commits into
mainfrom
benchmark/pi-deepseek-flash-dev34-20261024

Conversation

@minixalpha

@minixalpha minixalpha commented Oct 24, 2026 •

Copy link
Copy Markdown
Owner

The standard Harbor workflow now runs a pinned Pi distribution with explicit model registration, preserved native artifacts, and auditable usage. Independent configuration snapshots and verified concurrent cache publication prevent job inputs from being replaced. Process supervision confirms cancellation before grading, and failed streams retain partial accounting.

Original task images can be cached with verified ranged downloads, registry-token renewal, native OCI metadata, and bounded batch storage. Original package sources, task resources, and official tests are preserved.

The existing 34-task development pool received one Pi + DeepSeek Flash attempt per task: 26 official passes, six failures, and two unscored legacy timeouts. The bilingual reports and audited JSON preserve raw exceptions, source amendments, public-reference exposure, credential-output findings, and partial costs. No complete, uncontaminated 34-task pass rate is claimed; raw traces remain private.

Validation: 269 Harbor tests pass after integration with current main. Frozen-output hashes and the synchronized English/Chinese report pair verified. The four Pi workflow review findings have regression coverage. The changelog records the new baseline workflow.

@minixalpha minixalpha changed the title feat(benchmarks): add pi deepseek flash dev34 baseline feat(harbor): add pi agent baselines and audit dev34 results Oct 30, 2026
@minixalpha
minixalpha marked this pull request as ready for review October 30, 2026 15:06
@minixalpha
minixalpha merged commit c9a95fa into main Oct 30, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant