diff --git a/benchmarks/harbor/README.md b/benchmarks/harbor/README.md index 932834b..592052a 100644 --- a/benchmarks/harbor/README.md +++ b/benchmarks/harbor/README.md @@ -192,6 +192,12 @@ preserves all 34 attempts, the supervision amendment, two unscored legacy timeouts, public-reference exposure, and partial accounting. It does not claim a complete, uncontaminated 34-task pass rate. +The authorized [three-task Pi rerun](reports/en/pi-deepseek-flash-dev3-rerun-20261102.md) +recorded two official passes and one scored failure after confirmed timeout +cancellation. Both new passes accessed public reference solutions and tests; +neither is an independent solution. Original outcomes remain unchanged. The +rerun also fixed Pi installation checks when Harbor returns absent output streams. + ### DeepSeek Harness baselines `harbor_adapter:DeepSeekHarnessAgent` runs the upstream Python SDK's diff --git a/benchmarks/harbor/README.zh-CN.md b/benchmarks/harbor/README.zh-CN.md index 0d69135..76b5979 100644 --- a/benchmarks/harbor/README.zh-CN.md +++ b/benchmarks/harbor/README.zh-CN.md @@ -151,6 +151,11 @@ Docker 空间受限时,可以通过 `batch_images` 计划按批恢复已注册 保留了全部 34 次尝试、进程托管修正、两次不评分的旧超时、公开参考材料接触和 不完整用量统计,不宣称完整且无答案接触的 34 题通过率。 +经授权的[三题 Pi 重跑](reports/zh-CN/pi-deepseek-flash-dev3-rerun-20261102.md) +记录了两次官方通过,以及一次超时后确认停止的计分失败。两次新的通过都访问了 +公开参考解法和测试,均不属于独立解题。原始结果保持不变。本次重跑还修复了 +Harbor 返回缺失输出流时 Pi 安装检查失败的问题。 + ### DeepSeek Harness baseline `harbor_adapter:DeepSeekHarnessAgent` 使用上游 Python SDK 的 `sdk-minimal` diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/README.md b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/README.md new file mode 100644 index 0000000..1dedaa2 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/README.md @@ -0,0 +1,94 @@ +# Authorized three-task Pi rerun + +Status: completed and audited on November 2, 2026. **Two official passes, both +reference-exposed, and one scored failure after confirmed timeout cancellation; +zero unscored attempts.** WAL and pipeline both read reference solutions and +tests whose reconstructed bytes match the registered task hashes. Sampler failed +all nine official tests after the supervisor stopped 11 processes. These results +provide no independent passing solution from the three reruns. + +See the complete [English report](../../reports/en/pi-deepseek-flash-dev3-rerun-20261102.md), +[Simplified Chinese report](../../reports/zh-CN/pi-deepseek-flash-dev3-rerun-20261102.md), +and [structured results](../../results/pi-deepseek-flash-dev3-rerun-20261102.json). +[Preflight evidence](preflight-evidence.json) and [launch evidence](launch-evidence.json) +precede the model jobs. The [freeze manifest](outputs-freeze.json) covers 326 +files before trajectory review; all 1,449 original frozen artifacts remain +unchanged. WAL exposed the credential in two private raw logs. All three official +verifiers completed on their first attempt. No containers remain running; +the six initially protected images remain present. See +[validation evidence](validation-evidence.json). + +The maintainer requested one fresh Pi attempt on `adaptive-rejection-sampler`, +`db-wal-recovery`, and `torch-pipeline-parallelism`. The first two attempts in the +[original baseline](../../reports/en/pi-deepseek-flash-dev34-20261024.md) were +unscored because their timeout boundaries were not confirmed. The original +pipeline attempt passed after reading public reference tests and a solution. +This is an outcome-selected development supplement, not an independent benchmark. +Keep every original outcome and every new attempt separately; do not select the +better score. + +The [v2 registration](experiment-v2.json) preserves Pi 1.4.0 and its complete +distribution digest, the original public model configuration, +`deepseek/deepseek-flash`, the official OpenAI-compatible endpoint, `high` +reasoning, native prompts/tools/compaction/retries, and unset temperature. +The alias is mutable; identical backend weights across dates are not established. +All three tasks retain their original revisions, files, Docker image identities, +resources, 900-second agent limit, and 900-second per-attempt verifier limit. +Setup has its separate 1,800-second limit. Concurrency is two with pair barriers, +in the original relative task order. There are no whole-trial retries. + +The registered standard adapter source is +`f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c`. Each job snapshots and hashes it through +`python -m harbor_adapter run --config ...`. Confirmed cancellation and mandatory +agent quiescence protect the agent/verifier boundary. Only verifier timeouts may +retry, at most three attempts on the same output with the original time limit. +Setup errors and verifier errors remain distinct from scored failures. + +The original registration used `46c554c2cf31a2d93bb37493083aab5b8fb82012`. +All three checks failed during Pi model-registration validation because Harbor +returned `None` for empty stderr. [Initial evidence](preflight-v1-evidence.json) +preserves those setup failures and confirms zero model or verifier executions. +The v2 source normalizes absent output streams while retaining rejection of +diagnostics and missing provider/model rows. Three regression cases failed +before the fix; all 304 Harbor tests passed after it. The initial model queue +remains unexecuted; v2 has distinct job names and orchestration records. + +Installation checks precede all model work. Pipeline now has the repository's +reviewed `torch-pipeline-parallelism-v1` dependency profile. Sampler and WAL +explicitly remain `not_configured`; successful Pi installation does not certify +their verifier dependencies. Existing verified APT, uv/Python-release, and PyPI +wheel caches preserve original packages and sources. The PyPI transport checks +each wheel's filename, size, SHA256, and original URL against current release +metadata. `UV_HTTP_TIMEOUT=600` permits verification before serving cold wheels; +task-native limits remain unchanged. Cache events have a separate audit log. + +Each task starts a fresh container and Pi session. Earlier trajectories, failure +diagnostics, tests, and solutions are not supplied to the agent. No new prompt or +network restriction is introduced. Freeze all new outputs before reviewing new +trajectories or hidden assertions, then audit reference-material access. A rerun +alone does not establish absence of answer exposure. Keep raw logs private and +publish complete English and Simplified Chinese reports with the same evidence. + +The following commands document the original launch from its pinned source +revision. Completed job names and registrations are immutable. A future repeat +needs a new registration and fresh job names, with its source checkout explicitly +pinned; do not reuse this completed queue from a newer repository revision. + +```bash +export UV_PROJECT_ENVIRONMENT="$PWD/jobs/dsh-harbor-venv" +export UV_CACHE_DIR="$PWD/.cache/uv" +uv run --frozen --project benchmarks/harbor \ + python benchmarks/harbor/scripts/run_comparison.py \ + --plan benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json \ + --phase preflight +# Supply the authorized DeepSeek credential through DEEPSEEK_API_KEY. +uv run --frozen --project benchmarks/harbor \ + python benchmarks/harbor/scripts/run_comparison.py \ + --plan benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json \ + --phase model +``` + +Native job outputs are under `jobs/pi-dev3-rerun-*-20261102/`, with immutable +inputs in the corresponding `-input/` directories. Orchestration records are in +`jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/`, with initial failed +checks retained under `jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/`. diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json new file mode 100644 index 0000000..e8e2d8c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf new file mode 100644 index 0000000..e191ddb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf @@ -0,0 +1,8 @@ +// Route only public APT HTTP origins through the verified index cache. +Acquire::http::Proxy::deb.debian.org "http://127.0.0.1:18742/"; +Acquire::http::Proxy::security.debian.org "http://127.0.0.1:18742/"; +Acquire::http::Proxy::archive.debian.org "http://127.0.0.1:18742/"; +Acquire::http::Proxy::archive.ubuntu.com "http://127.0.0.1:18742/"; +Acquire::http::Proxy::security.ubuntu.com "http://127.0.0.1:18742/"; +Acquire::http::Proxy::ports.ubuntu.com "http://127.0.0.1:18742/"; +Acquire::http::Proxy::old-releases.ubuntu.com "http://127.0.0.1:18742/"; diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json new file mode 100644 index 0000000..8bf9db8 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json new file mode 100644 index 0000000..c8258de --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json @@ -0,0 +1,148 @@ +{ + "services": { + "main": { + "volumes": [ + { + "type": "bind", + "source": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf", + "target": "/etc/apt/apt.conf.d/99nanopy-index-cache", + "read_only": true + } + ], + "environment": { + "UV_INSTALLER_GITHUB_BASE_URL": "http://127.0.0.1:18743", + "UV_PYTHON_INSTALL_MIRROR": "http://127.0.0.1:18743/astral-sh/python-build-standalone/releases/download", + "UV_HTTP_TIMEOUT": "600", + "UV_DEFAULT_INDEX": "http://127.0.0.1:18744/simple" + } + }, + "apt-index-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/apt_index_proxy.py", + "--bind", + "127.0.0.1", + "--port", + "18742", + "--cache", + "/cache", + "--workers", + "16" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "mem_limit": "256m", + "cpus": 0.5, + "volumes": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/scripts/apt_index_proxy.py:/proxy/apt_index_proxy.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/apt-indexes:/cache" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18742), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + }, + "uv-release-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/uv_release_cache.py", + "serve", + "--cache", + "/cache", + "--manifest", + "/manifest.json" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "cpus": 0.5, + "mem_limit": "256m", + "volumes": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/scripts/uv_release_cache.py:/proxy/uv_release_cache.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/uv-releases:/cache:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json:/manifest.json:ro" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18743), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + }, + "pypi-wheel-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/pypi_wheel_cache.py", + "--cache", + "/cache", + "--bind", + "127.0.0.1", + "--port", + "18744", + "--workers", + "16", + "--downloads", + "2", + "--audit-log", + "/cache/audit/pi-dev3-rerun-20261102.jsonl" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "mem_limit": "512m", + "cpus": 1.0, + "volumes": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/scripts/pypi_wheel_cache.py:/proxy/pypi_wheel_cache.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/pypi-ranges:/cache" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18744), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json new file mode 100644 index 0000000..e762efb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json @@ -0,0 +1,58 @@ +{ + "registered_at": "2026-10-18T13:25:43.131474+00:00", + "metadata_source": "Original release metadata preserved by the source manifests; no versions or payloads substituted.", + "source_manifests": { + "benchmarks/harbor/configs/dev34-main-baseline-20261012/dependency-release-manifest-v5.json": "bdf3d3ef8b7743700c5294c4a511c686a3928c178a77fd4abcba90ee4557ffcc", + "python-agent-release-manifest.json": "78aecd7c397c469196f9d246a160bf1fc99ad2d51cb71320b7d26ebf839647ce" + }, + "assets": [ + { + "version": "0.9.11", + "filename": "uv-x86_64-unknown-linux-gnu.tar.gz", + "url": "https://github.com/astral-sh/uv/releases/download/0.9.11/uv-x86_64-unknown-linux-gnu.tar.gz", + "digest": "sha256:817c0722b437b4b45b9a7e0231616a09db76bab1b8d178ba7a9680c690db19f0", + "size": 21699161, + "asset_id": 318993196, + "release_metadata_sha256": "c1a1378ece65ce1aa8343e57bdbd305289a8fc6d3d9180e8bb6b0f69acd91413" + }, + { + "version": "0.9.5", + "filename": "uv-x86_64-unknown-linux-gnu.tar.gz", + "url": "https://github.com/astral-sh/uv/releases/download/0.9.5/uv-x86_64-unknown-linux-gnu.tar.gz", + "digest": "sha256:2cf10babba653310606f8b49876cfb679928669e7ddaa1fb41fb00ce73e64f66", + "size": 21370871, + "asset_id": 306970260, + "release_metadata_sha256": "1921b17d3370aafb690b29d1d121ee8719ef3d3dbc0c7cf3b948c3059a927a5d" + }, + { + "version": "20251014", + "filename": "cpython-3.13.9+20251014-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251014/cpython-3.13.9%2B20251014-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "digest": "sha256:c0ccda275948c79996e46993c2c5476ff5cca606dee530f1dea712179131b348", + "size": 33517986, + "asset_id": 304305340, + "release_metadata_sha256": "8e5db21cfcc2bcafcb2127f269f00cbfdd1f74842a685acd2a14e3c1ad6f8394", + "repository": "astral-sh/python-build-standalone" + }, + { + "version": "20251120", + "filename": "cpython-3.14.0+20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251120/cpython-3.14.0%2B20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "digest": "sha256:66f385305ae0eefd6b65e2cf942bc91d943a61e26fb7581e029f760a2b04f393", + "size": 35376131, + "asset_id": 318881850, + "release_metadata_sha256": "e3a737862d2dddf11c73bfef6c2f2579e9d4ce5bd271cca53790c998590dee58", + "repository": "astral-sh/python-build-standalone" + }, + { + "version": "20251120", + "filename": "cpython-3.13.9+20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251120/cpython-3.13.9%2B20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "digest": "sha256:1bc9a94ab2425fb6ae58e45f69cab61647759b57895649701390822d976fa923", + "size": 34257108, + "asset_id": 318881608, + "release_metadata_sha256": "705690ab2fd466e90b4c868635c529f3c7fd0cb597d399e350f399e3f3e35e6c", + "repository": "astral-sh/python-build-standalone" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json new file mode 100644 index 0000000..05e7a4f --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json @@ -0,0 +1,196 @@ +{ + "schema_version": 1, + "registered_at": "2026-11-02T14:26:13.145168+00:00", + "experiment": "pi-deepseek-flash-dev3-rerun-20261102", + "total_model_runs": 3, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": ".", + "source_ref": "f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "torch-pipeline-parallelism-v1", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json", + "config_sha256": "ddea6527865a08d9b843bf536ed253604f090f883840c270f72491709389fc7e", + "job_name": "pi-dev3-rerun-install-pair-01-v2-20261102" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json", + "config_sha256": "b4aa98d085e1ca45bb51ac1cb8126c7b2b2e8aa40ed451164ba6748caf0324c1", + "job_name": "pi-dev3-rerun-install-pair-02-v2-20261102" + } + ], + "queue": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json", + "config_sha256": "31b593777e86bf20c52cd1d6f388a32dc76a20027912a11ddf621b13a8c15d7f", + "task": "terminal-bench/adaptive-rejection-sampler", + "repeat": 1, + "job_name": "pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json", + "config_sha256": "85316e8eb8a2cb3a17a00e78ebf6292ed32ade9af37120e828c79822afd567c9", + "task": "terminal-bench/db-wal-recovery", + "repeat": 1, + "job_name": "pi-dev3-rerun-db-wal-recovery-v2-20261102" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json", + "config_sha256": "f97e5181e4e09412358fd0f069a8983a5307a7d6acf82e58549c12437ef0ec0d", + "task": "terminal-bench/torch-pipeline-parallelism", + "repeat": 1, + "job_name": "pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json": "b4aa98d085e1ca45bb51ac1cb8126c7b2b2e8aa40ed451164ba6748caf0324c1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json": "f97e5181e4e09412358fd0f069a8983a5307a7d6acf82e58549c12437ef0ec0d", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json": "ddea6527865a08d9b843bf536ed253604f090f883840c270f72491709389fc7e", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/selection.json": "16e57fe4f15f8b6ab86603b09b8f0345002962bd2a0227c81069e1d6d847c9ca", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json": "e441052e83ec359a19833df8196f4a09fee1dd327dd1a54658fc177e2ad3e960", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json": "85316e8eb8a2cb3a17a00e78ebf6292ed32ade9af37120e828c79822afd567c9", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json": "b1e49942dc18ba55501092bfc9d63a5a5084d6032610e2ff9e659d5ec4d4070d", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json": "31b593777e86bf20c52cd1d6f388a32dc76a20027912a11ddf621b13a8c15d7f", + "benchmarks/harbor/scripts/run_comparison.py": "228dbff33c0ec8a2359e612da632ef543dcac4273db2f9a7d0573a24e856fbae", + "benchmarks/harbor/scripts/cache_image.py": "11ddde3fe86a6e4bfdefe7c3578794df98d237f701e40f219d76a541911c9f0c", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/pypi_wheel_cache.py": "ced6bd7c1bf2b6ac1964c39d057708ba0e52c37c61a609651c49105ea145ea74", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json": "8921379e1e5a2e98eed75cbd73ae1f4763aa7d65fe463a12e325be80cd3d432a" + }, + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Reuse the original mutable deepseek-flash alias; no identical backend checkpoint claim across dates." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh native Pi install-only checks for all three tasks before model work. Pipeline uses the current reviewed dependency profile; sampler and WAL explicitly remain not_configured. Every formal job repeats setup.", + "information_policy": "Outcome-selected supplement explicitly authorized by the maintainer. One fresh model attempt per task, original task prompts and native Pi behavior, no prior trajectories or solutions supplied, no new prompt or network restrictions. Keep original results unchanged. Freeze all new outputs before reviewing new trajectories or hidden assertions; audit reference exposure afterward. Other tasks remain unopened.", + "authorization": "The maintainer requested rerunning the two unscored Pi tasks and the reference-exposed pipeline task in the same way as before.", + "infrastructure_changes": [ + "Current standard adapter retains confirmed process cancellation and mandatory verifier quiescence.", + "Existing reviewed pipeline dependency preflight is now available.", + "Reuse verified APT, uv release, and original PyPI wheel transport from the completed Harness supplement; package sources, task bytes, native limits, and Pi configuration are unchanged." + ], + "runtime_environment": { + "UV_PROJECT_ENVIRONMENT": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/dsh-harbor-venv", + "UV_CACHE_DIR": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/uv" + }, + "transport_audit_log": ".cache/nanopy-harbor/pypi-ranges/audit/pi-dev3-rerun-20261102.jsonl", + "original_results": "benchmarks/harbor/results/pi-deepseek-flash-dev34-20261024.json", + "supersedes_model_queue": "experiment.json: zero model calls; installation checks failed on absent stderr.", + "amendment": "Normalize absent stdout/stderr in Pi model-registration validation. Retain rejection of missing models and diagnostics. Three new regression cases reproduced the bug before the fix; all 304 Harbor tests pass after the fix. No Pi, task, model, prompt, transport, or budget changes." +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment.json new file mode 100644 index 0000000..945f17c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment.json @@ -0,0 +1,193 @@ +{ + "schema_version": 1, + "registered_at": "2026-11-02T14:22:22.159372+00:00", + "experiment": "pi-deepseek-flash-dev3-rerun-20261102", + "total_model_runs": 3, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": ".", + "source_ref": "46c554c2cf31a2d93bb37493083aab5b8fb82012", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "torch-pipeline-parallelism-v1", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json", + "config_sha256": "ddea6527865a08d9b843bf536ed253604f090f883840c270f72491709389fc7e", + "job_name": "pi-dev3-rerun-install-pair-01-20261102" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json", + "config_sha256": "b4aa98d085e1ca45bb51ac1cb8126c7b2b2e8aa40ed451164ba6748caf0324c1", + "job_name": "pi-dev3-rerun-install-pair-02-20261102" + } + ], + "queue": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json", + "config_sha256": "31b593777e86bf20c52cd1d6f388a32dc76a20027912a11ddf621b13a8c15d7f", + "task": "terminal-bench/adaptive-rejection-sampler", + "repeat": 1, + "job_name": "pi-dev3-rerun-adaptive-rejection-sampler-20261102" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json", + "config_sha256": "85316e8eb8a2cb3a17a00e78ebf6292ed32ade9af37120e828c79822afd567c9", + "task": "terminal-bench/db-wal-recovery", + "repeat": 1, + "job_name": "pi-dev3-rerun-db-wal-recovery-20261102" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json", + "config_sha256": "f97e5181e4e09412358fd0f069a8983a5307a7d6acf82e58549c12437ef0ec0d", + "task": "terminal-bench/torch-pipeline-parallelism", + "repeat": 1, + "job_name": "pi-dev3-rerun-torch-pipeline-parallelism-20261102" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json": "b4aa98d085e1ca45bb51ac1cb8126c7b2b2e8aa40ed451164ba6748caf0324c1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json": "f97e5181e4e09412358fd0f069a8983a5307a7d6acf82e58549c12437ef0ec0d", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json": "ddea6527865a08d9b843bf536ed253604f090f883840c270f72491709389fc7e", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/selection.json": "16e57fe4f15f8b6ab86603b09b8f0345002962bd2a0227c81069e1d6d847c9ca", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json": "e441052e83ec359a19833df8196f4a09fee1dd327dd1a54658fc177e2ad3e960", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json": "85316e8eb8a2cb3a17a00e78ebf6292ed32ade9af37120e828c79822afd567c9", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json": "b1e49942dc18ba55501092bfc9d63a5a5084d6032610e2ff9e659d5ec4d4070d", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json": "31b593777e86bf20c52cd1d6f388a32dc76a20027912a11ddf621b13a8c15d7f", + "benchmarks/harbor/scripts/run_comparison.py": "228dbff33c0ec8a2359e612da632ef543dcac4273db2f9a7d0573a24e856fbae", + "benchmarks/harbor/scripts/cache_image.py": "11ddde3fe86a6e4bfdefe7c3578794df98d237f701e40f219d76a541911c9f0c", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/pypi_wheel_cache.py": "ced6bd7c1bf2b6ac1964c39d057708ba0e52c37c61a609651c49105ea145ea74" + }, + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Reuse the original mutable deepseek-flash alias; no identical backend checkpoint claim across dates." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh native Pi install-only checks for all three tasks before model work. Pipeline uses the current reviewed dependency profile; sampler and WAL explicitly remain not_configured. Every formal job repeats setup.", + "information_policy": "Outcome-selected supplement explicitly authorized by the maintainer. One fresh model attempt per task, original task prompts and native Pi behavior, no prior trajectories or solutions supplied, no new prompt or network restrictions. Keep original results unchanged. Freeze all new outputs before reviewing new trajectories or hidden assertions; audit reference exposure afterward. Other tasks remain unopened.", + "authorization": "The maintainer requested rerunning the two unscored Pi tasks and the reference-exposed pipeline task in the same way as before.", + "infrastructure_changes": [ + "Current standard adapter retains confirmed process cancellation and mandatory verifier quiescence.", + "Existing reviewed pipeline dependency preflight is now available.", + "Reuse verified APT, uv release, and original PyPI wheel transport from the completed Harness supplement; package sources, task bytes, native limits, and Pi configuration are unchanged." + ], + "runtime_environment": { + "UV_PROJECT_ENVIRONMENT": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/dsh-harbor-venv", + "UV_CACHE_DIR": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/uv" + }, + "transport_audit_log": ".cache/nanopy-harbor/pypi-ranges/audit/pi-dev3-rerun-20261102.jsonl", + "original_results": "benchmarks/harbor/results/pi-deepseek-flash-dev34-20261024.json" +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json new file mode 100644 index 0000000..c68d4ea --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json @@ -0,0 +1,33 @@ +{ + "observed_at": "2026-10-24T14:07:09.186521+00:00", + "images": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "image": "alexgshaw/adaptive-rejection-sampler:20251031", + "image_id": "sha256:985bed39e8448bc95f138b5f766b59003a95c4be171c96af8e76448338f3dd8f", + "repo_digests": [ + "alexgshaw/adaptive-rejection-sampler@sha256:985bed39e8448bc95f138b5f766b59003a95c4be171c96af8e76448338f3dd8f" + ], + "size_bytes": 29731756 + }, + { + "task": "terminal-bench/db-wal-recovery", + "image": "alexgshaw/db-wal-recovery:20251031", + "image_id": "sha256:0e33ea5ec823975d1bd6c3778395c9f94251dd88f571146057bff6adb7e4594e", + "repo_digests": [ + "alexgshaw/db-wal-recovery@sha256:0e33ea5ec823975d1bd6c3778395c9f94251dd88f571146057bff6adb7e4594e" + ], + "size_bytes": 175276145 + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "image": "alexgshaw/torch-pipeline-parallelism:20251031", + "image_id": "sha256:3cb7b39d86f1704b20e668851da0b08b34b3270c01d583f136d65190984a8d1b", + "repo_digests": [ + "alexgshaw/torch-pipeline-parallelism@sha256:3cb7b39d86f1704b20e668851da0b08b34b3270c01d583f136d65190984a8d1b" + ], + "size_bytes": 29725531 + } + ], + "matches_prior_registration": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json new file mode 100644 index 0000000..413dba8 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json @@ -0,0 +1,43 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json new file mode 100644 index 0000000..796e76d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/launch-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/launch-evidence.json new file mode 100644 index 0000000..65e33cf --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/launch-evidence.json @@ -0,0 +1,10 @@ +{ + "registered_at": "2026-11-02T14:31:07.469972+00:00", + "registration_sha256": "4b1a2df292b461e68f25818917a6060f94c9b95f808922356b7fe14d4a512dd3", + "preflight_evidence_sha256": "3cbc66bea671662be2cb4dff7dc9d5dfb325a03592e94254ebc0d8b5da09852d", + "source_ref": "f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c", + "model_runs_authorized": 3, + "credential_source": "Existing repository DeepSeek credential supplied only through the process environment; value and fingerprint omitted.", + "same_pi_distribution_and_public_model_config": true, + "new_prompt_or_network_restriction": false +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/outputs-freeze.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/outputs-freeze.json new file mode 100644 index 0000000..8c71314 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/outputs-freeze.json @@ -0,0 +1,1359 @@ +{ + "schema_version": 1, + "experiment": "pi-deepseek-flash-dev3-rerun-20261102", + "frozen_at": "2026-11-02T15:05:23.352293+00:00", + "policy": "All three new model attempts and both generations of installation checks ended. Hashes were frozen before reviewing new model trajectories, official test outputs, or hidden assertions. Original outputs remain unchanged. Raw logs stay private.", + "model_attempts": 3, + "installation_attempts": 6, + "jobs": [ + { + "job_name": "pi-dev3-rerun-install-pair-01-20261102", + "installation_only": true, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/pi-dev3-rerun-install-pair-01-20261102.json" + }, + { + "job_name": "pi-dev3-rerun-install-pair-02-20261102", + "installation_only": true, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/pi-dev3-rerun-install-pair-02-20261102.json" + }, + { + "job_name": "pi-dev3-rerun-install-pair-01-v2-20261102", + "installation_only": true, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-install-pair-01-v2-20261102.json" + }, + { + "job_name": "pi-dev3-rerun-install-pair-02-v2-20261102", + "installation_only": true, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-install-pair-02-v2-20261102.json" + }, + { + "job_name": "pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102", + "installation_only": false, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102.json" + }, + { + "job_name": "pi-dev3-rerun-db-wal-recovery-v2-20261102", + "installation_only": false, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-db-wal-recovery-v2-20261102.json" + }, + { + "job_name": "pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102", + "installation_only": false, + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102.json" + } + ], + "file_count": 326, + "files": { + ".cache/nanopy-harbor/pypi-ranges/audit/pi-dev3-rerun-20261102.jsonl": { + "sha256": "1b7bd58e6da2936403534c256731bf2192400c20a169c8aa9563c5486f3ba443", + "size": 154111 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/adaptive-rejection-sampler.json": { + "sha256": "31b593777e86bf20c52cd1d6f388a32dc76a20027912a11ddf621b13a8c15d7f", + "size": 1212 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/apt-cache.conf": { + "sha256": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "size": 549 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/db-wal-recovery.json": { + "sha256": "85316e8eb8a2cb3a17a00e78ebf6292ed32ade9af37120e828c79822afd567c9", + "size": 1201 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json": { + "sha256": "e441052e83ec359a19833df8196f4a09fee1dd327dd1a54658fc177e2ad3e960", + "size": 4772 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-release-manifest.json": { + "sha256": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "size": 3250 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json": { + "sha256": "4b1a2df292b461e68f25818917a6060f94c9b95f808922356b7fe14d4a512dd3", + "size": 13096 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment.json": { + "sha256": "1e95f2ed1c8638f06c87f8f92ec87215074c830e3b6a95d54c7289082ec350aa", + "size": 12502 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/image-digests.json": { + "sha256": "b1e49942dc18ba55501092bfc9d63a5a5084d6032610e2ff9e659d5ec4d4070d", + "size": 1310 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-01.json": { + "sha256": "ddea6527865a08d9b843bf536ed253604f090f883840c270f72491709389fc7e", + "size": 1360 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/install-pair-02.json": { + "sha256": "b4aa98d085e1ca45bb51ac1cb8126c7b2b2e8aa40ed451164ba6748caf0324c1", + "size": 1212 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/launch-evidence.json": { + "sha256": "ec58e347d0744457699819c6fe15c17364e41248182e93fc8a31420b3451b238", + "size": 586 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-distribution.json": { + "sha256": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "size": 32391 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json": { + "sha256": "3cbc66bea671662be2cb4dff7dc9d5dfb325a03592e94254ebc0d8b5da09852d", + "size": 2291 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json": { + "sha256": "8921379e1e5a2e98eed75cbd73ae1f4763aa7d65fe463a12e325be80cd3d432a", + "size": 2728 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/selection.json": { + "sha256": "16e57fe4f15f8b6ab86603b09b8f0345002962bd2a0227c81069e1d6d847c9ca", + "size": 4116 + }, + "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json": { + "sha256": "f97e5181e4e09412358fd0f069a8983a5307a7d6acf82e58549c12437ef0ec0d", + "size": 1212 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/image-cache.jsonl": { + "sha256": "3631be9771ac65f006f3165771b2a2a0355888ae20f42a7949354070235525f6", + "size": 1457 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/image-downloads.log": { + "sha256": "411b54818c6270b3493dba3e3fd43ba3cd0b3374852df3b5e071d558456d209e", + "size": 2450 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/initial-docker-images.json": { + "sha256": "449d13f34e0b28d37629f60676696304188487bd8f7361f38ffae2ebbe0b4f60", + "size": 465 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/pi-dev3-rerun-install-pair-01-20261102.json": { + "sha256": "a025b4f175c6a151a27ea1ee0c427a29349fe94622beca859effb397354fa3a9", + "size": 42485 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/pi-dev3-rerun-install-pair-01-20261102.log": { + "sha256": "23dcab30b6328cf6c76423ff65aab8aa45e32b44864054ddfed59837aaed8dd8", + "size": 223 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/pi-dev3-rerun-install-pair-02-20261102.json": { + "sha256": "f373173384e3d691b741b7c6e9f57f027027f558aebba3d8c9ec0a7167a979a2", + "size": 28330 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/pi-dev3-rerun-install-pair-02-20261102.log": { + "sha256": "951d523cc1de7bac889fa0ad27c85582ef312d8d94c06b2ee24396310c7b5b9e", + "size": 223 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-orchestration/registration.json": { + "sha256": "fda8fe30ab4ae4e06fb0547409fa4c74a4d00bfdb8efe2a12c09a3490c0d91f6", + "size": 12985 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/image-cache.jsonl": { + "sha256": "046a3fd60f4fa52a8aa31df911ea1df830c1b3f7cbf3905b66946cb502a79fe8", + "size": 2914 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/image-downloads.log": { + "sha256": "cbb5d57cc14e6845b57d5acecdbf7dd8345697be425e360f1072caa7fbfece31", + "size": 4900 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/initial-docker-images.json": { + "sha256": "449d13f34e0b28d37629f60676696304188487bd8f7361f38ffae2ebbe0b4f60", + "size": 465 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102.json": { + "sha256": "d99a59461fb935f2dd70bb362f6f3fb8ed772ebb7044a01c676b016b9c417391", + "size": 28743 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102.log": { + "sha256": "2161dbf9fd5f90c22bf118b952fbd25636faace54a4ecc36988aaa46a3281f5d", + "size": 251 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-db-wal-recovery-v2-20261102.json": { + "sha256": "6b28e26b732acc3d357124e9fed3087b12acdd7e35ed50b0b4091d8d20c6931e", + "size": 15696 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-db-wal-recovery-v2-20261102.log": { + "sha256": "77a318e5a130e816099dbfdfca495c69c0d654d058fed39cfe76292a109a963b", + "size": 229 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-install-pair-01-v2-20261102.json": { + "sha256": "3692e45808c80ab83221529e37ea2d361bf6d1dfcd6da47949cc5851dfea5e76", + "size": 42378 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-install-pair-01-v2-20261102.log": { + "sha256": "ca435233f122df611bf81322288f2c33eb5f3ba4b928c58f33a8e18f264a9826", + "size": 229 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-install-pair-02-v2-20261102.json": { + "sha256": "c4998527bb719726993fe69d18ffe75cabb57c40449b6e84553a018cf369ead9", + "size": 28461 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-install-pair-02-v2-20261102.log": { + "sha256": "65bfed30f49d11845031e487590cc5477f952bbb7e3cd3bb3501987f921128ac", + "size": 229 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102.json": { + "sha256": "9f605bca2f01b4e7773be1f5b16cb45b026cdb11de6f657f29854e2300449b5e", + "size": 28958 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102.log": { + "sha256": "e66f65ce6552b86c3b9920dcad87cdeb626e60969cb0a7169dc55d4297f0d34f", + "size": 251 + }, + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/registration.json": { + "sha256": "28b44ce2d8578a90bcecbdc501630dff4751fcf0becd8b908810bfff6b1f193d", + "size": 13585 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/pi/sessions/2026-11-02T14-32-01-333Z_01a19cac-a035-7739-b265-0ad05941e900.jsonl": { + "sha256": "fbe68f870e06fef709f7cc3299fa5360e9bb50fcc92c3a94789ddb610aa94960", + "size": 16200 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/pi.txt": { + "sha256": "7aed534f73f9313b3352a9e8e9db1c5e4d550824d7ba29c2ad25bdd225c3f769", + "size": 39284 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/process-supervision.json": { + "sha256": "da8bb50683fe264d8f3f45d20e9b46e60348c9a3e8fbb946381866e4a4b5aad0", + "size": 76 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/config.json": { + "sha256": "4b63f1a4c5349fd341ba48bcc2bc9e4ad6a0cc60183d7920f86b4e7cc57aeb75", + "size": 1664 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/exception.txt": { + "sha256": "61e9ffcda5c7a2f65b9c347f6226116a2d1e7998b16f586559206a49c7e85c93", + "size": 5023 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/lock.json": { + "sha256": "39c6807e04902306997d355cb538d362f6a4877b9669ff86d5f2d3b85b255723", + "size": 2257 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/result.json": { + "sha256": "8309f93d11809990a15105b480c2b55e267e21f00173c5ec290b619d22eeb2d7", + "size": 43763 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/trial.log": { + "sha256": "cff6c8d2aa8d9362c8c083afdf591586a06d4a572facf4c27e7535b892541753", + "size": 15388 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/attempts/1/ctrf.json": { + "sha256": "2e89dd4e8c70e104e868e33bb4d9c2984e2f8d3b567eb3e8385a1d322c03025c", + "size": 49689 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/attempts/1/test-stdout.txt": { + "sha256": "9bfdf17f21c887bc39661be3d0f0490154aa3e15982054b4c5e29a70e04fda82", + "size": 46552 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/ctrf.json": { + "sha256": "2e89dd4e8c70e104e868e33bb4d9c2984e2f8d3b567eb3e8385a1d322c03025c", + "size": 49689 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/retry-summary.json": { + "sha256": "8cb3d145bb09c9d920a320d2460b0d9b2811d92aaefb5589477853b1f1897ed1", + "size": 264 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/test-stdout.txt": { + "sha256": "9bfdf17f21c887bc39661be3d0f0490154aa3e15982054b4c5e29a70e04fda82", + "size": 46552 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/config.json": { + "sha256": "a9173ad80350b7c223c8cfdac7679e23bba318c7ff27d60692ff259419eff068", + "size": 1690 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/job.log": { + "sha256": "ae2dee8814200c45f7a3ca07b20f8831df8c62f631bd0c4c42dfdb4a323e93f4", + "size": 15575 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/lock.json": { + "sha256": "a8adf6a911a772e6494e588e7c277c5e51f19a5b551a786c9cf4bf80936bfb0f", + "size": 3419 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/result.json": { + "sha256": "c687590d2c909c731be402f7f563c5547480987e4baf2e6851ed49acb46420bc", + "size": 1281 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/config.json": { + "sha256": "20177324e41ca3e1055acf370dd4bb2fc218c45d6c039924a1c5b094a38cc52c", + "size": 3226 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/harbor.log": { + "sha256": "638b80616b5f13c00db717b3f6ab2c6885846459dacf2c3e9308f4d1cf91f469", + "size": 1513 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/manifest.json": { + "sha256": "dafde8a1ec3eb104265f64f39f201ab24f956c25abcada8ba516cd2e33dc3472", + "size": 37129 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "2f07ed0b838d11e286a548c5671c7793c396de3422defe6aa8ef1689e2cb25ce", + "size": 565 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "180df30883fe10f3e640124bd55262370ce17018a74bfb9f65fece2df7b5f813", + "size": 18684 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "c28ef1415f64fca1bf621a154a1a000f090b6ce4ea47232a9610b45f203475a6", + "size": 14972 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "c59e2061245dda3010dcffc765a05a336494b32b3aca3c396ca9b2cb4c19719a", + "size": 15026 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "81641e8cebf1296e420f75c8ba2b55e01eeffd342fbeea6768860b7ca02658bc", + "size": 18439 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "cf4fd8d8ffd1b7fd34c57f9d1fb0c312b3e0f893dd94cd756b60b7848565c3c9", + "size": 14674 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "1c6005b6b5c4a14c8032a5bfc8541c96a4a4ca5bed524fde86bf517d34e50e69", + "size": 12416 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/summary.json": { + "sha256": "0d3e12e1510b5fd8c7179b0ece436b0f3aa9ff2cc8dd754fe6a6699e3780e57e", + "size": 26573 + }, + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/config.json": { + "sha256": "f4e46f11028f41c331e4ff609ccb86ebef171c28c78eea906fcd825dfdf6e202", + "size": 1657 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/bootstrap.json": { + "sha256": "5815e57bc6e873824175e0ed3918b4a1c95bb61e53f83dc400ef4ab5adcb3fe0", + "size": 11901 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/pi/sessions/2026-11-02T14-31-38-781Z_01a19cac-481d-7552-83b4-4dcfe1ccce9d.jsonl": { + "sha256": "5058bc633f0b7125a511778504a0ff6946ae0f4d10214557f695a147326fecf2", + "size": 462710 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/pi.txt": { + "sha256": "91e7912cc49f447753487185b4888bb117a81cf64805f045de45a61bd90b98d0", + "size": 3203983 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/config.json": { + "sha256": "0869ef045e37b61c7792614cad468da63fc9f90c3805e2224205c1149f4bf813", + "size": 1620 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/lock.json": { + "sha256": "227e0700ac90906b25192285832237f889d96be2b341f9cb425fe1b038c10df3", + "size": 2235 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/result.json": { + "sha256": "5f964859806db7d27e7452613245844c3f29d98179f13414f8a2c398322d8a98", + "size": 22444 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/trial.log": { + "sha256": "df336008a45d9d433c2766d24ff759e6752197d8b0de732e403212a809361c12", + "size": 13919 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/attempts/1/ctrf.json": { + "sha256": "16828666e70c6d13ee3200f110471c023cffa287865148fb989bac2a129c0d46", + "size": 2877 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/attempts/1/test-stdout.txt": { + "sha256": "2782d10936127751b643e612a69c6d8a4db4d1cb803f88fada8faf4e638e7ae6", + "size": 1927 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/ctrf.json": { + "sha256": "16828666e70c6d13ee3200f110471c023cffa287865148fb989bac2a129c0d46", + "size": 2877 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/retry-summary.json": { + "sha256": "ee2517d612b17d7c63bd35417867402f39b2e0764f5a63f476ce496a1ed0ad8c", + "size": 269 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/test-stdout.txt": { + "sha256": "2782d10936127751b643e612a69c6d8a4db4d1cb803f88fada8faf4e638e7ae6", + "size": 1927 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/job.log": { + "sha256": "df336008a45d9d433c2766d24ff759e6752197d8b0de732e403212a809361c12", + "size": 13919 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/lock.json": { + "sha256": "9f19b8a9264a141672d792ac6cab06ea14b183a568e468e598ff29e60eb89cfb", + "size": 3397 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/result.json": { + "sha256": "0e470b878dd5ccb7ca27165d916a79740e66a09606c18d385d64b18a59204e75", + "size": 1141 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/config.json": { + "sha256": "33e0e6860ff86ab386dc1dac87b9e6896fc5effd257e13792910d76a9ee8c83b", + "size": 3193 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/harbor.log": { + "sha256": "4e4f682262fd9c3af28dd9f5c312b31c38b7d0246ca07c22988ca454d097c23d", + "size": 1153 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/manifest.json": { + "sha256": "2a68f7a0b4d824dd997238b61538d6fe105b73d962a4542f98e575a29ac0e816", + "size": 37129 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "7d145d8a9a5a2e8f1a792abdc29b984459d6d3377b07c083495d29a6aff8278b", + "size": 554 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "8edec09c44a50508d567625146d0b27ad603a8f4251eb32d613a645b27e73ea2", + "size": 18673 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "662150bc1d0f6bc4559e773c560e97ec12b17b320b1fc7f3783c39666b88986b", + "size": 14961 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "e086c6ccac32bd82f6ef61b7290fbbb4e48a2c4814c9fbb27ec1a23a40b8bb35", + "size": 15015 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "a6028874cbd11666d144a0b631baff5325e592e8369d0efc24ce04334301c0b6", + "size": 18428 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "b9309740a2e7ff2f1c5f0369f13cf6586047734f04e2b7b59f125914a6c84a00", + "size": 14663 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "1c6005b6b5c4a14c8032a5bfc8541c96a4a4ca5bed524fde86bf517d34e50e69", + "size": 12416 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/summary.json": { + "sha256": "d8535afe40590a33fd4338465d194c6021e2521efa3953ef95e8458c9ca3c0e3", + "size": 14022 + }, + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/agent/bootstrap.json": { + "sha256": "c38914403bd502084e8d106a6eaa84d9a413cd8000b01c99bb62f8f376bfec1c", + "size": 23176 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/config.json": { + "sha256": "505f35662f19e244a1bb0b688a7cae9e604d7883e17bfc41936ad7cd9e97af3a", + "size": 1687 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/exception.txt": { + "sha256": "77c4ea58bf0b59c011a3f97936de754f09deebac2c65c6b0629eda5a6595982c", + "size": 2196 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/lock.json": { + "sha256": "939602966342abf4448e115ad182a11245b01a4a708a2e1605526e3f66c8e9aa", + "size": 2241 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/result.json": { + "sha256": "e22663dcc70df4d73a1d2035729f3df0af50d019fc865d73d010b5e813b83efe", + "size": 7070 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/trial.log": { + "sha256": "1eab56e636f7ba4fdcc6352cbe08745fa9975e85b0d29ece1b446ba44e921c34", + "size": 6594 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/config.json": { + "sha256": "ec3ca582b2708a49a703536fed9aff5cd994dc3f956d64fde05210ea252c1e40", + "size": 1881 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/agent/bootstrap.json": { + "sha256": "2b6fc50d240d22bb62b95caafd54ffe8b42f91dc00d400ed3ff968df1133078a", + "size": 11941 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/config.json": { + "sha256": "ec19b8407d60a4c07d73fd2eb296579543773495dc42a8ae1e9540feebfc7a8d", + "size": 1665 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/exception.txt": { + "sha256": "77c4ea58bf0b59c011a3f97936de754f09deebac2c65c6b0629eda5a6595982c", + "size": 2196 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/lock.json": { + "sha256": "46bd7dcf08d4a091169cae34a36dd30ab7539e900d563781214950d32c57020f", + "size": 2230 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/result.json": { + "sha256": "e2afb6f64ec65d703bb73e2d297cfefa68eaec85f19bc4464572ce11b6b68354", + "size": 7004 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/trial.log": { + "sha256": "c0cac108751e08d6ac856b1ea11b39d1ec1f8e1f3d8fafe005470fc7902bd12d", + "size": 6583 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/job.log": { + "sha256": "dd7a1facda07002783b6bfe2c0dbb6af78eaefe23f7c8d45d8cdbbbfb163771c", + "size": 13325 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/lock.json": { + "sha256": "0c701e562f9eb4f9dd93dc9201716c92f26a23aa267595c65695ffc4bfef6739", + "size": 6075 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102/result.json": { + "sha256": "b194f03872f9b5e6d02047e179595c542a6f9642360a0331c90c62e7c9cb440e", + "size": 1124 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/config.json": { + "sha256": "a775ccdce803832ba1d84c460e0d15c015d6566e1e43a6189f030a1d35d0d217", + "size": 3493 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/harbor.log": { + "sha256": "a01edffb3a8ba8c2e992bbb8e212f23a98b571b4d2e77358a99ff9d4aa85cd14", + "size": 1213 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/manifest.json": { + "sha256": "84239a2ec90c6b7e87ab91a17229e54cdbee4fa7c696715ee449474962f4acac", + "size": 37129 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "be588e7f6f7a5e9a5add1303036325b3261fbf0030ea8cb4c131165f649fdd98", + "size": 551 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "cbfae3319c3457e230aab7811e8adab7511ed71be445d034787dfbe450e45328", + "size": 18670 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "1ca8f1b8d0362f912812d74682955410299cf8c913c4a0cb42e876db803deb89", + "size": 14958 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "9941ecaf4f362a748f707d770c376557d49335bc2e5223854653873ae0fb1007", + "size": 15012 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "8be7c32a21ba9940e16c74bc6bb1f34d17cbeeb47724433cfd41eb44a4ec2a44", + "size": 18361 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "483bd93e230d059e785035a2d476048e8c7c31c9b0918b958747e3a87ad2241f", + "size": 14660 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "e703174d76a1b670466ba06904db6b08a26769d42ac974274f61423ff0778fd6", + "size": 12355 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/summary.json": { + "sha256": "c6ab1e9458859b70f763348f0305c05b980a7133765f567b719ef8d2fafeb778", + "size": 39941 + }, + "jobs/pi-dev3-rerun-install-pair-01-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/config.json": { + "sha256": "08a340bad2550bbd6d4dec4fa9733a69222a59c4a0fc0081ecf72f205deb1640", + "size": 1693 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/lock.json": { + "sha256": "c13a14858d7c269fb86e19021cd0ee33d6c26ce12acb90e5989bf6ebded50d16", + "size": 2244 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/result.json": { + "sha256": "5fa9b42c08b9862af1940b29749e58e647a3cd6de563622e9c884462dbed7d91", + "size": 4603 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/trial.log": { + "sha256": "da201beecba9d03d75c4ef4ade14675b5cf5b2764a62eea7938a18509f781f9a", + "size": 6445 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/config.json": { + "sha256": "da3463763c211f25f1561ca699e62945cf1a3168679a117b5c8e06b010c3d259", + "size": 1887 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/agent/bootstrap.json": { + "sha256": "5815e57bc6e873824175e0ed3918b4a1c95bb61e53f83dc400ef4ab5adcb3fe0", + "size": 11901 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/config.json": { + "sha256": "a5e0aa9e72769d4646338b7e27282cac6767afed0218da886e238a0e41901a7d", + "size": 1671 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/lock.json": { + "sha256": "5922480435a2d99a41e22ea57233c8c855f869d45dfee061b85627922017f087", + "size": 2233 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/result.json": { + "sha256": "9f7386437030ec248d95431cdbf4a02fabe9b67733383e2e6bceac1df8c30372", + "size": 4537 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/trial.log": { + "sha256": "e6e1a38ee126fff4a19a9486379c33715180578b5e0584891715d88f35e2cb82", + "size": 6445 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/job.log": { + "sha256": "77757654f7785d4e74b3287754f79777c73e771f54d2c6f6e7d6f9b316aebfb4", + "size": 12890 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/lock.json": { + "sha256": "80c27e32690a5cac6b29c397b965cbb6dcafbe9d4fdf54c1133bf1b4f1b6cd6c", + "size": 6081 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/result.json": { + "sha256": "540fd4aaf79d1f892dd9cc810cce5491288670e3af72841fe53ac6ac7d67f667", + "size": 933 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/config.json": { + "sha256": "0db9a3573e8182c2666f2724fff313fa772bfd66ae894abb422ebbb9521f1efa", + "size": 3499 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/harbor.log": { + "sha256": "2bd3115c3018ab21152f32e3a577ff151d254bf1e89711144c7e669dd835e9ef", + "size": 934 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/manifest.json": { + "sha256": "14b9d578128cc52197442c7839ba1208389d4c09489e0bc6ab3163b7cb8596c4", + "size": 37129 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "5e18e7a98fbf39208502d2a854411fac12158586a4ebc7495dfdabb3bdc6d9cc", + "size": 554 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "aa5ebd8c68d2425e7fa93de85d42d9715b37ec3e861c262e52f168e736275e10", + "size": 18673 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "2a2ee7b668f35d2e1c8a5bd557af6f39c8fc13e5b3700ea388e36a5636884159", + "size": 14961 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "77490aec61791f7baa9ca55cf8fea487ccf2aa1abd781399ccc1f5e17ae48c1c", + "size": 15015 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "ba84e380312c191c53c6016df505c418c9aee7058656aee7556e71dd37b0f9b4", + "size": 18428 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "6e79cc6f9a04f09ba6e65623be56ad4d10af6bb0edb5a0218355eea8d101ab1e", + "size": 14663 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "1c6005b6b5c4a14c8032a5bfc8541c96a4a4ca5bed524fde86bf517d34e50e69", + "size": 12416 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/summary.json": { + "sha256": "c9aae9d7b7464b3b9f03f8c592141a95a9285bd606b56cbb88613629c25d5b6b", + "size": 39829 + }, + "jobs/pi-dev3-rerun-install-pair-01-v2-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/config.json": { + "sha256": "a308f0e2240c0665d24a05a687ed8671fdafc512bc5ee74d48fde558fcc18020", + "size": 1713 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/job.log": { + "sha256": "0d682dbebbc0d9decf291a6d1c8f9c46a65b0a87447e8114d224356c79dd913d", + "size": 6668 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/lock.json": { + "sha256": "3d6250369ca1306ff8bad394cc1873cbecbaf8fa9f7e69f0da6f9051f464f56f", + "size": 3403 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/result.json": { + "sha256": "be631f7cf46b12b82f155790236b84dbd866b262c5b69bc5f1cf9ee3bb3f1f0c", + "size": 1072 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/agent/bootstrap.json": { + "sha256": "1d6ad688edd691d586897ee2bab87fdb64e419aea117dfcf89212e5289bdb705", + "size": 23199 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/config.json": { + "sha256": "c68b45a68e7c4457253b26899a13f969429ad9e8a968d548a4e262500b984821", + "size": 1687 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/exception.txt": { + "sha256": "125c7f707d8a78ee446c8d1e13d003e09fa23134dc212361219ce5b97c03bb8f", + "size": 2196 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/lock.json": { + "sha256": "e242875d220beb14c3f8201666ddd1b3c87cccc46e1cbffe7ef24eb9c019e79a", + "size": 2241 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/result.json": { + "sha256": "d1b9a45e50862d4ea1760e93a7fa86abaacbb1f78bb0f179acc1698db32d92db", + "size": 7070 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/trial.log": { + "sha256": "6a4da3aecbe031b2b25102b3b9b992d1e4bb16b8ac0a1a49efd2405ca8dca22c", + "size": 6594 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/config.json": { + "sha256": "4edfe199184db4d07ed0cce116d544bf95004d90ba8b614fe49837e389c4122b", + "size": 3197 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/harbor.log": { + "sha256": "a265486c244af9eec0152bba1eeec3b1aa23f32fc160c162e2a594ac4009e4f4", + "size": 1210 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/manifest.json": { + "sha256": "bc035c72d250d58dd617c4d64bee34dedddad2f90b3ea2849c760f756e7cb50a", + "size": 37129 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "ebae1cd4360bd7cd22e83c13c9a108580e2b0d304043f0f8d882a6c253c4b48c", + "size": 551 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "3550fadcc7c8ffddbcd07b0660fc473fcbfcaef3edb3356fb25eb0e59042cd82", + "size": 18670 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "3c450752b0b92c8b3ef7ad7f5929d293375e753a979497ea5fd4c733ea464318", + "size": 14958 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "9e89bb7d349b7def169401d900e453abade0db4fc97a4e6c10dc218583a07494", + "size": 15012 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "71ebe5d12eed2104ffd8b40434dd40bc012ab316772d601a8ccdda3b9f9a764c", + "size": 18361 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "ac077e401b6778859800f4b12ed000d6de676e24a44d875cb188de2bd3838d4b", + "size": 14660 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "e703174d76a1b670466ba06904db6b08a26769d42ac974274f61423ff0778fd6", + "size": 12355 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/summary.json": { + "sha256": "25b562f30fb0233852a0a6c47a7ed62dfd9c9c810b53294fdf8ef4ea0780d5f8", + "size": 26296 + }, + "jobs/pi-dev3-rerun-install-pair-02-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/config.json": { + "sha256": "500de12acdb8d51dea706c01bd4acbfa93e4c3135fbfc3a1fc84a78083b996cb", + "size": 1719 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/job.log": { + "sha256": "7a2b3924a7637694d44c633645437966ad3d5fa9222ce22f69fb92f06ae88856", + "size": 7111 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/lock.json": { + "sha256": "27c6178952a18a7f74516ab2e712fe8502cbeeeab3d4b7915df82693e7791170", + "size": 3406 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/result.json": { + "sha256": "c6d1d687fb175021b93737daca27bc993f7fd8372886fc93b197ef3f78edcb4a", + "size": 933 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/agent/bootstrap.json": { + "sha256": "4b425000886e200a143eb692d2f097823af70a182b1a0f37777793c25db5f929", + "size": 23275 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/config.json": { + "sha256": "f5330f3a32761cd0db2c71d7f48ed22b7ebeb6e6def4d7cfabbdda3c162083d8", + "size": 1693 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/lock.json": { + "sha256": "c44510b853c71780b171ed7a0cc47eddd3a092907839ffa5f052ee25947d4055", + "size": 2244 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/result.json": { + "sha256": "504353590714162f38255370a9c8a47a20d8c713d8ffc53874b95dd795e9a259", + "size": 4603 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/trial.log": { + "sha256": "7a2b3924a7637694d44c633645437966ad3d5fa9222ce22f69fb92f06ae88856", + "size": 7111 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/config.json": { + "sha256": "84d682ec4efc8182f887a80d3e7d77603205b6a0d4f3b406e815dd32c22e11ef", + "size": 3203 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/harbor.log": { + "sha256": "e35004b65a70433e1c09210c516fd706f9490c4c86cebd1b59ef60b460121695", + "size": 936 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/manifest.json": { + "sha256": "cec502cce51d7378042e4f4efecaf20989211fbadf2d6b44781ba2d0bb585477", + "size": 37129 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "c3778a241a311f815bb89179a12067a8d6939696f8a4acc1c5d6038f58c60f99", + "size": 554 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "f6ae363159746b5c87af893f00db36426506dd66e20947aa4eb4adf043734c46", + "size": 18673 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "632ae6bc91ecfd810b6d9f043cbd554cd283a50f04db317a36a0b3503b63628d", + "size": 14961 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "b2d26af4e9e859f2ddd4ba8193a75389df782fb8200663e2e8e7f1902b5510f7", + "size": 15015 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "c63c9677c02418bbf99db2730c923756ca47433dd42e45c180faf18aaf466d4e", + "size": 18428 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "81828bedc47bb85886fb56bc2ab802c257be95539dd6c5a3cb422cf0301ce9d6", + "size": 14663 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "1c6005b6b5c4a14c8032a5bfc8541c96a4a4ca5bed524fde86bf517d34e50e69", + "size": 12416 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/summary.json": { + "sha256": "fb6fc1b26c6f9f1d4abc17f2893ed18083a9e339d1d85d6ddcc9a295fc3ed0bc", + "size": 26404 + }, + "jobs/pi-dev3-rerun-install-pair-02-v2-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/config.json": { + "sha256": "8d31e1a4c6d6b48adcaafe59fe4a47f6e7cbbcb978efc721c50af2e0b4fcd37a", + "size": 1690 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/job.log": { + "sha256": "ab09d9b1327a1c6e5cbc072e79566cc39e4810849fb1d7e01f471b1d883184ef", + "size": 15796 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/lock.json": { + "sha256": "dc061a3b9f4702ff111fa812e3e2041bfb01f494bfc2754bd32c7c7df88e4584", + "size": 3419 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/result.json": { + "sha256": "b1694b6971bd2206bda1fab38496d6cdb0c03f4d20d734b1708b49c14d34309f", + "size": 1143 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/bootstrap.json": { + "sha256": "4b425000886e200a143eb692d2f097823af70a182b1a0f37777793c25db5f929", + "size": 23275 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/pi/sessions/2026-11-02T14-52-08-081Z_01a19cbf-0a11-7754-a16f-cdffd52c16fc.jsonl": { + "sha256": "d61b0bc6ee7c03cde25b3b9978bf6ee543c6a71e283441308b59fa8e0116aac9", + "size": 391769 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/pi.txt": { + "sha256": "0b2f7053831bd965f36c813d1c8acaeb19e01df10f5b07fc839b6c9e0ba09e80", + "size": 2844992 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/config.json": { + "sha256": "c4f77f3c737e440264c5cf5b018c3235440a878636306d51d950549e2e3fe2cc", + "size": 1664 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/lock.json": { + "sha256": "ddabac11af822e9b729539d9d65ae1c29b7288096080a791cfb80c0a13f923f6", + "size": 2257 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/result.json": { + "sha256": "97870897ea64b90ae7839eec9f63f8dd8e2b52be61ecae09fbf1c9d5f4fd3fe4", + "size": 38681 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/trial.log": { + "sha256": "ab09d9b1327a1c6e5cbc072e79566cc39e4810849fb1d7e01f471b1d883184ef", + "size": 15796 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/attempts/1/ctrf.json": { + "sha256": "08934078df58a3fea7c1ade8e0cf1f746bbd5266409974d11573a746dd2f946b", + "size": 1536 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/attempts/1/test-stdout.txt": { + "sha256": "e1d21c0e930fccc821c5ab29f4ef4ed58c64a6c92236977c8832870c6fac25f6", + "size": 1385 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/ctrf.json": { + "sha256": "08934078df58a3fea7c1ade8e0cf1f746bbd5266409974d11573a746dd2f946b", + "size": 1536 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/retry-summary.json": { + "sha256": "0e209faeb095380dededf11d004c7636b941d524cdee0dc937f248e80a25a36c", + "size": 269 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/test-stdout.txt": { + "sha256": "e1d21c0e930fccc821c5ab29f4ef4ed58c64a6c92236977c8832870c6fac25f6", + "size": 1385 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/config.json": { + "sha256": "37134a513a5412aab8cac41fb0865ae1421dd8e2f3b483c4eb0942a6d52d78a9", + "size": 3226 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/harbor.log": { + "sha256": "b283bca2216170f0b087910dc51e86db6297f24ad122d65bf0a9463156b0fb19", + "size": 1176 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/manifest.json": { + "sha256": "a21646caf7fac999f1f230fd26f519de1cde8cd0113193813ed03acfe04301e0", + "size": 37129 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__init__.py": { + "sha256": "d25ed286e8a9f9f78ca9980ef041e12b4a12d0a274ab60984c633fd7e2e5ef1a", + "size": 315 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__main__.py": { + "sha256": "e322e82a6b7ac37a9dc3429c2752ad4d212a82acefa4f2bc0072b2c87e82afa4", + "size": 15679 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "986eaceb1dd56fdbba59b2a66712923261bb7fe2b3a3b3127a3c83627cc58597", + "size": 565 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "4ba181168f32ac68b5a25f3b389d1a620c9448b1c971c98b565521eccbd404fa", + "size": 18684 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "118237afdcfbf5dd587f4773021a9a7347624cf56eb88747f975075632558a66", + "size": 14972 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__pycache__/deepseek.cpython-313.pyc": { + "sha256": "cf31cdf736f4144a096280d8af72f9f19fde8beffa45decaf40b6494b3c59b65", + "size": 15026 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "0894fad8553368ade79ced48e040dec17ca0f8367de026a030aa410a3240371b", + "size": 18439 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "b1ac8747d0238cd79ee9b285cf3d9b347b935290c952a09ce9c70ec15f5afe9e", + "size": 14674 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/_dsh_client.py": { + "sha256": "4c49c2bc682993044117ac36be0b17164c6f1122e4598f4245c21afc061d6780", + "size": 2140 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "1733a2681bd9d13a30fcc1ab5317df1025d28d2d082b8cc9de85c0d9d95e723c", + "size": 11487 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/bootstrap.py": { + "sha256": "5d7f086e5eb92301e4e43f5473b8f4bcdc012f6950bc65574ee3ff07d66ef7b8", + "size": 11246 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/deepseek.py": { + "sha256": "e76e130701744a02f26ccdc3da5b41ac98c491044025bd506b5fd52901b55b86", + "size": 9697 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/pi.py": { + "sha256": "1c6005b6b5c4a14c8032a5bfc8541c96a4a4ca5bed524fde86bf517d34e50e69", + "size": 12416 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/retained.py": { + "sha256": "2f1bc92128ade805e9fa6cbf02bb8aae3da60486423785f422152f837b57e36d", + "size": 10584 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/source/harbor_adapter/verifier.py": { + "sha256": "772c036a60002ee140ce37a87f7a859923ee307d9ad229f4349e2d44be127767", + "size": 10016 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/summary.json": { + "sha256": "e7c61176a3e1d4c72d3bdff356e8d2df3e2f6ea41eabebbec47154384e6ee1a8", + "size": 26768 + }, + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + } + }, + "original_freeze_verification": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json", + "manifest_sha256": "76b775d898bbb84b21f80eee1061806d7d949465f9eef60dcab6dedeb326734b", + "files_verified": 1449, + "missing": 0, + "changed": 0 + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-distribution.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-distribution.json new file mode 100644 index 0000000..bc28496 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-distribution.json @@ -0,0 +1,852 @@ +{ + "version": "1.4.0", + "source": "Existing local upstream binary distribution; no rebuild or source modification. Its registered file hashes pin the local bytes; an upstream archive checksum has not been independently verified.", + "original_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "files": { + "CHANGELOG.md": { + "sha256": "b0eac55f99b20254f81cf925a52f054ec87c1320454a3fd7ee624683e585add6", + "mode": 420 + }, + "README.md": { + "sha256": "fbc7abec7d0829ac8bc4b217e950794e1ba4d8d1b3e0850f970da318ad72e863", + "mode": 420 + }, + "assets/clankolas.png": { + "sha256": "169acd0dfe6fbb8d8742ed24a3fc654fd0b2e2d4223c733249c5493723f1b72d", + "mode": 420 + }, + "docs/cli-integration.md": { + "sha256": "edf8e723a70b4c3c40275db0391219e55d30597936212a9cfdef33963dd37f93", + "mode": 420 + }, + "docs/cli.md": { + "sha256": "c45e59f31574de71770a71c9a7c9b6ac03bf9cad4928aa31bb6614d661733679", + "mode": 420 + }, + "docs/codemode.md": { + "sha256": "3a96ee10da36395bbd22beb60ac029f8ca8a859f2fd3c3104cfc38f9d5a1f95a", + "mode": 420 + }, + "docs/compaction.md": { + "sha256": "b5b23d10bb86d8ab32b67f80a7e57febd953222c6da6fd422ae91d232ef28faf", + "mode": 420 + }, + "docs/configuration.md": { + "sha256": "fc48d1ce7d885c5253222415fb7a4b46d75f80e6431eb924174657ff775099eb", + "mode": 420 + }, + "docs/containerization.md": { + "sha256": "e309fcb9a0a7592c0dba000ef8ee90f2d7ef1e01e6a1ac2f0cd9760dd63dee0c", + "mode": 420 + }, + "docs/custom-provider.md": { + "sha256": "fe926ed6194756c85ab6b56b7bfd12e81abe385acf39e0bb2a470eb6c70b8c59", + "mode": 420 + }, + "docs/docs.json": { + "sha256": "fc0bccd6bf66716ad1067d4fbf7258f674d89c89a12cb0b2587a027b9d7fdc49", + "mode": 420 + }, + "docs/environment-variables.md": { + "sha256": "097f2101e285f51f509be0b176d458825b83b929a635ae29d2a43703168170dd", + "mode": 420 + }, + "docs/extensions.md": { + "sha256": "46fc0baeb00db8ca8db399e33e0f506e86e425693b291258357e8063e2f3353c", + "mode": 420 + }, + "docs/how-pi-works.md": { + "sha256": "7d43ab8e2b886cd1047dcee503588a1c828b92333dd4f177526b89e6db8aaa8b", + "mode": 420 + }, + "docs/images/doom-extension.png": { + "sha256": "c3b89b5e1fba711a5bcfa188e13d8695a84f7cdda12d9e127bb45b36d13cbb27", + "mode": 420 + }, + "docs/images/exy.png": { + "sha256": "a8f253347440aed01ff659f41f573d340bcfd6010d829726ae92fe18abf601de", + "mode": 420 + }, + "docs/images/interactive-mode.png": { + "sha256": "4078001e39978151bd66b1d0a20123e7ca2ec2d1b572dfd70f552be5e88c0692", + "mode": 420 + }, + "docs/images/tree-view.png": { + "sha256": "ca61e7bbf300dfb3840cc191dc66dc4e9f5166b2f1f4089e6fb13fd04a2c3c66", + "mode": 420 + }, + "docs/index.md": { + "sha256": "c107426e88eac570e9ecbea325cb45c2a6b778a324fd6e424bd032380a65614d", + "mode": 420 + }, + "docs/json.md": { + "sha256": "f103191816fdcf3627dfc98b89acad8a28eee737697130ddaeb2581131213543", + "mode": 420 + }, + "docs/keybindings.md": { + "sha256": "413aa1c5e95a934d216aa5aba20052fa2cf35940ba5f02d1f4b22ff53c6ec678", + "mode": 420 + }, + "docs/llama-cpp.md": { + "sha256": "dccba80951463657c57a831aa2da3a576b92c05455f9b73610b208c942dfc8f4", + "mode": 420 + }, + "docs/mcp.md": { + "sha256": "23b9cca35d1340e8adce6bed8b495da0ebb58931179a937db87d610a1b40b9bf", + "mode": 420 + }, + "docs/message-types.md": { + "sha256": "b832893fb762f87fdbbfd694241d484592942823791f3e81a6c4d915421ff5db", + "mode": 420 + }, + "docs/models-configuration.md": { + "sha256": "1eb60b0f1faad6f9fffd62ea257b545dbc983dbe4c5decc4803b1a241d604155", + "mode": 420 + }, + "docs/models.md": { + "sha256": "c9914f5411db5ddd158b5391ec1da76c1f65788cfaea3a90e1e1555451b3d12e", + "mode": 420 + }, + "docs/nvidia-rtx-llama-cpp.md": { + "sha256": "bcd12ebfd283d8f528cf206ee4ec74f981ab36d3604419acc434c595decbebdb", + "mode": 420 + }, + "docs/packages.md": { + "sha256": "5fe4a3009d7260fe1e2f54b64086fc3c8b4d678232bfa2dbc7e91fe588fc9ea5", + "mode": 420 + }, + "docs/prompt-templates.md": { + "sha256": "ad5113cce0d05ef0ac77b3c71f9ebe41ea68c072275f9273b9616e4924feda42", + "mode": 420 + }, + "docs/providers.md": { + "sha256": "bc8798e4cc3209e455e3dc50c6652ae73c057d40122a4a741d8b5b65a041c2f1", + "mode": 420 + }, + "docs/quickstart.md": { + "sha256": "10c69b394304f6e7d6409b6bde02b0165f5723ff4acfd49cfa8f7414f977266a", + "mode": 420 + }, + "docs/rpc-commands.md": { + "sha256": "4deeee367852f4633ef594bc079f9e9fb14640a4e3ba8a69b7c89b6f54740838", + "mode": 420 + }, + "docs/rpc-extension-ui.md": { + "sha256": "3c6f2c6edfc8b1ace6642343727aef97a7e9234c85509850921b52d6ff09a48e", + "mode": 420 + }, + "docs/rpc.md": { + "sha256": "37a03d180115460a0a275022f4aeabef1e5345458df6a9b6e8a6de18474de720", + "mode": 420 + }, + "docs/sdk.md": { + "sha256": "d9034ccbf2219a58712d8dd4fcdf47e687d77131e71dd5d119d8a71e9bd2a2e8", + "mode": 420 + }, + "docs/security.md": { + "sha256": "3b1e40a8d28b17c9eb7e496a7ba88fbe04665e3e6ae02873a58b943e325ad504", + "mode": 420 + }, + "docs/session-format.md": { + "sha256": "4f8fc42179b332bcf71fe4c85cc95d914853ff7486d68742954a45316b512a4a", + "mode": 420 + }, + "docs/sessions.md": { + "sha256": "152d6489f61a6de9349c8ceaade95d5498fbc74f21771f41ba8e87ce0e5d8936", + "mode": 420 + }, + "docs/settings.md": { + "sha256": "2915d26f91f4c46ab1b92e63cbc46b8d48888fa0e491168642a5a90f973bd814", + "mode": 420 + }, + "docs/shell-aliases.md": { + "sha256": "ebb55456870455b4da91cfd054ae2ac9b3a417e6ff48a009587c33e68ecd7807", + "mode": 420 + }, + "docs/skills.md": { + "sha256": "5b9805525b194474ef1fe5455a0917ef15ed7f710aa9bec2eee11879556469e5", + "mode": 420 + }, + "docs/slash-commands.md": { + "sha256": "d7e1db36e9186ff146a6c27de31fe169671f52cb6f7b93e1bf8d7fe76209a79c", + "mode": 420 + }, + "docs/terminal-setup.md": { + "sha256": "583381fb9dc466135a865ca406a3be299a7a8da2c9d810c5b73f401912a0a357", + "mode": 420 + }, + "docs/termux.md": { + "sha256": "1b0687157f2384385d07800ec23e6c225ef28fbf6648f55453cc7ce62f4c64ea", + "mode": 420 + }, + "docs/themes.md": { + "sha256": "6f11eb43ca6dad4c71bcfa2fc72ac13c013e3cb9383688b88923e0139b2d097c", + "mode": 420 + }, + "docs/tmux.md": { + "sha256": "9ec0abdbf74c57d51d4a93f308fb4aec4d8a3574a24cbfd9ba481d2e39f1273c", + "mode": 420 + }, + "docs/tui.md": { + "sha256": "20d3695ebd11e179a194f535f73870247544f655ebf22e42726c3be559c6d786", + "mode": 420 + }, + "docs/usage.md": { + "sha256": "6dfe7446c81d601d5ebef5769523115229f802da364fe90fcdf96e322ae9fbcd", + "mode": 420 + }, + "docs/virtual-models.md": { + "sha256": "5ab04dee3b3712b317522bb0784e28a3fef2e6b7fe1aeb1c1ca27778685a38f7", + "mode": 420 + }, + "docs/windows.md": { + "sha256": "e5a51792381a9eec859f034153c67f9ff52c36d3cff4c7a1faa0651f77f627c3", + "mode": 420 + }, + "examples/README.md": { + "sha256": "294836cfa47cd2cff1cab628533aefc5fe2d888621e43fe9f4101c09c2917a98", + "mode": 420 + }, + "examples/extensions/README.md": { + "sha256": "b51e1095d338ed8bc6cb0a40dca12da66ffd12d6f4ec7bfd57815fac84d87662", + "mode": 420 + }, + "examples/extensions/auto-commit-on-exit.ts": { + "sha256": "b89b5cb4c6f1f07b0b4291ab85a3c5f025d7d31ebb285821716502452b46af62", + "mode": 420 + }, + "examples/extensions/bash-spawn-hook.ts": { + "sha256": "c79c87653e20ac6bd9f4813aa4bb01a483b3637700fa2edda220703958532662", + "mode": 420 + }, + "examples/extensions/bookmark.ts": { + "sha256": "a71d9449d415ad75c9f3d0d13afde129b9bea2d695b889566db17ecfca256d79", + "mode": 420 + }, + "examples/extensions/border-status-editor.ts": { + "sha256": "ccbbfddd4ff402432293b7adb21f58e869f37a5d12d4da81cea10cc5fc2311cd", + "mode": 420 + }, + "examples/extensions/built-in-tool-renderer.ts": { + "sha256": "4bfd0953e3dbf96ee0cdb26989a60ec91d4850323852ef09eafe5c2efdf23c00", + "mode": 420 + }, + "examples/extensions/claude-rules.ts": { + "sha256": "8a935f14f62c1c1a3e937f5c912c2215e25aa5fa591716e23c86224ce1dc34cd", + "mode": 420 + }, + "examples/extensions/commands.ts": { + "sha256": "36716b53da169936c7e1360a4fde1e2fc0c3356a6505f177235f09c5538c6e4f", + "mode": 420 + }, + "examples/extensions/confirm-destructive.ts": { + "sha256": "55502f82109df07e922ef3ec41af1b1a5ed33d4d6d56076654ab2bc88e2e0888", + "mode": 420 + }, + "examples/extensions/custom-compaction.ts": { + "sha256": "ea797b3508fd8600526f381466e8b08bb84f3d234932a9d6ce512698527d0aa9", + "mode": 420 + }, + "examples/extensions/custom-footer.ts": { + "sha256": "1e5f17bec95a48d407fbcc092faf7e1f6021c30cee4a66ec09bc120412292cb2", + "mode": 420 + }, + "examples/extensions/custom-header.ts": { + "sha256": "5de09cf861d3965201991d6e837d6b68e8a2e80cb89279ee9177cc05d31f723d", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/index.ts": { + "sha256": "52bb733e1a25d9e175a644e6a6e17bbf61071beecf0f2903869baa9579a05ab3", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/package-lock.json": { + "sha256": "4c1ccae817de13a0d0eee610fb12ee699b3c1f97cf28cbe83ec50cab0f338a77", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/package.json": { + "sha256": "b2ceede9b12cef79f37d02c8e850132afdab1b3316c16d622c0c8343244e85c3", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/index.ts": { + "sha256": "0c4adac82eff25fef23223af77e11cd53535834fece9731009b125446e213158", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/package.json": { + "sha256": "4af767a958f402efc19fc6157c75c209edc45c8ab6103b60f13973369ba41990", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/test.ts": { + "sha256": "58dfa6fdb63521ccad551b7463ff56d4aecf735074934a8e9fc34b71e7083d00", + "mode": 420 + }, + "examples/extensions/debug-provider.ts": { + "sha256": "a915f86a26a0257537939701797b1f613072275e76bdd8a27dd941bbf519643a", + "mode": 420 + }, + "examples/extensions/dirty-repo-guard.ts": { + "sha256": "0037a69abdf4cf98181db799bef5bdee8981861315eb82ddab78126c2b0f13c4", + "mode": 420 + }, + "examples/extensions/doom-overlay/.gitignore": { + "sha256": "b26aa26693c610d46d9c8b387b28d20dc5196a73b979f5522e24dd689aab7bc8", + "mode": 420 + }, + "examples/extensions/doom-overlay/README.md": { + "sha256": "e17bc73b76c7977ffeef99120c7896ff992cf5938a60c4d7427a7b19fc180d97", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom/build/doom.js": { + "sha256": "c3147cf4881c88b4210c017c26828f52aff83f0c4db27d7dad3fe6cda9ade1fb", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom/build/doom.wasm": { + "sha256": "571d161956593508cf4ade732ae93753f00484bb526667a8676571cca14dec7d", + "mode": 493 + }, + "examples/extensions/doom-overlay/doom/build.sh": { + "sha256": "ab09c9b6dc98ea7c1fa8e74c13981ce69b698c01aec77309c5c7d40ad114d38c", + "mode": 493 + }, + "examples/extensions/doom-overlay/doom/doomgeneric_pi.c": { + "sha256": "98210909e1e7dee71cbbdba64f1db4bebd88b504e861ee36ea42093c59b251a1", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom-component.ts": { + "sha256": "d95dc47c92db0fdc5b0e0b56a277c67f277012d3f7296df745d8a93f64d5a42e", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom-engine.ts": { + "sha256": "706e43f70bd374b73928e5a44d196d1dd5c03d93979d08bdfc01d049b5083855", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom-keys.ts": { + "sha256": "e8bd99a4ce90dd44ad120c58f3b2ba7bc0b0c662bcdc4ed4994d059130be9fdd", + "mode": 420 + }, + "examples/extensions/doom-overlay/index.ts": { + "sha256": "f89818dd75c7d8970458b1dbb9c5e85fea9b86534fd740c898c1f6f9c9f9ceba", + "mode": 420 + }, + "examples/extensions/doom-overlay/wad-finder.ts": { + "sha256": "20254888262081bf96932253bfd346971deefa48d8df011fd3baa7aa2d653b7d", + "mode": 420 + }, + "examples/extensions/dynamic-resources/SKILL.md": { + "sha256": "f8f34b0d81aa791a2d56501756fe0ace21fb78ca2754025d093f41d3b0ccaf69", + "mode": 420 + }, + "examples/extensions/dynamic-resources/dynamic.json": { + "sha256": "2bf0164779b4ea3c2572f33b1e979c427a4839193998fda19bfc5cba2a4692c5", + "mode": 420 + }, + "examples/extensions/dynamic-resources/dynamic.md": { + "sha256": "e61e2c971d02d628a58f7c36b9b37bd3e636e4ad60618a7f65e2b41410c38f35", + "mode": 420 + }, + "examples/extensions/dynamic-resources/index.ts": { + "sha256": "dfbc49418fe46ebc9696fa335709df2ed0d4c4703da8c1ad0eb317d11fce8433", + "mode": 420 + }, + "examples/extensions/dynamic-tools.ts": { + "sha256": "4394098c11f98ffc26e2487acda2e09b9467ef9ff5867def7273cb0c82a2b6c0", + "mode": 420 + }, + "examples/extensions/entry-renderer.ts": { + "sha256": "e4c01fcf382972d570ebf37d044e32065eae062fe70987b1b885d75d46aa0b11", + "mode": 420 + }, + "examples/extensions/event-bus.ts": { + "sha256": "1185942b7afd83692d9e8da029e8594d702157398c693ec8b3ddb02b0780828a", + "mode": 420 + }, + "examples/extensions/file-trigger.ts": { + "sha256": "6f5fc3dc8fa387d8ca6973786c521feb94da58838b0e171a881fb6b3a5243ac0", + "mode": 420 + }, + "examples/extensions/git-checkpoint.ts": { + "sha256": "b60c85f797091a1f21ccb2a998027c6229fe995bdd62dbb1cdcc22946768f40b", + "mode": 420 + }, + "examples/extensions/git-merge-and-resolve.ts": { + "sha256": "a3c077c87544122abe83b275f3d527f60c80c87f460ea2499d66bde9b4880133", + "mode": 420 + }, + "examples/extensions/github-issue-autocomplete.ts": { + "sha256": "4646ef360e57aa98c787ba21ca2f2e8acb3db30c4db0503e64cdbea4a4c2ac65", + "mode": 420 + }, + "examples/extensions/gondolin/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/gondolin/index.ts": { + "sha256": "9a9897639de8127a2abe6bb0d00f0005cd9e9cf9d51fd15893a827e67326bf62", + "mode": 420 + }, + "examples/extensions/gondolin/package-lock.json": { + "sha256": "256d3e45b7c6d293bdfd88a3c4884d8bb38ecdbc4064c290255d2b4d193ea535", + "mode": 420 + }, + "examples/extensions/gondolin/package.json": { + "sha256": "010f1a949005f706239d788a2481c355028eb4c366b4ea16c87577e63caab905", + "mode": 420 + }, + "examples/extensions/handoff.ts": { + "sha256": "ea07b6f680357b677d0f43f0016961ee8cb1331d37ae113e9466a3acffb2e381", + "mode": 420 + }, + "examples/extensions/hello.ts": { + "sha256": "0aa4e9800c2526914d4c1edb00b2cfa9bd9dd6da5218289994cccd5f5bfa4934", + "mode": 420 + }, + "examples/extensions/hidden-thinking-label.ts": { + "sha256": "179dc3704b5c563b49a7f4577222bf9bbaa5becf98b85c75e56ab26c001c5b22", + "mode": 420 + }, + "examples/extensions/inline-bash.ts": { + "sha256": "69b9dfc34e7b1ada9afe10cda3384b0b1a6f3c3a47da5fb7d16f914399b023d4", + "mode": 420 + }, + "examples/extensions/input-transform-streaming.ts": { + "sha256": "5c251ebda6163c8279e3c6d5f0aee56421ed150255dbb954f08ca7d2280b3ad2", + "mode": 420 + }, + "examples/extensions/input-transform.ts": { + "sha256": "cf0f65d610631ca75d18aae1c8f1408139702f674cb2835f8c009b943776474b", + "mode": 420 + }, + "examples/extensions/interactive-shell.ts": { + "sha256": "36a7efb5b132e666c643129ce09c3f83f56aed9924563e6b7269e0e126611ecc", + "mode": 420 + }, + "examples/extensions/jev-router.ts": { + "sha256": "cf2c9ef4baf9233faa133e99255786b00518cb4977d068c811a11b3c96f0a2d6", + "mode": 420 + }, + "examples/extensions/mac-system-theme.ts": { + "sha256": "d7349bae999a798c71b283165e06d0142d41b0264bfdad11ddcb1ec8b62f8710", + "mode": 420 + }, + "examples/extensions/message-renderer.ts": { + "sha256": "bedc198df5215c0a758e0743224599003630741a4ff72d4327837ce309e5ec8c", + "mode": 420 + }, + "examples/extensions/minimal-mode.ts": { + "sha256": "040cbc7619b2e903888c8626e2fa55ca5e3f820535dfd1aadcb2060e156a8770", + "mode": 420 + }, + "examples/extensions/modal-editor.ts": { + "sha256": "4742283930949fad1920359540aa9d42c833d2d24eaa49564bfc08f90e328b0d", + "mode": 420 + }, + "examples/extensions/model-status.ts": { + "sha256": "b6c073c0bdf26ac17e2fed01d02b6447fd4e2763b7b6ca2f2b493574ced25247", + "mode": 420 + }, + "examples/extensions/notify.ts": { + "sha256": "70e4333e09ce00d546c116fd2e918abf7616c70b5da6e88ba8ee21a326afd483", + "mode": 420 + }, + "examples/extensions/overlay-qa-tests.ts": { + "sha256": "4b1778e9e247c9eb24fe48e13fff0272f6f78b8102b4ed085d03598c3b98773f", + "mode": 420 + }, + "examples/extensions/overlay-test.ts": { + "sha256": "801b7c2eeec57c3569187b4b4d851f2159223ec6e9772e406ec65ed728ca6db9", + "mode": 420 + }, + "examples/extensions/permission-gate.ts": { + "sha256": "e32086ba9dd9b4ba0fd308b65241d9affa0b5ecd3f90ef7ea1b9f2b30cab8cc1", + "mode": 420 + }, + "examples/extensions/pirate.ts": { + "sha256": "dd6ce684bbe7630e4a749fe133ef8b8b875adbc7e31284b17fe0b8157d51e013", + "mode": 420 + }, + "examples/extensions/plan-mode/README.md": { + "sha256": "1ff8d611b8fbafef4ca1e3dbc72136fbcc166461b69f859b5e961b15147bff39", + "mode": 420 + }, + "examples/extensions/plan-mode/index.ts": { + "sha256": "bef2bf0712aea7b7d0f73d174e24ac5309679e6cc143f7bcbf1a9e0121222af8", + "mode": 420 + }, + "examples/extensions/plan-mode/utils.ts": { + "sha256": "12ea28d57cc55e68e8d52a2f1d16df8b2e7f2749cd694d41453028002a2bdad9", + "mode": 420 + }, + "examples/extensions/preset.ts": { + "sha256": "7942b7f933f97bcc29024a87f8526969413ba6f87b0bec537648a2926a0a4e29", + "mode": 420 + }, + "examples/extensions/project-trust.ts": { + "sha256": "78781c1f2a1613cf31c02a29ff6a6abd86b60bdf34343465e46cd85c01491578", + "mode": 420 + }, + "examples/extensions/prompt-customizer.ts": { + "sha256": "429546f46795bccda4078deace5729b8230f6103807d424eaf71817c6e07da0a", + "mode": 420 + }, + "examples/extensions/protected-paths.ts": { + "sha256": "9ca66b1f3b1b9a61cc5ddc660f92cca2ea66c4264a1bdb6b00e0da5844399fe2", + "mode": 420 + }, + "examples/extensions/provider-payload.ts": { + "sha256": "e0016077798098f528b1129954c30a28809a1513924d9c2e0efda9328d503a49", + "mode": 420 + }, + "examples/extensions/qna.ts": { + "sha256": "a639a65a486fd05a42afcf14aa59dc32e505e41ada43869167ddfd789a3c7da6", + "mode": 420 + }, + "examples/extensions/question.ts": { + "sha256": "a1625d2c72a65d859154a7e66bef222d48ca9e3654a8bb910829ad350378a4c4", + "mode": 420 + }, + "examples/extensions/questionnaire.ts": { + "sha256": "3f1f719d81152ff013f5d36a97cc3976388d43f2d89bff0f19827d3443d903e6", + "mode": 420 + }, + "examples/extensions/rainbow-editor.ts": { + "sha256": "f95f17ba8a0376180bd23510c20151e83aadc74142e989f14227bf5c4196883f", + "mode": 420 + }, + "examples/extensions/reload-runtime.ts": { + "sha256": "911f26b872860a551572fa1d94d251ddd476d4ea9483b6052e9658d2784b38fc", + "mode": 420 + }, + "examples/extensions/rpc-demo.ts": { + "sha256": "3d54a396eee12eb04ea12d0ea03716a6edd3e8743502b6a529e3b8d429ae640d", + "mode": 420 + }, + "examples/extensions/sandbox/.gitignore": { + "sha256": "16d30e4462189fb14dd611bdb708c510630c576a1f35b9383e89a4352da36c97", + "mode": 420 + }, + "examples/extensions/sandbox/index.ts": { + "sha256": "b5e527aac27254002f27b8bb9de7bc62e242e39eeb7c963683096039e0aa173e", + "mode": 420 + }, + "examples/extensions/sandbox/package-lock.json": { + "sha256": "cd4e34dbd8a9d4b0efdb58becb6c4201eb2ad9e44228b9a272b6853bfb54539d", + "mode": 420 + }, + "examples/extensions/sandbox/package.json": { + "sha256": "4b4bfe0ebeecf2af747a3e023a5edc143cba9c36137d10051e1124d1c1ad9732", + "mode": 420 + }, + "examples/extensions/send-user-message.ts": { + "sha256": "9261e433c782f354530105e1c9723a19798868f72fd1f5d68af3a4624ad7e446", + "mode": 420 + }, + "examples/extensions/session-name.ts": { + "sha256": "a0b9fd94e2cc827931150cc409238b473da635ab1a1cddce633cae5937a3ae24", + "mode": 420 + }, + "examples/extensions/shutdown-command.ts": { + "sha256": "68dc7f9561aa9b1c0f39cae19c0f41d56171a99875972e5998de3caa75f96e09", + "mode": 420 + }, + "examples/extensions/snake.ts": { + "sha256": "3ef506ab52927e866fad096f83b7cbfd0eb50e1504dcabf1b1eb28a06745c937", + "mode": 420 + }, + "examples/extensions/space-invaders.ts": { + "sha256": "43281ae25bea645d44db01234f24df80e2f866771d02de0b416ab32fb3883a1f", + "mode": 420 + }, + "examples/extensions/ssh.ts": { + "sha256": "c03bdef919457013e3798d10250f34255742fd3dc79885ac3d321b34f6a1149d", + "mode": 420 + }, + "examples/extensions/status-line.ts": { + "sha256": "26e27b1158e980551f63f46e55dad055ac613779c2b8c39df4cddc11be26103d", + "mode": 420 + }, + "examples/extensions/structured-output.ts": { + "sha256": "72a3a665ce3de1419c837db6eb8a0e7a890310c6e02e4d59c454e5c9514c15ec", + "mode": 420 + }, + "examples/extensions/subagent/README.md": { + "sha256": "6a888330bf974657751e09f5d375a9039872d59d6606fc217c5105c7eca51b1a", + "mode": 420 + }, + "examples/extensions/subagent/agents/planner.md": { + "sha256": "452f21f65957753cec3b0772004209c108071be168e627fffd7191234c0ce64e", + "mode": 420 + }, + "examples/extensions/subagent/agents/reviewer.md": { + "sha256": "b5c3f13b67207e8e32de631f8cd52a373d9b956028e2f8c1fd8b737b5f239265", + "mode": 420 + }, + "examples/extensions/subagent/agents/scout.md": { + "sha256": "3b84e8e01c6e05c8dfbff1c845c0c6371f4f16350735d4d1dbd02bfa98b1d4e6", + "mode": 420 + }, + "examples/extensions/subagent/agents/worker.md": { + "sha256": "ca2b2298fcbd8bfafcfe61977dc8075001a64b9c4d8c289c3b5eaa896efe6238", + "mode": 420 + }, + "examples/extensions/subagent/agents.ts": { + "sha256": "cc85ecd33b89aa575bea3d2a9a9da7654a6305da4de02e5041aa5342a1de29b4", + "mode": 420 + }, + "examples/extensions/subagent/index.ts": { + "sha256": "1d11858d163e2a3a258af51c1923fc55ad825813acd6224462fd25ad61f5401b", + "mode": 420 + }, + "examples/extensions/subagent/prompts/implement-and-review.md": { + "sha256": "019ba4bf42d596ea03fc4564be5b2cc55a06227c95db19a4d5a5bb96697a75f9", + "mode": 420 + }, + "examples/extensions/subagent/prompts/implement.md": { + "sha256": "65462de33c97ac1372cfed270dad9af164605b05028c069dd4c013ff9792675b", + "mode": 420 + }, + "examples/extensions/subagent/prompts/scout-and-plan.md": { + "sha256": "9506399391e0d2c71826c45f26cc332175dd964465f99a8bb880e109f8ed89f4", + "mode": 420 + }, + "examples/extensions/summarize.ts": { + "sha256": "9c75e838071773d5bacce27ac8c5305160c56958018884f960c3db96b3d09368", + "mode": 420 + }, + "examples/extensions/system-prompt-header.ts": { + "sha256": "aa97f1add4fe347b3fc37cd3709950a105b0db5d6cdaf19325b58a9fbfd05223", + "mode": 420 + }, + "examples/extensions/tic-tac-toe.ts": { + "sha256": "479597f4e76ef1eae3dae71ec9d5c26978c5b9221f950e3e134078c5b532314b", + "mode": 420 + }, + "examples/extensions/timed-confirm.ts": { + "sha256": "c35304599e6f07238ef5472efe9c3f2fef2a022f801750ed792ef9061c7de10f", + "mode": 420 + }, + "examples/extensions/titlebar-spinner.ts": { + "sha256": "ec56e7edf954966b5991bcad63141be1a09d17753cdd0c59efca1b9e649dadf2", + "mode": 420 + }, + "examples/extensions/todo.ts": { + "sha256": "e46824d00217e25242c186d41837cc84ca81b23f978500323448502a9a424ee2", + "mode": 420 + }, + "examples/extensions/tool-override.ts": { + "sha256": "5d3cc796311a16132db810fcd9e437c16abd46978b723b9120734e57cc545726", + "mode": 420 + }, + "examples/extensions/tools.ts": { + "sha256": "5eaebc40cffe885fee25c95e3d8aec79cfbd595b6eac8ac5a6de6c4afec3d5d8", + "mode": 420 + }, + "examples/extensions/trigger-compact.ts": { + "sha256": "92c3b831f9f7bf01a3e0a2df11f4e404f5ac5843000b94e8340a5d78f3cb3add", + "mode": 420 + }, + "examples/extensions/truncated-tool.ts": { + "sha256": "4ed5fbeb6da53eab8d1e04721d77961b8d1b3bf5e11d1512a17fc5a3f58f9955", + "mode": 420 + }, + "examples/extensions/widget-placement.ts": { + "sha256": "ffabd33cf58efb908de2025aac4cb41daf13dbeee24b5ad8dcd0361b43eff6d7", + "mode": 420 + }, + "examples/extensions/with-deps/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/with-deps/index.ts": { + "sha256": "8d4dbc7028c891eac7f0386d25eecea3e9ac0de7fd841b5bb08137e2ec5ab443", + "mode": 420 + }, + "examples/extensions/with-deps/package-lock.json": { + "sha256": "a4863bc4e63ff31dd43a6b469835dd0057f8792d940af0a349ec35187ad03663", + "mode": 420 + }, + "examples/extensions/with-deps/package.json": { + "sha256": "1cedd74eb4891d548ff4ae0b90adb3ce73086fa21b11fe489e0d8b1ead1c5267", + "mode": 420 + }, + "examples/extensions/working-indicator.ts": { + "sha256": "e88295d765b78fd6aac0e93db92b183ee9b7103dc21577e3c5a42665dfad032c", + "mode": 420 + }, + "examples/extensions/working-message-test.ts": { + "sha256": "91ee09378f13cce142a4314bec1bfe9784e10484c33463c4fe3139c391c117a0", + "mode": 420 + }, + "examples/rpc-client.ts": { + "sha256": "b46b19c291fbcb59f8f9c539c2dbe12a43eae25e4d7528084844bbb79d27b6f7", + "mode": 420 + }, + "examples/rpc-extension-ui.ts": { + "sha256": "09bc33c28fdc9032e3752a487d659ac21c6776c6f589c5b1ae0967ef4ee5f127", + "mode": 420 + }, + "examples/sdk/01-minimal.ts": { + "sha256": "ffe5215e36f750f5ca7146cc7dfa5be36461292036d55b8db652b4d26331f351", + "mode": 420 + }, + "examples/sdk/02-custom-model.ts": { + "sha256": "dccc8dbdb1742ea00beeed87d302b8a23ac7f7e36a0b6a4c43b283c21be0ee55", + "mode": 420 + }, + "examples/sdk/03-custom-prompt.ts": { + "sha256": "d6db1f36a8e04466d9cbcc24ffb6fe0697858aa6c0e5013d5d90f12798314ef5", + "mode": 420 + }, + "examples/sdk/04-skills.ts": { + "sha256": "4cbf189bf88d14155281349d4d39cc236066d98095aae2d184d4ba7267ab5e42", + "mode": 420 + }, + "examples/sdk/05-tools.ts": { + "sha256": "5df5f9577b343d22d072d839341916536299c8191dc66c6c801a8bbddda23544", + "mode": 420 + }, + "examples/sdk/06-extensions.ts": { + "sha256": "8e01ee409200b6516e9a6a0d9f3b3f61d2ad78aaaa18bd2ef9a8f77820d149b2", + "mode": 420 + }, + "examples/sdk/07-context-files.ts": { + "sha256": "37bd18a284e961903d646e0cd1abdeda72cd968dbe4f40892e8d345a66916b36", + "mode": 420 + }, + "examples/sdk/08-prompt-templates.ts": { + "sha256": "a1b314988f1f5b2e61c80c19c79b68630a2f3c443008a200376286710a2744ed", + "mode": 420 + }, + "examples/sdk/09-api-keys-and-oauth.ts": { + "sha256": "d1d2b1bd4e57a3b2ab6f910068e8382759d12a94fbecf1332e96a9ebe4683eb4", + "mode": 420 + }, + "examples/sdk/10-settings.ts": { + "sha256": "558f82b9d0655935e6a28ec09f6c35f0f9593f8b034f25d45400c02bc77a778a", + "mode": 420 + }, + "examples/sdk/11-sessions.ts": { + "sha256": "538c7028bf6823c84be930eb6745731f2c87ba05a45df2d86d93e66f01674576", + "mode": 420 + }, + "examples/sdk/12-full-control.ts": { + "sha256": "3cf794d470ddbde4504ae965d24a23c42378d1ac86033522e46907ac05518343", + "mode": 420 + }, + "examples/sdk/13-session-runtime.ts": { + "sha256": "0e9f408720a561a62b2630eccb18ece49f9dba567dc65494757da0a91608827a", + "mode": 420 + }, + "examples/sdk/14-codemode-mcp.ts": { + "sha256": "02ae7714e22753a76b0f44033366d43529185b325b92d47bb5d1d0eb5ae34f5e", + "mode": 420 + }, + "examples/sdk/README.md": { + "sha256": "c1f9d8f58c300195d03a420222d2d5370d2884a18a4190a1d8c8ea9371c016ff", + "mode": 420 + }, + "export-html/ansi-to-html.d.ts": { + "sha256": "57d23f45aef05ba2b1a52a8e6b0e1f1ba51b55ab0b54e8d42938a438b088977a", + "mode": 420 + }, + "export-html/ansi-to-html.d.ts.map": { + "sha256": "2847dc4f48c9ec5c8cd7aeacf1a6083ab4fc44f4b7e6667547b1549215c68576", + "mode": 420 + }, + "export-html/ansi-to-html.js": { + "sha256": "44e5fd3585104e5e8cc0c7bea91a10efa7cb7e3b5bc5e2fb9090d5be4419df14", + "mode": 420 + }, + "export-html/ansi-to-html.js.map": { + "sha256": "d24d23d27ad31a21a4fb133c6e7ba2dc2c876f28656ff1254b1af3da234dc51c", + "mode": 420 + }, + "export-html/index.d.ts": { + "sha256": "cd83b5d85e893b8ca18742b28038883dd7cf06c98a203e00d74040f78a6a656d", + "mode": 420 + }, + "export-html/index.d.ts.map": { + "sha256": "1c31d2cbab7deb811274f473316e1fed48bc2a93707f2f964db86ba2f200d522", + "mode": 420 + }, + "export-html/index.js": { + "sha256": "9b7a893c46341d2557ba79dd88307797054577c8789665281d1dfb4134d347a3", + "mode": 420 + }, + "export-html/index.js.map": { + "sha256": "f2bf1920c337f74e83454665e5a1af9621cd9e67fbcca9aade7231eb624bf9a7", + "mode": 420 + }, + "export-html/template.css": { + "sha256": "8ee19851f8e583277ed396cbb76496687aa556707fdfc1f78d1118bff87c740a", + "mode": 420 + }, + "export-html/template.html": { + "sha256": "47d208235b066c856dae69c60281a3b99d164b99be9b81153d880375c9327105", + "mode": 420 + }, + "export-html/template.js": { + "sha256": "946bd9e3e8255c1ebb40f806def18007aa9397816291a7bf2ac0e4e99cec07a4", + "mode": 420 + }, + "export-html/tool-renderer.d.ts": { + "sha256": "ea982244e87c3568ad7bbb28427b83a19506136b5a498c7b496454338483ffd4", + "mode": 420 + }, + "export-html/tool-renderer.d.ts.map": { + "sha256": "2fab18f196e020f0b896b27db7cf790c233f0d0af55de04194e60b5bff137fce", + "mode": 420 + }, + "export-html/tool-renderer.js": { + "sha256": "8d3bd1f1b068996fe8f7d003bdb6623284043e1393a2db92ff45aeb9d6c33b9a", + "mode": 420 + }, + "export-html/tool-renderer.js.map": { + "sha256": "7bead30f87d1b345c70c66953a517fb51d1e1092089158ad5e92280cc97f6ea8", + "mode": 420 + }, + "export-html/vendor/highlight.min.js": { + "sha256": "837a6fa5b0c736b52bbde2b2b6190f305da3fc9ed41681db5321507057b5c846", + "mode": 420 + }, + "export-html/vendor/marked-cjk-friendly.min.js": { + "sha256": "93d2c938754cb24eef5b9ffecc268a98ef6beeb75dad614b0e874176aeeba565", + "mode": 420 + }, + "export-html/vendor/marked.min.js": { + "sha256": "438eedfcf932a414d0d0bfeea32dc365c063563b8ca713b4687fc8f8b501e5e4", + "mode": 420 + }, + "native/linux/prebuilds/linux-x64/linux-platform-x11.node": { + "sha256": "522d2550f7fa5d0d5bd204f45983f786ce2be9cf80f27f00179640d31d0689d6", + "mode": 493 + }, + "package.json": { + "sha256": "df6883685382d6a914e5797232744c26579fe7db5576ea0e9c859d018ca2d54e", + "mode": 420 + }, + "photon_rs_bg.wasm": { + "sha256": "10468181565c56004c867f3a4af96f89a0ef5a63a72f2b5fb12c1f1992a3615c", + "mode": 420 + }, + "pi": { + "sha256": "0d7611d0b86d38ec8f65dda8e8bee9c5e907c7bae381e9cfff9161b21e8087d0", + "mode": 493 + }, + "theme/dark.json": { + "sha256": "e1f96982f8cb7d2a48eca7db03c9cb282cb1e14a243680cd855638ecd6a251b6", + "mode": 420 + }, + "theme/light.json": { + "sha256": "2dee6a4be7f765e7fd7728166b7ebf8c29e4ce313f8ec6b38ae50699e3abc2f3", + "mode": 420 + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json new file mode 100644 index 0000000..32fd1b7 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json @@ -0,0 +1,42 @@ +{ + "providers": { + "deepseek": { + "api": "openai-completions", + "baseUrl": "https://api.deepseek.com", + "models": [ + { + "id": "deepseek-flash", + "name": "DeepSeek V4.1 Flash", + "reasoning": true, + "input": [ + "text", + "image" + ], + "thinkingLevelMap": { + "minimal": null, + "low": "low", + "medium": null, + "high": "high", + "max": "max" + }, + "contextWindow": 1000000, + "maxTokens": 384000, + "cost": { + "input": 0.3, + "output": 1.2, + "cacheRead": 0.006, + "cacheWrite": 0 + }, + "compat": { + "supportsStore": false, + "supportsDeveloperRole": false, + "maxTokensField": "max_tokens", + "requiresReasoningContentOnAssistantMessages": true, + "thinkingFormat": "deepseek" + } + } + ], + "apiKey": "${DEEPSEEK_API_KEY}" + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json new file mode 100644 index 0000000..fc17cbc --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json @@ -0,0 +1,50 @@ +{ + "recorded_at": "2026-11-02T14:31:07.469336+00:00", + "registration_sha256": "4b1a2df292b461e68f25818917a6060f94c9b95f808922356b7fe14d4a512dd3", + "model_calls": 0, + "official_test_executions": 0, + "trials": [ + { + "task": "terminal-bench/db-wal-recovery", + "trial": "db-wal-recovery__Lao7WXJ", + "result_path": "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/db-wal-recovery__Lao7WXJ/result.json", + "result_sha256": "9f7386437030ec248d95431cdbf4a02fabe9b67733383e2e6bceac1df8c30372", + "bootstrap_sha256": "5815e57bc6e873824175e0ed3918b4a1c95bb61e53f83dc400ef4ab5adcb3fe0", + "status": "ready", + "verifier_preflight": { + "profile": null, + "status": "not_configured" + }, + "process_supervision_probe": "passed", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + }, + { + "task": "terminal-bench/adaptive-rejection-sampler", + "trial": "adaptive-rejection-sampler__E9uvv9V", + "result_path": "jobs/pi-dev3-rerun-install-pair-01-v2-20261102/adaptive-rejection-sampler__E9uvv9V/result.json", + "result_sha256": "5fa9b42c08b9862af1940b29749e58e647a3cd6de563622e9c884462dbed7d91", + "bootstrap_sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "status": "ready", + "verifier_preflight": { + "profile": null, + "status": "not_configured" + }, + "process_supervision_probe": "passed", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "trial": "torch-pipeline-parallelism__mCgsZWg", + "result_path": "jobs/pi-dev3-rerun-install-pair-02-v2-20261102/torch-pipeline-parallelism__mCgsZWg/result.json", + "result_sha256": "504353590714162f38255370a9c8a47a20d8c713d8ffc53874b95dd795e9a259", + "bootstrap_sha256": "4b425000886e200a143eb692d2f097823af70a182b1a0f37777793c25db5f929", + "status": "ready", + "verifier_preflight": { + "profile": "torch-pipeline-parallelism-v1", + "status": "passed" + }, + "process_supervision_probe": "passed", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json new file mode 100644 index 0000000..cece82d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json @@ -0,0 +1,52 @@ +{ + "recorded_at": "2026-11-02T14:25:50.067473+00:00", + "registration_sha256": "1e95f2ed1c8638f06c87f8f92ec87215074c830e3b6a95d54c7289082ec350aa", + "outcome": "All three installation checks failed before model calls because Harbor returned absent stderr and the Pi model-listing validator called strip() on None. Preserve these setup failures; supersede the unexecuted model queue after a tested adapter fix.", + "trials": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "trial": "adaptive-rejection-sampler__stcSVVn", + "result_path": "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/result.json", + "result_sha256": "e22663dcc70df4d73a1d2035729f3df0af50d019fc865d73d010b5e813b83efe", + "bootstrap_path": "jobs/pi-dev3-rerun-install-pair-01-20261102/adaptive-rejection-sampler__stcSVVn/agent/bootstrap.json", + "bootstrap_sha256": "c38914403bd502084e8d106a6eaa84d9a413cd8000b01c99bb62f8f376bfec1c", + "stage": "agent_install", + "status": "failed", + "error_type": "AttributeError", + "model_execution": false, + "verifier_execution": false + }, + { + "task": "terminal-bench/db-wal-recovery", + "trial": "db-wal-recovery__5wSAPDN", + "result_path": "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/result.json", + "result_sha256": "e2afb6f64ec65d703bb73e2d297cfefa68eaec85f19bc4464572ce11b6b68354", + "bootstrap_path": "jobs/pi-dev3-rerun-install-pair-01-20261102/db-wal-recovery__5wSAPDN/agent/bootstrap.json", + "bootstrap_sha256": "2b6fc50d240d22bb62b95caafd54ffe8b42f91dc00d400ed3ff968df1133078a", + "stage": "agent_install", + "status": "failed", + "error_type": "AttributeError", + "model_execution": false, + "verifier_execution": false + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "trial": "torch-pipeline-parallelism__hNdqw2j", + "result_path": "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/result.json", + "result_sha256": "d1b9a45e50862d4ea1760e93a7fa86abaacbb1f78bb0f179acc1698db32d92db", + "bootstrap_path": "jobs/pi-dev3-rerun-install-pair-02-20261102/torch-pipeline-parallelism__hNdqw2j/agent/bootstrap.json", + "bootstrap_sha256": "1d6ad688edd691d586897ee2bab87fdb64e419aea117dfcf89212e5289bdb705", + "stage": "agent_install", + "status": "failed", + "error_type": "AttributeError", + "model_execution": false, + "verifier_execution": false + } + ], + "original_outputs_checked": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json", + "files": 1449, + "missing": 0, + "changed": 0 + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/selection.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/selection.json new file mode 100644 index 0000000..852c8de --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/selection.json @@ -0,0 +1,83 @@ +[ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "torch-pipeline-parallelism-v1", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } +] diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json new file mode 100644 index 0000000..2c16a76 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/torch-pipeline-parallelism.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/validation-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/validation-evidence.json new file mode 100644 index 0000000..087ded1 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/validation-evidence.json @@ -0,0 +1,13 @@ +{ + "checked_at": "2026-11-02T15:08:38.069177+00:00", + "active_containers": [], + "protected_initial_images": 6, + "protected_images_preserved": true, + "new_job_directory_mode": "0700", + "harbor_tests": { + "passed": 304, + "failed": 0, + "command": "uv run --frozen --project benchmarks/harbor pytest benchmarks/harbor/tests -q --tb=short" + }, + "notes": "The three regression cases failed before the nullable-output fix. All 304 tests passed with loopback sockets permitted. No model calls were used for these tests." +} diff --git a/benchmarks/harbor/reports/en/pi-deepseek-flash-dev3-rerun-20261102.md b/benchmarks/harbor/reports/en/pi-deepseek-flash-dev3-rerun-20261102.md new file mode 100644 index 0000000..779c17e --- /dev/null +++ b/benchmarks/harbor/reports/en/pi-deepseek-flash-dev3-rerun-20261102.md @@ -0,0 +1,153 @@ +# Pi + DeepSeek Flash: authorized three-task rerun + +**All three fresh attempts finished: two official passes and one scored failure, +with no unscored attempt. Both passes accessed public reference solutions and +tests, so neither is an independent solution.** Pipeline repeated its previous +reference exposure; WAL also accessed reference material in this rerun. + +The maintainer requested rerunning the two unscored tasks and the reference-exposed +pipeline task from the [original 34-task baseline](pi-deepseek-flash-dev34-20261024.md). +This is an outcome-selected development supplement. Preserve the original +26 official passes, six failures, and two unscored attempts, including the +original pipeline exposure; do not choose the better attempt or describe this +supplement as an independent benchmark or a complete clean 34-task pass rate. + +## Protocol and preparation + +The [v2 registration](../../configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json) +preserves unmodified Pi 1.4.0, the byte-identical public model configuration, +`deepseek/deepseek-flash`, the official OpenAI-compatible endpoint, +`high` reasoning, unset temperature, and native prompts, tools, compaction, +and request retries. The model metadata remains 1,000,000 context tokens and +384,000 maximum output tokens. The mutable alias does not establish identical +backend weights across dates. Personal Pi settings and earlier trajectories +were not supplied. No new prompt or network restriction was introduced. + +All three original task revisions, file hashes, Docker image identities, +resources, 900-second agent limits, and 900-second per-attempt verifier limits +were retained. Setup had a separate 1,800-second limit. Concurrency remained two +with pair barriers, in the original relative order. Each task received one fresh +model attempt. Only verifier timeouts could retry, at most three attempts on the +same output; actual whole-trial and verifier retries were both zero. + +The initial three installation checks failed before model calls: Harbor returned +`None` for empty stderr, and Pi's model-registration validator called `.strip()` +on it. [Initial evidence](../../configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json) +preserves those setup failures and the unexecuted original model queue. Source +`f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c` normalizes absent output streams while +still rejecting missing models and diagnostics. Three regression cases failed +before the fix; all **304 Harbor tests passed** afterward. + +All three fresh v2 installation checks then passed, including their native Pi +supervision probes. Pipeline used the existing reviewed +`torch-pipeline-parallelism-v1` dependency profile; sampler and WAL explicitly +remained `not_configured`. Installation success does not certify their verifier +dependencies. Existing verified APT, uv/Python-release, and original PyPI wheel +transport was reused from the completed Harness supplement. Wheel cache hits +were checked against current release filename, size, SHA256, and original URL. +`UV_HTTP_TIMEOUT=600` allows cold-wheel verification before serving bytes; it +does not extend task-native limits. Every model job repeated setup successfully. +See [preflight evidence](../../configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json) +and [launch evidence](../../configs/pi-deepseek-flash-dev3-rerun-20261102/launch-evidence.json). + +## Results and timeout accounting + +| Task | Original audited outcome | New raw reward | New audited outcome | Official tests passed / failed | Agent seconds | +| --- | --- | ---: | --- | ---: | ---: | +| adaptive-rejection-sampler | Unscored: invalid boundary and verifier error | 0 | Scored failure after confirmed timeout cancellation | 0 / 9 | 899.4 | +| db-wal-recovery | Unscored: unconfirmed timeout boundary | 1 | Official pass with reference exposure | 7 / 0 | 371.8 | +| torch-pipeline-parallelism | Official pass with reference exposure | 1 | Official pass with reference exposure again | 4 / 0 | 574.7 | + +Sampler spent its attempt trying to install R. Its first installation tool call +timed out after 600 seconds; a later installation was interrupted at the native +agent limit. The supervisor confirmed cancellation and stopped **11 processes** +before verification. Official pytest then ran all nine tests and failed them: +`/app/ars.R`, `Rscript`, the required test function, and sample outputs were +missing. This is an accepted official zero after a confirmed boundary, not an +unscored verifier setup error. It does not isolate algorithm quality. + +The native `AgentTimeoutError` and standard `execution_error` summary remain +unchanged. WAL and pipeline completed normally. No message timestamp was found +after the recorded agent end or at/after verifier start in any trial. All three +official verifiers completed on their first attempt, with **11 tests passed and +nine failed; zero errors or skips**. Pipeline's four parametrized pytest cases +are retained; its native CTRF summary deduplicates them to three. Agent-run tests +are not added to the official total. + +## Reference exposure and private logs + +After freezing outputs, the audit scanned **100 tool-start records** and targeted +tool results, then reviewed reference-access matches and surrounding evidence. + +- **WAL:** downloaded and read `tests/test_outputs.py` and `solution/solve.sh` + from `harbor-framework/terminal-bench-2/main/db-wal-recovery/`, together with + pristine database/WAL inputs and task metadata. It reconstructed its submitted + output from those downloaded inputs. The remote `main` was not pinned. + Retrieval call: `call_00_9Q7Xnle2bihTwhpNK4oR6311`; subsequent use: + `call_00_OGtcueFRqUeCJ2LfCm4h4955`. +- **Pipeline:** downloaded and read 178 lines of tests and a 112-line reference + solution from `harbor-framework/terminal-bench-1` at + `d28711d0da2675d0bb1d56de45ae5df6082438a3`, under + `original-tasks/torch-pipeline-parallelism/`. It ran the downloaded tests + (two passed, two failed), then changed its backward schedule before passing + official verification. It also downloaded a test copy from + `evolvent-ai/RSIBench-Data` at `4c807610243e7b481d382c5ed360c71c79a22f61`; + the recorded diff was empty. Initial retrieval calls: + `call_00_TxnCAiv8NWaprmymZHMu0466` and `call_01_BpZq8uCPQVNhoYzX5AWW7419`. + +Complete downloaded test and solution bytes reconstructed from tool output match +the registered task hashes in both cases: + +| Task / file | Bytes | SHA256 | +| --- | ---: | --- | +| WAL tests | 4263 | `88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49` | +| WAL solution | 3080 | `ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8` | +| Pipeline tests | 6065 | `23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f` | +| Pipeline solution | 3681 | `787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70` | + +WAL and pipeline also listed `/logs/verifier` while it was empty; those listings +did not expose official tests. No public reference retrieval was observed in +sampler's six tool starts. This targeted audit does not prove that every other +exposure or exfiltration route was absent. + +WAL printed the provider credential through `env` in +`call_00_Qf8Dvmzoyjpo3B5FqM636065`. An exact scan of all 326 frozen files found +the key in **two raw WAL files**: its native session and `agent/pi.txt`. +No credential value or fingerprint, raw trajectory, or downloaded reference +contents are published. New job directories have mode `0700`. The exposed +credential should be rotated; this run did not rotate it. + +## Evidence, time, and usage + +All **326 files** were frozen at **2026-11-02 15:05:23.352293 UTC**, before new +trajectory or official-test-output review. All **1,449 original frozen files** +were rechecked and remain unchanged. No containers remain running, and all six +initially protected image identities remain present. See the +[freeze manifest](../../configs/pi-deepseek-flash-dev3-rerun-20261102/outputs-freeze.json), +[structured results](../../results/pi-deepseek-flash-dev3-rerun-20261102.json), and +[validation evidence](../../configs/pi-deepseek-flash-dev3-rerun-20261102/validation-evidence.json). +The structured results preserve trial IDs, raw exceptions/rewards, audit flags, +timestamps, original outcomes, and evidence hashes. + +The formal queue ran from **2026-11-02 14:31:11.398775 UTC** through +**15:05:11.078689 UTC**: **2,039.679914 seconds**, approximately 34.0 minutes, +including setup, verification, and pair barriers. Agent execution totaled +**1,845.967929 seconds**. Known usage totals are **3,076,743 input tokens**, +including **2,930,560 cached tokens**, and **58,536 output tokens**. +Registered-rate estimated cost is **$0.131681460**. These are partial subtotals: +sampler was interrupted, and pipeline had one failed assistant stream with +missing usage before native recovery. Costs are estimates, not reconciled billing. + +```text +adapter source: f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c +Pi distribution SHA256: 60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9 +model config SHA256: 8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299 +registration SHA256: 4b1a2df292b461e68f25818917a6060f94c9b95f808922356b7fe14d4a512dd3 +freeze SHA256: 40e22638769ccafe697ac09ca7707cde94b5181210b0be342c6c82f5bac9bc0d +results SHA256: d0c77c75f2a7d4eddd6c12dc5dda5519cea95559e456626e7d4f06d85de61919 +``` + +Launch commands and artifact locations are retained in the +[configuration README](../../configs/pi-deepseek-flash-dev3-rerun-20261102/README.md). +All jobs used the standard repository entry point; no per-job adapter or +bootstrap replacement was introduced. The other 55 task packages remained unopened. diff --git a/benchmarks/harbor/reports/zh-CN/pi-deepseek-flash-dev3-rerun-20261102.md b/benchmarks/harbor/reports/zh-CN/pi-deepseek-flash-dev3-rerun-20261102.md new file mode 100644 index 0000000..da795ba --- /dev/null +++ b/benchmarks/harbor/reports/zh-CN/pi-deepseek-flash-dev3-rerun-20261102.md @@ -0,0 +1,138 @@ +# Pi + DeepSeek Flash:经授权的三题重跑 + +**三次全新尝试均已结束:两题官方通过、一题计分失败,没有不评分的尝试。 +两次通过均接触了公开参考解法和测试,因此都不属于独立解题。** Pipeline +再次发生了上一轮的参考答案接触;WAL 在本次重跑中也访问了参考材料。 + +维护者要求重跑[原始 34 题 baseline](pi-deepseek-flash-dev34-20261024.md)中 +两道不评分的题目,以及接触过参考答案的 pipeline 题。这是根据已有结果选择的 +开发集补充实验。原始的 26 次官方通过、六次失败、两次不评分及 pipeline 的 +原始答案接触标记均保留;不选取两次尝试中的较好分数,也不将本次补充实验称为 +独立 benchmark 或完整、无答案接触的 34 题通过率。 + +## 协议与准备 + +[v2 登记](../../configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json) +保留未修改的 Pi 1.4.0、字节完全一致的公开模型配置、 +`deepseek/deepseek-flash`、官方 OpenAI 兼容端点、`high` 推理、未设置的 +temperature,以及原生 prompt、工具、上下文压缩和请求重试。模型元数据仍为 +1,000,000 token 上下文和 384,000 token 最大输出。可变别名不能证明跨日期的 +后端权重相同。没有向 agent 提供个人 Pi 设置或先前轨迹,也没有新增提示词或 +网络访问限制。 + +三题的原始任务 revision、文件哈希、Docker 镜像身份、资源、900 秒 agent +时限及每次 900 秒 verifier 时限全部保留。安装使用独立的 1,800 秒时限。 +并发仍为二,按原有相对顺序执行,每对任务之间设置屏障。每题进行一次全新的 +模型尝试。只有 verifier 超时可以对同一输出重试,最多三次;实际整题重试和 +verifier 重试均为零。 + +最初的三次安装检查均在模型调用前失败:Harbor 将空 stderr 返回为 `None`, +Pi 的模型注册校验器对其调用了 `.strip()`。 +[初始证据](../../configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json) +保留这些安装失败及未执行的原始模型队列。源码 +`f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c` 对缺失输出流进行归一化,仍然拒绝 +缺失模型和诊断信息。三个回归用例在修复前失败;修复后 **304 项 Harbor 测试 +全部通过**。 + +随后三次全新的 v2 安装检查全部通过,包括原生 Pi 进程托管探针。Pipeline +使用现有、已审阅的 `torch-pipeline-parallelism-v1` 依赖 profile;sampler +和 WAL 明确保留为 `not_configured`。安装成功不能证明它们的 verifier 依赖 +已通过预检。本轮复用已完成的 Harness 补充实验中的经核验 APT、uv/Python +发行包和原始 PyPI wheel 传输。每次 wheel 缓存命中均根据当前发行元数据核对 +文件名、大小、SHA256 和原始 URL。`UV_HTTP_TIMEOUT=600` 允许冷 wheel 在 +返回字节前完成校验,不延长任务原生时限。每个正式模型任务都重新执行安装并 +成功完成。参见[预检证据](../../configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json) +和[启动证据](../../configs/pi-deepseek-flash-dev3-rerun-20261102/launch-evidence.json)。 + +## 结果与超时记账 + +| 任务 | 原始审计结论 | 新原始 reward | 新审计结论 | 官方测试通过 / 失败 | Agent 秒数 | +| --- | --- | ---: | --- | ---: | ---: | +| adaptive-rejection-sampler | 不评分:边界无效及 verifier 错误 | 0 | 超时后确认停止,计分失败 | 0 / 9 | 899.4 | +| db-wal-recovery | 不评分:超时边界未确认 | 1 | 官方通过,但接触参考答案 | 7 / 0 | 371.8 | +| torch-pipeline-parallelism | 官方通过,但接触参考答案 | 1 | 官方通过,但再次接触参考答案 | 4 / 0 | 574.7 | + +Sampler 的尝试时间用于安装 R。第一次安装工具调用在 600 秒后超时,之后的 +安装在原生 agent 时限处被中断。进程托管器在验证前确认取消,并停止了 +**11 个进程**。官方 pytest 随后执行了全部九项测试,全部失败:缺少 +`/app/ars.R`、`Rscript`、必需的测试函数和采样输出。这是在确认边界后采纳的 +官方零分,不属于不评分的 verifier 安装错误;它不能单独反映算法能力。 + +原生 `AgentTimeoutError` 和标准 `execution_error` 摘要保持不变。WAL 和 +pipeline 正常结束。三次尝试中均未发现晚于所记录 agent 结束时间,或不早于 +verifier 启动时间的消息时间戳。三个官方 verifier 均在第一次尝试完成,合计 +**11 项测试通过、九项失败,错误和跳过均为零**。保留 pipeline 的四个参数化 +pytest 用例;其原生 CTRF 摘要将它们去重为三个。Agent 自行运行的测试不计入 +官方测试总数。 + +## 参考答案接触与私有日志 + +输出冻结后,审计扫描了 **100 条工具启动记录**和针对性的工具结果,并审阅了 +参考材料访问命中及其上下文证据。 + +- **WAL:** 从 `harbor-framework/terminal-bench-2/main/db-wal-recovery/` + 下载并读取 `tests/test_outputs.py` 和 `solution/solve.sh`,同时下载了原始 + 数据库/WAL 输入及任务元数据。它使用这些下载的输入重新构造提交输出。 + 远端 `main` 未固定 revision。下载调用为 + `call_00_9Q7Xnle2bihTwhpNK4oR6311`;后续使用调用为 + `call_00_OGtcueFRqUeCJ2LfCm4h4955`。 +- **Pipeline:** 从 `harbor-framework/terminal-bench-1` 的 + `d28711d0da2675d0bb1d56de45ae5df6082438a3` revision 下 + `original-tasks/torch-pipeline-parallelism/` 路径下载并读取了 178 行测试和 + 112 行参考解法。它运行下载的测试,得到两项通过、两项失败,随后修改了反向 + 调度,最终通过官方验证。它还从 `evolvent-ai/RSIBench-Data` 的 + `4c807610243e7b481d382c5ed360c71c79a22f61` revision 下载了另一份测试, + 记录中的 diff 为空。最初的下载调用为 + `call_00_TxnCAiv8NWaprmymZHMu0466` 和 `call_01_BpZq8uCPQVNhoYzX5AWW7419`。 + +根据工具输出还原的完整测试和解法字节,在两题中都与登记任务哈希一致: + +| 任务 / 文件 | 字节数 | SHA256 | +| --- | ---: | --- | +| WAL 测试 | 4263 | `88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49` | +| WAL 解法 | 3080 | `ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8` | +| Pipeline 测试 | 6065 | `23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f` | +| Pipeline 解法 | 3681 | `787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70` | + +WAL 和 pipeline 还曾列出当时为空的 `/logs/verifier`;这些目录列表未暴露 +官方测试。Sampler 的六次工具启动中未观察到公开参考材料下载。这种针对性审计 +不能证明所有其他答案接触或数据外传途径都不存在。 + +WAL 在 `call_00_Qf8Dvmzoyjpo3B5FqM636065` 中通过 `env` 打印了 provider +凭据。对全部 326 个冻结文件的精确扫描发现,该 key 出现在 **两份 WAL 原始 +文件**中:原生 session 和 `agent/pi.txt`。不公开凭据值或指纹、原始轨迹和 +下载的参考内容。新任务目录权限为 `0700`。应轮换已暴露的凭据;本轮没有执行 +轮换。 + +## 证据、时间与用量 + +全部 **326 个文件**于 **2026-11-02 15:05:23.352293 UTC** 完成冻结,早于 +新的轨迹或官方测试输出审阅。原始 **1,449 个冻结文件**经重新核验均未变化。 +当前没有运行中的容器,最初受保护的六个镜像身份均仍存在。参见 +[冻结清单](../../configs/pi-deepseek-flash-dev3-rerun-20261102/outputs-freeze.json)、 +[结构化结果](../../results/pi-deepseek-flash-dev3-rerun-20261102.json)和 +[验证证据](../../configs/pi-deepseek-flash-dev3-rerun-20261102/validation-evidence.json)。 +结构化结果保留 trial ID、原始异常/reward、审计标记、时间戳、原始结论和证据 +哈希。 + +正式队列从 **2026-11-02 14:31:11.398775 UTC** 运行至 +**15:05:11.078689 UTC**,共 **2,039.679914 秒**,约 34.0 分钟,包含安装、 +验证及成对屏障。Agent 执行累计 **1,845.967929 秒**。已知用量为 +**3,076,743 输入 token**,其中 **2,930,560 为缓存 token**,以及 +**58,536 输出 token**。按登记费率估算的费用为 **$0.131681460**。 +这些都是不完整的小计:sampler 被中断;pipeline 有一次失败的 assistant +流缺少用量记录,随后由原生行为恢复。费用是估算值,不是账单对账结果。 + +```text +adapter source: f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c +Pi distribution SHA256: 60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9 +model config SHA256: 8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299 +registration SHA256: 4b1a2df292b461e68f25818917a6060f94c9b95f808922356b7fe14d4a512dd3 +freeze SHA256: 40e22638769ccafe697ac09ca7707cde94b5181210b0be342c6c82f5bac9bc0d +results SHA256: d0c77c75f2a7d4eddd6c12dc5dda5519cea95559e456626e7d4f06d85de61919 +``` + +启动命令和产物位置保存在 +[配置 README](../../configs/pi-deepseek-flash-dev3-rerun-20261102/README.md)。 +所有任务均使用仓库标准入口,没有引入每题专用的 adapter 或 bootstrap 替代 +实现。其余 55 个任务包保持未打开状态。 diff --git a/benchmarks/harbor/results/pi-deepseek-flash-dev3-rerun-20261102.json b/benchmarks/harbor/results/pi-deepseek-flash-dev3-rerun-20261102.json new file mode 100644 index 0000000..cc2e08c --- /dev/null +++ b/benchmarks/harbor/results/pi-deepseek-flash-dev3-rerun-20261102.json @@ -0,0 +1,474 @@ +{ + "schema_version": 1, + "experiment": "pi-deepseek-flash-dev3-rerun-20261102", + "audited_at": "2026-11-02T15:10:00.995259+00:00", + "registration": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/experiment-v2.json", + "registration_sha256": "4b1a2df292b461e68f25818917a6060f94c9b95f808922356b7fe14d4a512dd3", + "frozen_at": "2026-11-02T15:05:23.352293+00:00", + "freeze_manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/outputs-freeze.json", + "freeze_sha256": "40e22638769ccafe697ac09ca7707cde94b5181210b0be342c6c82f5bac9bc0d", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "source_ref": "f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c", + "scope": "Three outcome-selected, separately authorized development reruns. Preserve the original 34 attempts; this supplement is not an independent benchmark and supplies no complete reference-unexposed 34-task pass rate.", + "summary": { + "model_attempts": 3, + "official_passes_with_accepted_boundary": 2, + "reference_exposed_official_passes": 2, + "passes_without_observed_reference_exposure": 0, + "scored_failures": 1, + "confirmed_timeout_scored_failures": 1, + "unscored_attempts": 0, + "native_agent_timeouts": 1, + "model_setup_errors": 0, + "verifier_errors": 0, + "verifier_retries": 0, + "whole_trial_retries": 0, + "initial_installation_errors": 3, + "replacement_installation_checks_passed": 3, + "pytest": { + "passed": 11, + "failed": 9, + "errors": 0, + "skipped": 0 + }, + "credential_exposure_tasks": 1, + "credential_exposure_files": 2, + "reviewed_tool_calls": 100 + }, + "timing": { + "queue_started_at": "2026-11-02T14:31:11.398775+00:00", + "queue_finished_at": "2026-11-02T15:05:11.078689+00:00", + "queue_seconds": 2039.679914, + "agent_seconds": 1845.9679290000001 + }, + "usage": { + "n_input_tokens": 3076743, + "n_cache_tokens": 2930560, + "n_output_tokens": 58536, + "cost_usd": 0.13168146, + "usage_is_partial": true, + "cost_is_partial": true, + "cost_is_estimated": true, + "note": "Known subtotals only. Sampler was interrupted; pipeline had one failed assistant stream with missing usage despite a successful native retry. Preserve registered Pi peak-rate estimates; do not treat them as reconciled charges." + }, + "accounting_policy": "Keep native rewards, AgentTimeoutError, and standard execution_error summaries unchanged. Accept the sampler zero only because cancellation was confirmed before all nine official tests ran. Keep both official ones but explicitly mark reference exposure; neither is an independent pass. No rerun or best-of selection after reviewing outcomes.", + "preflight_evidence": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-evidence.json", + "initial_preflight_evidence": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/preflight-v1-evidence.json", + "validation_evidence": "benchmarks/harbor/configs/pi-deepseek-flash-dev3-rerun-20261102/validation-evidence.json", + "exposure_audit": { + "method": "Scanned all 100 tool-start argument records and targeted tool-result records in the frozen native events, then manually reviewed reference-access matches and their surrounding results. Reconstructed downloaded test/solution bytes from complete tool outputs and matched their hashes against the registered task files. Exact credential scan covered all 326 frozen files; only two WAL raw logs matched. This targeted review does not prove the absence of every other exposure or exfiltration route.", + "empty_verifier_directory_probes": [ + "db-wal-recovery", + "torch-pipeline-parallelism" + ], + "credential_values_or_fingerprints_published": false, + "raw_artifacts_published": false + }, + "original_outputs": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json", + "manifest_sha256": "76b775d898bbb84b21f80eee1061806d7d949465f9eef60dcab6dedeb326734b", + "files_verified": 1449, + "missing": 0, + "changed": 0 + }, + "trials": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "task_ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "job_name": "pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102", + "trial": "adaptive-rejection-sampler__egMLPF4", + "native_root": "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4", + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102.json", + "source_ref": "f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c", + "original_outcome": { + "analytical_classification": "unscored_invalid_boundary_and_verifier_error", + "analytical_score": null, + "external_benchmark_material": false + }, + "native_reward": 0.0, + "native_exception_type": "AgentTimeoutError", + "standard_summary_status": "execution_error", + "standard_summary_rewards": null, + "analytical_classification": "confirmed_timeout_scored_failure", + "analytical_score": 0.0, + "passes_without_observed_reference_exposure": false, + "boundary_status": "confirmed_cancelled", + "process_supervision": { + "status": "cancelled", + "processes_stopped": true, + "stopped_processes": 11 + }, + "pytest": { + "passed": 0, + "failed": 9, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 9, + "passed": 0, + "failed": 9, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793630914.2949007, + "stop": 1793630915.0345078 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "retry_collection_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 93.544050814 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-11-02T14:31:20.888592Z", + "finished_at": "2026-11-02T14:31:23.087712Z" + }, + "agent_setup": { + "started_at": "2026-11-02T14:31:23.087756Z", + "finished_at": "2026-11-02T14:32:00.914107Z" + }, + "agent_execution": { + "started_at": "2026-11-02T14:32:00.914312Z", + "finished_at": "2026-11-02T14:47:00.336079Z" + }, + "verifier": { + "started_at": "2026-11-02T14:47:00.579364Z", + "finished_at": "2026-11-02T14:48:35.152725Z" + } + }, + "agent_seconds": 899.421767, + "usage": { + "n_input_tokens": 12060, + "n_cache_tokens": 11008, + "n_output_tokens": 580, + "cost_usd": 0.001077648, + "pi_metadata": { + "version": "1.4.0", + "usage_records": 5, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": true, + "cost_is_partial": true + } + }, + "operational_metadata": { + "event_count": 57, + "tool_calls": 6, + "malformed_event_lines": 0, + "post_agent_end_messages": 0, + "messages_after_verifier_start": 0 + }, + "reference_exposure": null, + "credential_files": [], + "diagnosis": "The agent spent its attempt trying to install R. Its first APT tool call timed out after 600 seconds; a later installation was cancelled at the native agent limit. Official pytest ran all nine tests, failing on absent ars.R, missing Rscript, missing test function, and missing sample outputs. This is an accepted official zero after confirmed cancellation, not an unscored verifier dependency setup error. It does not isolate algorithm quality.", + "evidence_sha256": { + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/result.json": "8309f93d11809990a15105b480c2b55e267e21f00173c5ec290b619d22eeb2d7", + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102-input/manifest.json": "dafde8a1ec3eb104265f64f39f201ab24f956c25abcada8ba516cd2e33dc3472", + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/pi.txt": "7aed534f73f9313b3352a9e8e9db1c5e4d550824d7ba29c2ad25bdd225c3f769", + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/agent/process-supervision.json": "da8bb50683fe264d8f3f45d20e9b46e60348c9a3e8fbb946381866e4a4b5aad0", + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/test-stdout.txt": "9bfdf17f21c887bc39661be3d0f0490154aa3e15982054b4c5e29a70e04fda82", + "jobs/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102/adaptive-rejection-sampler__egMLPF4/verifier/retry-summary.json": "8cb3d145bb09c9d920a320d2460b0d9b2811d92aaefb5589477853b1f1897ed1", + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-adaptive-rejection-sampler-v2-20261102.json": "d99a59461fb935f2dd70bb362f6f3fb8ed772ebb7044a01c676b016b9c417391" + } + }, + { + "task": "terminal-bench/db-wal-recovery", + "task_ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "job_name": "pi-dev3-rerun-db-wal-recovery-v2-20261102", + "trial": "db-wal-recovery__ERAbXW8", + "native_root": "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8", + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-db-wal-recovery-v2-20261102.json", + "source_ref": "f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c", + "original_outcome": { + "analytical_classification": "unscored_unverified_timeout_boundary", + "analytical_score": null, + "external_benchmark_material": false + }, + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "reference_exposed_official_pass", + "analytical_score": 1.0, + "passes_without_observed_reference_exposure": false, + "boundary_status": "completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 7, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 7, + "passed": 7, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793630333.6283495, + "stop": 1793630333.676173 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "retry_collection_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 63.09575602900031 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-11-02T14:31:17.077409Z", + "finished_at": "2026-11-02T14:31:20.425762Z" + }, + "agent_setup": { + "started_at": "2026-11-02T14:31:20.425824Z", + "finished_at": "2026-11-02T14:31:38.352303Z" + }, + "agent_execution": { + "started_at": "2026-11-02T14:31:38.352437Z", + "finished_at": "2026-11-02T14:37:50.197825Z" + }, + "verifier": { + "started_at": "2026-11-02T14:37:50.451064Z", + "finished_at": "2026-11-02T14:38:53.775128Z" + } + }, + "agent_seconds": 371.845388, + "usage": { + "n_input_tokens": 1486503, + "n_cache_tokens": 1397248, + "n_output_tokens": 22038, + "cost_usd": 0.061605587999999996, + "pi_metadata": { + "version": "1.4.0", + "usage_records": 28, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + } + }, + "operational_metadata": { + "event_count": 447, + "tool_calls": 42, + "malformed_event_lines": 0, + "post_agent_end_messages": 0, + "messages_after_verifier_start": 0 + }, + "reference_exposure": { + "retrieval_tool_ids": [ + "call_00_9Q7Xnle2bihTwhpNK4oR6311" + ], + "use_tool_ids": [ + "call_00_OGtcueFRqUeCJ2LfCm4h4955" + ], + "sources": [ + "https://raw.githubusercontent.com/harbor-framework/terminal-bench-2/main/db-wal-recovery/" + ], + "description": "Downloaded and read official tests and the reference solution, downloaded pristine database/WAL inputs, then reconstructed the submitted output from those downloaded inputs. The remote main branch was not revision-pinned.", + "credential_tool_ids": [ + "call_00_Qf8Dvmzoyjpo3B5FqM636065" + ], + "verifier_probe_tool_ids": [ + "call_01_hcrZpMyZyn0ix4W6cL9p7262" + ], + "retrieved_file_evidence": { + "tests/test_outputs.py": { + "sha256": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49", + "size": 4263, + "matches_registered_task": true + }, + "solution/solve.sh": { + "sha256": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "size": 3080, + "matches_registered_task": true + } + } + }, + "credential_files": [ + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/pi/sessions/2026-11-02T14-31-38-781Z_01a19cac-481d-7552-83b4-4dcfe1ccce9d.jsonl", + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/pi.txt" + ], + "diagnosis": "Downloaded and read official tests and the reference solution, downloaded pristine database/WAL inputs, then reconstructed the submitted output from those downloaded inputs. The remote main branch was not revision-pinned.", + "evidence_sha256": { + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/result.json": "5f964859806db7d27e7452613245844c3f29d98179f13414f8a2c398322d8a98", + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102-input/manifest.json": "2a68f7a0b4d824dd997238b61538d6fe105b73d962a4542f98e575a29ac0e816", + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/pi.txt": "91e7912cc49f447753487185b4888bb117a81cf64805f045de45a61bd90b98d0", + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/test-stdout.txt": "2782d10936127751b643e612a69c6d8a4db4d1cb803f88fada8faf4e638e7ae6", + "jobs/pi-dev3-rerun-db-wal-recovery-v2-20261102/db-wal-recovery__ERAbXW8/verifier/retry-summary.json": "ee2517d612b17d7c63bd35417867402f39b2e0764f5a63f476ce496a1ed0ad8c", + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-db-wal-recovery-v2-20261102.json": "6b28e26b732acc3d357124e9fed3087b12acdd7e35ed50b0b4091d8d20c6931e" + } + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "task_ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "job_name": "pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102", + "trial": "torch-pipeline-parallelism__fsj6t6S", + "native_root": "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S", + "record": "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102.json", + "source_ref": "f3a171fcbc390c6bcb0c941be6b9cdd9dc7c456c", + "original_outcome": { + "analytical_classification": "passed", + "analytical_score": 1.0, + "external_benchmark_material": true + }, + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "reference_exposed_official_pass", + "analytical_score": 1.0, + "passes_without_observed_reference_exposure": false, + "boundary_status": "completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 4, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793631777.3940585, + "stop": 1793631884.5038652 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "retry_collection_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 183.2791553040006 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-11-02T14:49:10.621305Z", + "finished_at": "2026-11-02T14:49:12.903407Z" + }, + "agent_setup": { + "started_at": "2026-11-02T14:49:12.903472Z", + "finished_at": "2026-11-02T14:52:07.681828Z" + }, + "agent_execution": { + "started_at": "2026-11-02T14:52:07.681965Z", + "finished_at": "2026-11-02T15:01:42.382739Z" + }, + "verifier": { + "started_at": "2026-11-02T15:01:42.631691Z", + "finished_at": "2026-11-02T15:04:45.635858Z" + } + }, + "agent_seconds": 574.700774, + "usage": { + "n_input_tokens": 1578180, + "n_cache_tokens": 1522304, + "n_output_tokens": 35918, + "cost_usd": 0.068998224, + "pi_metadata": { + "version": "1.4.0", + "usage_records": 34, + "missing_usage_records": 1, + "assistant_errors": [ + "error" + ], + "cost_is_estimated": true, + "usage_is_partial": true, + "cost_is_partial": true + } + }, + "operational_metadata": { + "event_count": 511, + "tool_calls": 52, + "malformed_event_lines": 0, + "post_agent_end_messages": 0, + "messages_after_verifier_start": 0 + }, + "reference_exposure": { + "retrieval_tool_ids": [ + "call_00_TxnCAiv8NWaprmymZHMu0466", + "call_01_BpZq8uCPQVNhoYzX5AWW7419", + "call_00_JhmnVfhx8iI1wolA4TAI0191" + ], + "use_tool_ids": [ + "call_00_G29Yx18i7kItmcP2UMTK3606", + "call_00_VXpOz8sIGdqiatGitAdp4246" + ], + "sources": [ + "https://raw.githubusercontent.com/harbor-framework/terminal-bench-1/d28711d0da2675d0bb1d56de45ae5df6082438a3/original-tasks/torch-pipeline-parallelism/", + "https://raw.githubusercontent.com/evolvent-ai/RSIBench-Data/4c807610243e7b481d382c5ed360c71c79a22f61/benchmarks/terminal_bench_2/datasets/full_89/torch-pipeline-parallelism/tests/test_outputs.py" + ], + "description": "Downloaded and read 178-line tests and a 112-line reference solution, ran the downloaded tests (two passed and two failed), then changed the backward schedule before the final official pass. A second downloaded test copy produced no diff.", + "credential_tool_ids": [], + "verifier_probe_tool_ids": [ + "call_00_yPW0FlSbEARhXASm3gBf9633" + ], + "retrieved_file_evidence": { + "tests/test_outputs.py": { + "sha256": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f", + "size": 6065, + "matches_registered_task": true + }, + "solution/solve.sh": { + "sha256": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "size": 3681, + "matches_registered_task": true + } + } + }, + "credential_files": [], + "diagnosis": "Downloaded and read 178-line tests and a 112-line reference solution, ran the downloaded tests (two passed and two failed), then changed the backward schedule before the final official pass. A second downloaded test copy produced no diff.", + "evidence_sha256": { + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/result.json": "97870897ea64b90ae7839eec9f63f8dd8e2b52be61ecae09fbf1c9d5f4fd3fe4", + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102-input/manifest.json": "a21646caf7fac999f1f230fd26f519de1cde8cd0113193813ed03acfe04301e0", + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/pi.txt": "0b2f7053831bd965f36c813d1c8acaeb19e01df10f5b07fc839b6c9e0ba09e80", + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/test-stdout.txt": "e1d21c0e930fccc821c5ab29f4ef4ed58c64a6c92236977c8832870c6fac25f6", + "jobs/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102/torch-pipeline-parallelism__fsj6t6S/verifier/retry-summary.json": "0e209faeb095380dededf11d004c7636b941d524cdee0dc937f248e80a25a36c", + "jobs/pi-deepseek-flash-dev3-rerun-20261102-v2-orchestration/pi-dev3-rerun-torch-pipeline-parallelism-v2-20261102.json": "9f605bca2f01b4e7773be1f5b16cb45b026cdb11de6f657f29854e2300449b5e" + } + } + ] +} diff --git a/benchmarks/harbor/src/harbor_adapter/pi.py b/benchmarks/harbor/src/harbor_adapter/pi.py index 9a42ba2..67f8e7d 100644 --- a/benchmarks/harbor/src/harbor_adapter/pi.py +++ b/benchmarks/harbor/src/harbor_adapter/pi.py @@ -52,14 +52,17 @@ def read_model_config(path: Path) -> dict: return config -def validate_model_registration(stdout: str, stderr: str, model_name: str) -> None: +def validate_model_registration( + stdout: str | None, stderr: str | None, model_name: str, +) -> None: """Pi can exit zero after rejecting models.json and using built-in models.""" requested = tuple(model_name.split("/", 1)) - rows = {tuple(line.split()[:2]) for line in stdout.splitlines()} - if stderr.strip() or requested not in rows: + rows = {tuple(line.split()[:2]) for line in (stdout or "").splitlines()} + diagnostic = (stderr or "").strip() + if diagnostic or requested not in rows: raise ValueError( f"Pi model registration failed for {model_name}: " - f"{stderr.strip() or 'requested provider/model missing from listing'}" + f"{diagnostic or 'requested provider/model missing from listing'}" ) diff --git a/benchmarks/harbor/tests/test_pi.py b/benchmarks/harbor/tests/test_pi.py index 500d9ca..01310ce 100644 --- a/benchmarks/harbor/tests/test_pi.py +++ b/benchmarks/harbor/tests/test_pi.py @@ -204,6 +204,23 @@ def test_install_rejects_model_fallback_or_missing_registration(tmp_path, inputs assert json.loads((tmp_path / "bootstrap.json").read_text())["status"] == "failed" +def test_install_accepts_absent_stderr_from_harbor(tmp_path, inputs): + environment = Environment() + environment.model_warning = None + asyncio.run(adapter(tmp_path, inputs).install(environment)) + assert json.loads((tmp_path / "bootstrap.json").read_text())["status"] == "ready" + + +@pytest.mark.parametrize("warning", [None, "Error loading models.json"]) +def test_install_rejects_absent_model_listing(tmp_path, inputs, warning): + environment = Environment() + environment.model_listing = None + environment.model_warning = warning + with pytest.raises(ValueError, match="Pi model registration failed"): + asyncio.run(adapter(tmp_path, inputs).install(environment)) + assert json.loads((tmp_path / "bootstrap.json").read_text())["status"] == "failed" + + def test_install_probes_the_requested_model_instead_of_a_fixed_id(tmp_path, inputs): models = Path(inputs["model_config_path"]) models.write_text(models.read_text().replace('deepseek-flash', 'registered-alternate'))