diff --git a/benchmarks/harbor/README.md b/benchmarks/harbor/README.md index d784f9f..eb667fa 100644 --- a/benchmarks/harbor/README.md +++ b/benchmarks/harbor/README.md @@ -95,6 +95,59 @@ when it runs; a successful preflight is not a guarantee against later outages. Full standard runs enable the timeout retry policy below by default, with zero whole-trial retries. CI checks these contracts without model calls or Docker. +### Pi agent baselines + +The same entry point also supports `harbor_adapter:PiCodeAgent`. Register a local +unmodified Pi binary distribution with `distribution_path` and +`distribution_sha256`, a public `model_config_path`, and optionally `thinking` +in the agent kwargs. The digest covers every distribution file and its mode. +The workflow saves one content-addressed distribution snapshot in the shared +cache, verifies it again before installation, and records its file manifest and +public model configuration with every job. Pi runs its native prompt, tools, +compaction, and retry behavior; no nanoPyCodeAgent wheel is installed or built. + +Currently this adapter supports the official DeepSeek OpenAI-compatible API. +The public models file must reference `${DEEPSEEK_API_KEY}`; supply the actual +key through the process environment. Personal Pi configuration, extensions, +skills, and session history are not copied into task environments. Catalog +refresh is disabled so the recorded model configuration remains reproducible. +Installation-only checks validate the actual binary and provider registration +without a model call and use the same reviewed verifier dependency profiles. +They reject model-loading diagnostics even when Pi exits zero, and require the +requested provider/model in the offline listing. Jobs with multiple Pi agents +preserve distinct public configurations in immutable job-local files; the +manifest records each `model_config_file` and its digest. Concurrent preparations +reuse a verified distribution published by another process and clean staging +directories on success or failure. +Native sessions and completed JSON events remain in `agent/pi/` and `agent/pi.txt`. +Usage includes compaction when Pi supplies it, and reported costs are estimates. +Missing usage stays diagnosed. Pi does not currently generate an ATIF trajectory. +Failed assistant streams with initialized zero usage remain unknown even if a +native retry succeeds. Known token and cost subtotals are retained, with partial +accounting flags for missing usage or cost. +On Linux, the Pi command runs under a child subreaper. Cancellation stops its +process tree, including detached tool sessions, before control returns to Harbor. +Successful runs retain intentional background services. Set +`verifier.kwargs.require_agent_quiescence=true` to require a completed or +confirmed-cancelled supervision record before any official assertion executes. +Missing or incomplete records stop grading. Installation checks exercise the +supervisor without a model call; interrupted usage/cost totals remain partial. + +The comparison runner supports Pi arms and `--phase all` for installation checks +followed by the model queue. A registered `minimum_free_disk_bytes` stops new +jobs before they start when disk space is insufficient. Existing jobs and +failed attempts are preserved. Agent-specific budget differences must be +declared in the experiment instead of assuming native agent defaults match. +For constrained Docker storage, an optional `batch_images` plan restores each +batch by registered repository digest, verifies image identities, and evicts +only image references absent from the run's initial inventory. It checks the +actual Docker storage volume and never forces image or container deletion. + +The first [Pi + DeepSeek Flash dev34 report](reports/en/pi-deepseek-flash-dev34-20261024.md) +preserves all 34 attempts, the supervision amendment, two unscored legacy +timeouts, public-reference exposure, and partial accounting. It does not claim +a complete, uncontaminated 34-task pass rate. + ### Cache original task images If Docker image pulls repeatedly stall, the optional cache utility downloads the @@ -113,14 +166,24 @@ uncompressed layer must match the original image configuration. The resulting preserves the original manifest, configuration, and compressed layers byte for byte. Docker import must retain the original repository digest. -This Linux utility supports public single-platform `linux/amd64` images with -gzip layers and SHA256 descriptors. `--load` requires Docker's containerd image -store. Use one cache writer at a time; `--workers` controls parallel range +This Linux utility supports public `linux/amd64` images with gzip layers and +SHA256 descriptors. For OCI indexes and Docker manifest lists, it retains the +unmodified parent index and the unique native image manifest, downloading only +that platform's layers. Attestations associated with the selected image retain +their original manifest, configuration, and in-toto payloads, each verified by +digest. Other platforms and their attestations remain referenced in the +original index without fetching their blobs. Docker loads the native +platform explicitly and must retain the original parent repository digest. +`--load` requires Docker's containerd image store and platform-selective imports. +Use one cache writer at a time; `--workers` controls parallel range requests (default eight, maximum sixteen). Completed blobs and partial ranges can be reused after an interrupted download, with complete blobs revalidated. Keep the manifest lock records with the run evidence. This only warms Docker's image cache: continue using the standard benchmark entry point and installation preflight, and retain any earlier environment-start failures separately. +Long image transfers refresh anonymous registry tokens before expiry and once +after HTTP 401. Interrupted transfers reuse saved ranges, then verify the full +blob digest before import. HTTP errors report status codes without signed URLs. ### Cache immutable APT indexes diff --git a/benchmarks/harbor/README.zh-CN.md b/benchmarks/harbor/README.zh-CN.md index 7ceeda3..d7b3d9c 100644 --- a/benchmarks/harbor/README.zh-CN.md +++ b/benchmarks/harbor/README.zh-CN.md @@ -76,6 +76,46 @@ uv 0.9.5、Python 3.13、pytest 8.4.1 和 pytest-json-ctrf 0.3.5,不执行题 通过不能保证后续没有网络故障。完整标准运行默认启用下述验证超时重试,整题重试 次数为零。CI 会验证这些约定,不调用模型、不启动 Docker。 +### Pi agent baseline + +同一个标准入口也支持 `harbor_adapter:PiCodeAgent`。在 agent kwargs 中用 +`distribution_path` 和 `distribution_sha256` 注册本地未修改的 Pi 二进制发行包, +用 `model_config_path` 指定公开模型配置,并可选设置 `thinking`。 +摘要覆盖发行包内每个文件及其权限。工作流会在共享缓存中保存一份按内容寻址的 +发行包快照,安装前再次校验,并为每个任务记录文件清单与公开模型配置。 +Pi 使用原生提示词、工具、上下文压缩和重试行为;不会构建或安装 nanoPyCodeAgent wheel。 + +目前该 adapter 支持 DeepSeek 官方 OpenAI 兼容 API。公开模型文件必须引用 +`${DEEPSEEK_API_KEY}`,真实密钥通过进程环境提供。个人 Pi 配置、扩展、skills +和历史会话不会复制到任务环境。禁用模型目录自动更新,以保证已记录的配置可复现。 +仅安装检查会验证实际二进制和模型注册,不调用模型,并沿用相同的已审阅 verifier +依赖 profile。原生会话及完整事件保存在 `agent/pi/` 和 `agent/pi.txt`。 +即使 Pi 返回零退出码,模型加载诊断也会导致检查失败;离线列表必须包含请求的 +provider/model。同一任务中多个 Pi agent 的不同公开配置会分别保存为不可变的 +任务本地文件,清单记录各自的 `model_config_file` 和摘要。并发准备会复用其他 +进程已发布且通过核验的分发包,成功或失败后均清理暂存目录。 +用量统计在 Pi 提供数据时包含上下文压缩;费用为估算值,缺失用量会单独标注。 +Pi 目前不生成 ATIF trajectory。 +失败的 assistant 流即使随后通过原生重试成功,其初始化的零用量仍视为未知。 +保留已知 token 和费用小计,并对缺失用量或费用设置不完整标记。 +在 Linux 上,Pi 命令由 child subreaper 托管。取消执行时,运行器会先停止整个 +进程树(包括工具创建的独立会话),再将控制权交回 Harbor。正常完成的任务仍可 +保留有意启动的后台服务。设置 `verifier.kwargs.require_agent_quiescence=true` +后,执行官方断言前必须有正常完成或已确认取消的进程托管记录;记录缺失或不完整 +时停止评分。安装检查会在不调用模型的前提下验证运行器;中断执行的用量和费用 +总计会标记为部分数据。 + +比较运行器支持 Pi arm,以及先安装检查、再执行模型队列的 `--phase all`。 +注册的 `minimum_free_disk_bytes` 会在空间不足时阻止启动新任务。 +已有任务和失败尝试均保留。各 agent 的预算差异必须在实验中说明,不能假设原生默认值相同。 +Docker 空间受限时,可以通过 `batch_images` 计划按批恢复已注册 repository digest +对应的镜像、验证镜像身份,并仅清理本轮初始清单中不存在的镜像引用。 +运行器会检查 Docker 实际存储分区,且不会强制删除镜像或容器。 + +首份 [Pi + DeepSeek Flash dev34 报告](reports/zh-CN/pi-deepseek-flash-dev34-20261024.md) +保留了全部 34 次尝试、进程托管修正、两次不评分的旧超时、公开参考材料接触和 +不完整用量统计,不宣称完整且无答案接触的 34 题通过率。 + ### 缓存原始任务镜像 如果 Docker 拉取镜像反复停滞,可用缓存工具从同一 Docker Hub 公共仓库分段并行下载: @@ -91,13 +131,22 @@ python benchmarks/harbor/scripts/cache_image.py \ 生成的 [OCI 归档](https://github.com/opencontainers/image-spec/blob/main/image-layout.md) 逐字节保留原始 manifest、配置和压缩层;Docker 导入后必须保留原始仓库摘要。 -该 Linux 工具支持使用 gzip 层和 SHA256 描述符的公共单平台 `linux/amd64` 镜像。 -`--load` 需要 Docker 的 containerd 镜像存储。同一缓存目录同时只允许一个写入进程; +该 Linux 工具支持使用 gzip 层和 SHA256 描述符的公共 `linux/amd64` 镜像。 +对于 OCI index 和 Docker manifest list,会保留未修改的父索引及唯一的本机平台 +镜像 manifest,只下载该平台的层。本机镜像关联的 attestation 会保留原始 manifest、 +配置和 in-toto payload,并逐个校验摘要。其他平台及其 attestation 仍由原始索引引用, +但不会下载它们的 blob。Docker 显式加载本机平台,并且必须保留原始父索引的仓库摘要。 +`--load` 需要 Docker 的 containerd 镜像存储及按平台导入能力。 +同一缓存目录同时只允许一个写入进程; `--workers` 控制分段请求并发数,默认 8、最多 16。下载中断后可复用已完成的对象和分段, 完整对象会重新校验。请将 manifest 锁定记录纳入实验的证据归档。 这个步骤只准备 Docker 镜像缓存;随后仍需使用标准 benchmark 入口并执行安装预检, 之前出现的环境启动失败应单独保留。 +大镜像传输会在匿名 registry 令牌到期前刷新令牌,并在收到 HTTP 401 后刷新一次。 +中断的传输会复用已保存的分段,导入前仍校验完整 blob 摘要。 +HTTP 错误仅报告状态码,不暴露带签名的 URL。 + ### 缓存不可变的 APT 索引 如果原始 APT 索引下载反复停滞,可用可选的宿主机代理,通过校验过的并行分段请求 diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/README.md b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/README.md new file mode 100644 index 0000000..afa8678 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/README.md @@ -0,0 +1,343 @@ +# Pi + DeepSeek Flash dev34 agent baseline + +**Status: all 34 model attempts finished. The audited outcomes are 26 official +passes, six scored failures, and two unscored legacy timeout attempts. One pass +followed confirmed cancellation at the time limit; another accessed public +reference tests and a solution. No complete, uncontaminated 34-task pass rate +is claimed.** See the complete [English report](../../reports/en/pi-deepseek-flash-dev34-20261024.md), +[Simplified Chinese report](../../reports/zh-CN/pi-deepseek-flash-dev34-20261024.md), +and [audited results](../../results/pi-deepseek-flash-dev34-20261024.json). After the +October 27 repository relocation, the project volume had approximately 69 GiB +free and Docker's system volume approximately 20 GiB. The active +[v7 plan](experiment-v7.json) restores and verifies the original images one +batch at a time, then removes only image references absent from the initial +Docker inventory. Existing recovery containers and their images are preserved. +Model work remains gated on successful native installation checks for all tasks. +The [original registration](experiment.json) remains unchanged and unexecuted. + +This experiment establishes the first external-agent baseline on the existing +34-task development pool. The [selection](selection.json) is byte-identical to +the October 12 selection. All task revisions and registered task-file hashes +match, and all 34 local Docker image identities and repository digests were +checked against the October 19 registration. The other 55 tasks remain unopened. + +## Frozen agent and model + +- Agent: upstream Pi **1.4.0**, using the existing local binary distribution. +- Runtime adapter checkouts: the first six attempts retain `jobs/runtime-source`, + detached at `d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67`; the remaining + 28 use `jobs/supervised-source`, detached at + `29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d`. +- [Distribution manifest](pi-distribution.json): all files and permissions, + pinned by `60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9`. + This pins the installed local bytes; an upstream release archive checksum has + not been independently checked. No Pi source or binary is modified. +- Model: `deepseek/deepseek-flash`, through the official OpenAI-compatible API at + `https://api.deepseek.com`. +- Reasoning: **high**, selected before any model work. Temperature is unset. +- [Public model configuration](pi-models.json): the local DeepSeek provider's + capabilities and compatibility settings, with credentials replaced by the + `${DEEPSEEK_API_KEY}` environment reference. No other provider is copied. + +[DeepSeek's official documentation](https://api-docs.deepseek.com/quick_start/agent_integrations/) +was checked on October 24 and maps `deepseek-flash` to +DeepSeek-V4.1-Flash-1017. This alias is mutable. The registered model metadata +declares a 1,000,000-token context and 384,000-token output ceiling. Actual +generation behavior remains Pi's native behavior. The configuration retains +peak-rate token estimates; [official pricing](https://api-docs.deepseek.com/quick_start/pricing/) +also has off-peak discounts, so these estimates are not reconciled bills. + +Pi retains its own system prompt, default tools, context compaction, and internal +request retry behavior. Personal extensions, skills, session history, settings, +and other providers are not copied. Model catalog refresh is disabled with +`--offline`; model requests still reach the official API. The adapter keeps +native sessions and completed JSON events rather than generating an ATIF trace. + +## Task, budget, and infrastructure policy + +The [active experiment plan](experiment-v7.json) registers all 34 fresh model trials, +their fixed order, configurations, source revision, distribution, infrastructure +hashes, and installation checks. Maximum concurrency is two, with consecutive +pair barriers and no whole-trial retries. Preserve every failed setup or launch +attempt; do not replace an existing model output after seeing its outcome. + +Each task keeps its original image, resource limits, native agent timeout, test +scripts, package sources, and native verifier time limit. Setup separately +allows 1,800 seconds. Only verifier timeouts may retry, at most three total +attempts against the same output. Dependency-error recovery is disabled. + +This run uses stock Pi loop and generation behavior. It does not inject Nano's +100-reply cap or its native-timeout-minus-180-seconds work budget and finalization +reminders. The historical Nano configuration requested 65,536 output tokens; +the Pi model metadata retains the local 384,000-token ceiling. Report these +differences when comparing scores: this is an agent baseline, not an experiment +that isolates harness behavior under identical internal budgets. + +Fresh installation-only checks must finish for all 34 tasks before any model +trial. QEMU has the reviewed `qemu-alpine-ssh-v1` verifier profile. The other +33 tasks must explicitly report `not_configured`; successful Pi installation +does not certify their verifier dependencies. All 34 Pi installation checks have completed successfully, including +**QEMU's verifier dependency preflight**. The other 33 verifier profiles remain +explicitly `not_configured`; their installation success does not certify +verifier dependencies. + +The [dependency transport](dependency-cache.compose.json) reuses the reviewed +loopback APT-index and original uv/Python release-cache sidecars. The +[release manifest](dependency-release-manifest.json) is unchanged. Cache use +preserves upstream filenames, sizes, SHA256 digests, and native package checks. +The worktree's ignored `.cache` link points to the repository's shared cache. +No task images, official tests, or package origins are substituted. + +## Execute + +From this experiment worktree's root, provide `DEEPSEEK_API_KEY` in the process +environment and use an isolated virtual environment. The command below first +runs all installation checks; models start only if those checks succeed. + +```bash +export UV_PROJECT_ENVIRONMENT="$PWD/jobs/harbor-supervised-runtime-venv" +uv run --frozen --project jobs/supervised-source/benchmarks/harbor \ + python benchmarks/harbor/scripts/run_comparison.py \ + --plan benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json \ + --phase all +``` + +The runner executes each job through the standard repository entry point: + +```bash +uv run --project benchmarks/harbor python -m harbor_adapter run --config +``` + +Each invocation runs from the frozen runtime checkout. Native jobs and input +manifests are stored under `jobs/runtime-source/jobs/` for the first six attempts +and `jobs/supervised-source/jobs/` for the remaining 28; orchestration state is +stored under `jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/`. Existing +records cannot be overwritten. A previous interrupted job requires inspection. +If preflight fails, preserve it and register any corrective installation check +separately before authorizing model work from the resulting evidence. + +During execution, inspect operational stages only. Freeze all 34 outputs before +reviewing new task trajectories or hidden-test content. Keep setup and verifier +errors separate from scored task failures. Once the run finishes, produce both +complete English and Simplified Chinese reports together, plus audited results. +Both reports and the results are now complete; raw trajectories remain local. + +## Original preparation evidence (before execution) + +- The Harbor workflow suite passed **187 tests** before the final usage/error + accounting changes; all **51 relevant tests** passed after those changes. +- All **35** Harbor configurations and the frozen experiment plan validated. +- The actual Pi 1.4.0 binary accepted the registered provider via + `--offline --list-models deepseek-flash`, using a dummy key and no model call. +- All **34** registered task-file sets and Docker image identities matched the + previous run. See [image evidence](image-digests.json). +- Container dependency preflight: **pending**. Model trials: **0/34 started**. + +## October 27 storage amendment + +The v2 registration splits installation into 17 consecutive two-task batches. +Before each installation or model batch, the runner restores missing images +using their original repository digests and checks their recorded identities. +It requires 15 GiB free on Docker's actual storage volume before restoration. +After each batch, it removes newly introduced image references without force; +Docker must retain images referenced by any remaining container. Raw task files, +logs, official tests, original images, and frozen outputs remain unchanged. +Initial Docker image IDs and all restore/verify/evict events are recorded in the +orchestration directory. This changes image-cache residency, not image contents. +The existing cache sidecars remain isolated and unchanged. + +The agent runtime, Pi 1.4.0 distribution, high reasoning setting, model provider, +34 task revisions, model ordering, resources, and timeout policies are unchanged. +Canonical paths in `storage-v2/` reflect the repository relocation. The isolated +virtual environment resides on the project volume. The official model alias was +rechecked on October 27 and still maps to DeepSeek-V4.1-Flash-1017. All 51 v2 job +configurations and complete task-file sets validated; 55 relevant workflow tests +passed, including image identity, preservation, eviction, and disk-space guards. + +The v2 controller was stopped during the first stalled Docker layer download, +before any native job or model call. Its launch and image-download evidence +remain local. The active v3 plan uses the existing SHA256-verified range-cache +utility to restore the same registered images, then applies the same batch +eviction policy. No task configuration, agent output, or verifier policy changed. +All 56 relevant workflow tests passed after this amendment. + +## October 28 image-transfer recovery + +V3 completed eight installation jobs covering 16 tasks, all successfully. +It then stopped while restoring QEMU's original 991,743,246-byte image layer: +253 one-MiB ranges remained saved, with no QEMU native trial or model work. +The original logs reported a registry `HTTPError` but did not preserve its +status code, so the exact historical HTTP status is unknown. A fresh-token +manifest request succeeded and matched the registered image digest. + +The image-cache utility previously held one anonymous pull token for the entire +image transfer. It now refreshes expiring tokens and retries once with fresh +credentials on HTTP 401, coordinating refreshes between range workers. Saved +ranges remain resumable; complete blobs still require full SHA256 verification. +Sanitized failures retain HTTP status codes without exposing signed URLs. +These fixes address token expiry during long transfers; the original failure's +status cannot be established retrospectively. + +The [v4 plan](experiment-v4.json) binds the 16 successful native installation +records through [resume evidence](preflight-resume-evidence.json), including +results, bootstrap records, original input configurations, source manifests, +and Pi model snapshots. The strict existing acceptance checks validated every +record as installation-only from the same frozen runtime and distribution. +The v4 orchestration directory links to the original eight completed job +records; the runner skips their image preparation and native execution. +All remaining installations are fresh. Previous transfer failures and launch +evidence remain preserved. Model work is still gated on all 34 successful +installation checks, and the agent/model/task/budget/verifier policies are unchanged. +The complete Harbor workflow suite passed **199 tests** after these changes. + +## October 29 MTEB image restoration + +V4 successfully completed 30 installation checks, including QEMU's registered +verifier dependency profile. Before starting pair 16, the range-cache utility +rejected MTEB's multi-architecture OCI index, which is outside the utility's +documented single-platform support. No native pair-16 trial, model call, or +verifier execution occurred. This is an image-transfer failure, separate from +task setup and task scores. Its original logs remain preserved. + +The original registered reference is +`alexgshaw/mteb-leaderboard@sha256:a8538f4882ff132e0a20a75be3fffc07164318a7425ad19a6b4bf5268107feb4`. +A native Docker pull reached its 600-second timeout before installing the image. +The cache utility now supports this format: it retains the original parent +index, selects exactly one Linux amd64 image manifest by digest, verifies its +size and SHA256, and downloads the original native layers through the existing +verified range path. The native configuration and compressed/uncompressed layer +digests remain unchanged. Native-platform attestations preserve their original +manifest, configuration, and in-toto payloads with digest verification. Other +platform and attestation references remain unchanged in the parent index, with +their blobs unfetched. Docker imports with +`--platform linux/amd64`, and both the utility and runner require the original +parent image/repository digest after loading. Image residency follows the same +batch-cache policy as other task images. + +An isolated real-Docker import probe using the repository cache writer confirmed +that this preserves the parent image ID and repository digest. Its temporary +image was removed. See [import evidence](oci-index-import-evidence.json), which +also records the native-pull timeout. The complete Harbor workflow suite passed +**206 tests**, including native-platform selection, parent/child preservation, +and rejection of mismatched or ambiguous manifests. + +The [v5 plan](experiment-v5.json) reuses +[30 strictly validated native installation records](preflight-resume-30-evidence.json), +with all prior results, bootstrap records, manifests, configurations, and model +snapshots bound by hash. Completed checks are never rerun. The initial protected +inventory includes the previous protected images and the images already present +when recovery resumes. The four remaining checks run fresh, then the unchanged +34-task model queue begins automatically after complete installation success. +The adapter source, Pi distribution, model settings, task revisions, resources, +budgets, and verifier policy remain unchanged; no adapter code changed here. + +## Native-attestation completeness amendment + +The initial index probe covered an absent non-native image. A subsequent +isolated four-descriptor probe matching MTEB's index shape showed that Docker's +native-platform importer also requires attestations associated with that native +image. V5 was stopped during image transfer before pair 16 or any model work. +The new cache path fetches those original attestation manifests and their config +and in-toto payloads by descriptor, verifies all sizes and SHA256 digests, and +includes them unchanged in the OCI archive. It leaves the ARM image and ARM +attestation unfetched, preserving all their original references in the index. + +A real-Docker probe of the complete native-attestation graph now succeeds and +preserves the original parent image/repository digest. Its temporary image was +removed; see [attestation import evidence](oci-index-attestation-import-evidence.json). +The [v6 plan](experiment-v6.json) preserves the same 30 validated installation +records and all earlier transfer evidence, resumes the original image bytes, +and retains the unchanged model/preflight gate. No native agent or verifier +behavior changed. + +The full Harbor workflow suite passed **210 tests** after native-attestation +completeness checks. All v6 source, configuration, task, image, and reused +installation-evidence hashes validated before launch. + +## Installation complete; model execution started + +All 34 native installation checks have completed successfully and passed strict +provenance validation; see [complete installation evidence](preflight-complete-evidence.json). +QEMU's registered verifier profile passed, and the other 33 profiles remain +`not_configured`. MTEB's original parent image/repository digest was confirmed +after the real import, and its installation check passed. + +The model gate opened automatically. The first two tasks, +`adaptive-rejection-sampler` and `build-cython-ext`, have received actual model +replies and are executing tools. Operational counters alone were inspected; +new task trajectories and official hidden-test contents remain unreviewed. +The fixed queue continues with concurrency two. All 34 model outputs must +freeze before result/trace analysis and complete bilingual reporting. + +## Timeout-boundary correction after the first six attempts + +Operational timestamps confirmed that the adaptive rejection sampler continued +producing assistant/tool events after Harbor recorded its agent timeout and +started verification. The frozen-output boundary is invalid for that attempt. +WAL recovery also timed out without a supervision acknowledgement, so its +boundary remains unverified. Both native zero rewards and all original outputs +are preserved, with these attempts separated from ordinary scored failures. +See [timeout-boundary evidence](timeout-boundary-evidence.json); only event +timestamps and lifecycle metadata were reviewed, not solution or hidden-test +content. The other four completed native results remain unchanged. + +The original scheduling controller stopped at the completed-pair boundary before +starting task seven. The corrected adapter supervises Pi and all descendants +through a Linux child subreaper, including tools that start separate sessions. +Cancellation freezes and kills that process tree and waits for confirmation +before Harbor can start its verifier. A required verifier guard refuses missing +or incomplete confirmation. Normal completion retains intentional solution +servers. Agent timeouts, model settings, prompt, tools, and Pi distribution are +unchanged; interrupted usage/cost totals are explicitly partial. + +An isolated network-disabled Docker test confirmed that a detached tool stops +writing before cancellation returns; see +[supervision probe evidence](process-supervision-probe-evidence.json). +The corrected adapter will run from a newly frozen checkout. Its 28 unstarted +tasks require fresh installation checks, including a supervisor startup probe, +before any further model work. The six existing model attempts remain under +their original source reference and are never replaced. + +The corrected supervision and grading boundary passed **225 Harbor tests**. +A real network-disabled Docker probe also confirmed that a detached tool stops +writing before cancellation returns. Earlier timeout attempts remain excluded +from ordinary scored-failure counts because their frozen boundary is invalid or +unverified; the amendment does not retroactively validate those outputs. + +The [v7 plan](experiment-v7.json) preserves the six completed model states and +all original installation evidence. It registers a separate `pi_supervised` arm +for the 28 unstarted tasks, frozen at `29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d`. +Fourteen fresh two-task installation jobs exercise that source and its +supervisor before the remaining model work. New model configurations require +`verifier.kwargs.require_agent_quiescence=true`. The original six models retain +`jobs/runtime-source`; the remaining jobs and fresh installation evidence live +under `jobs/supervised-source/jobs/`. The original task ordering, Pi/model +settings, per-task budgets, and concurrency limit remain unchanged. + +All 28 corrected-source installation checks completed successfully, including +the process-supervision startup probe. The remaining 28 model attempts then +finished without replaying or replacing any of the first six outputs. + +## Completed output freeze and audit + +The last model job finished at `2026-10-29T19:11:52.019151+00:00`. The +[output freeze](outputs-freeze.json) hashed 1,449 files at +`2026-10-30T13:54:30.827662+00:00`, before new solution/hidden-test-content +review. All frozen hashes, source snapshots, task revisions, Pi distribution +pins, public model configurations, pair barriers, and installation gates were +verified. Final v7 plan validation passed. No model trial or verifier was retried. + +The standard summaries retain 25 normal passes, six normal scored failures, +and three `AgentTimeoutError` execution errors. Caffe's corrected-source timeout +acknowledged 13 stopped processes before all six official tests passed; the +audit accepts its raw one while retaining its exception. Sampler and WAL retain +raw zeros and analytically null outcomes because their old timeout boundaries +are invalid/unverified. Sampler also suffered a verifier dependency error before +pytest; this overlaps its boundary exclusion rather than adding another task. + +Pipeline parallelism's official pass is flagged because Pi downloaded and read +public reference tests and a solution. WAL and the inference scheduler printed +the provider key into four local event/session files. Raw traces and key values +are not published; rotate the exposed key before another experiment. The full +report pair documents these findings, partial usage, and differences from the +historical Nano budget. All controllers and trial containers have exited. diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/adaptive-rejection-sampler.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/adaptive-rejection-sampler.json new file mode 100644 index 0000000..e06791a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/adaptive-rejection-sampler.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf new file mode 100644 index 0000000..e191ddb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf @@ -0,0 +1,8 @@ +// Route only public APT HTTP origins through the verified index cache. +Acquire::http::Proxy::deb.debian.org "http://127.0.0.1:18742/"; +Acquire::http::Proxy::security.debian.org "http://127.0.0.1:18742/"; +Acquire::http::Proxy::archive.debian.org "http://127.0.0.1:18742/"; +Acquire::http::Proxy::archive.ubuntu.com "http://127.0.0.1:18742/"; +Acquire::http::Proxy::security.ubuntu.com "http://127.0.0.1:18742/"; +Acquire::http::Proxy::ports.ubuntu.com "http://127.0.0.1:18742/"; +Acquire::http::Proxy::old-releases.ubuntu.com "http://127.0.0.1:18742/"; diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-cython-ext.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-cython-ext.json new file mode 100644 index 0000000..4fa595d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-cython-ext.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-pov-ray.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-pov-ray.json new file mode 100644 index 0000000..ba6f8e5 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-pov-ray.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/caffe-cifar-10.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/caffe-cifar-10.json new file mode 100644 index 0000000..c8820d9 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/caffe-cifar-10.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cancel-async-tasks.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cancel-async-tasks.json new file mode 100644 index 0000000..71d40e3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cancel-async-tasks.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/circuit-fibsqrt.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/circuit-fibsqrt.json new file mode 100644 index 0000000..f200396 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/circuit-fibsqrt.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cobol-modernization.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cobol-modernization.json new file mode 100644 index 0000000..f452d50 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cobol-modernization.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/constraints-scheduling.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/constraints-scheduling.json new file mode 100644 index 0000000..706d120 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/constraints-scheduling.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/custom-memory-heap-crash.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/custom-memory-heap-crash.json new file mode 100644 index 0000000..1ceaa2e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/custom-memory-heap-crash.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/db-wal-recovery.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/db-wal-recovery.json new file mode 100644 index 0000000..9dab4a8 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/db-wal-recovery.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json new file mode 100644 index 0000000..b6c62ff --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json @@ -0,0 +1,100 @@ +{ + "services": { + "main": { + "volumes": [ + { + "type": "bind", + "source": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf", + "target": "/etc/apt/apt.conf.d/99nanopy-index-cache", + "read_only": true + } + ], + "environment": { + "UV_INSTALLER_GITHUB_BASE_URL": "http://127.0.0.1:18743", + "UV_PYTHON_INSTALL_MIRROR": "http://127.0.0.1:18743/astral-sh/python-build-standalone/releases/download" + } + }, + "apt-index-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/apt_index_proxy.py", + "--bind", + "127.0.0.1", + "--port", + "18742", + "--cache", + "/cache", + "--workers", + "16" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "mem_limit": "256m", + "cpus": 0.5, + "volumes": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/apt_index_proxy.py:/proxy/apt_index_proxy.py:ro", + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/home/minix/projects/nanoPyCodeAgent/.cache/nanopy-harbor/apt-indexes:/cache" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18742), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + }, + "uv-release-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/uv_release_cache.py", + "serve", + "--cache", + "/cache", + "--manifest", + "/manifest.json" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "cpus": 0.5, + "mem_limit": "256m", + "volumes": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/uv_release_cache.py:/proxy/uv_release_cache.py:ro", + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/home/minix/projects/nanoPyCodeAgent/.cache/nanopy-harbor/uv-releases:/cache:ro", + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json:/manifest.json:ro" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18743), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json new file mode 100644 index 0000000..e762efb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json @@ -0,0 +1,58 @@ +{ + "registered_at": "2026-10-18T13:25:43.131474+00:00", + "metadata_source": "Original release metadata preserved by the source manifests; no versions or payloads substituted.", + "source_manifests": { + "benchmarks/harbor/configs/dev34-main-baseline-20261012/dependency-release-manifest-v5.json": "bdf3d3ef8b7743700c5294c4a511c686a3928c178a77fd4abcba90ee4557ffcc", + "python-agent-release-manifest.json": "78aecd7c397c469196f9d246a160bf1fc99ad2d51cb71320b7d26ebf839647ce" + }, + "assets": [ + { + "version": "0.9.11", + "filename": "uv-x86_64-unknown-linux-gnu.tar.gz", + "url": "https://github.com/astral-sh/uv/releases/download/0.9.11/uv-x86_64-unknown-linux-gnu.tar.gz", + "digest": "sha256:817c0722b437b4b45b9a7e0231616a09db76bab1b8d178ba7a9680c690db19f0", + "size": 21699161, + "asset_id": 318993196, + "release_metadata_sha256": "c1a1378ece65ce1aa8343e57bdbd305289a8fc6d3d9180e8bb6b0f69acd91413" + }, + { + "version": "0.9.5", + "filename": "uv-x86_64-unknown-linux-gnu.tar.gz", + "url": "https://github.com/astral-sh/uv/releases/download/0.9.5/uv-x86_64-unknown-linux-gnu.tar.gz", + "digest": "sha256:2cf10babba653310606f8b49876cfb679928669e7ddaa1fb41fb00ce73e64f66", + "size": 21370871, + "asset_id": 306970260, + "release_metadata_sha256": "1921b17d3370aafb690b29d1d121ee8719ef3d3dbc0c7cf3b948c3059a927a5d" + }, + { + "version": "20251014", + "filename": "cpython-3.13.9+20251014-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251014/cpython-3.13.9%2B20251014-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "digest": "sha256:c0ccda275948c79996e46993c2c5476ff5cca606dee530f1dea712179131b348", + "size": 33517986, + "asset_id": 304305340, + "release_metadata_sha256": "8e5db21cfcc2bcafcb2127f269f00cbfdd1f74842a685acd2a14e3c1ad6f8394", + "repository": "astral-sh/python-build-standalone" + }, + { + "version": "20251120", + "filename": "cpython-3.14.0+20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251120/cpython-3.14.0%2B20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "digest": "sha256:66f385305ae0eefd6b65e2cf942bc91d943a61e26fb7581e029f760a2b04f393", + "size": 35376131, + "asset_id": 318881850, + "release_metadata_sha256": "e3a737862d2dddf11c73bfef6c2f2579e9d4ce5bd271cca53790c998590dee58", + "repository": "astral-sh/python-build-standalone" + }, + { + "version": "20251120", + "filename": "cpython-3.13.9+20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "url": "https://github.com/astral-sh/python-build-standalone/releases/download/20251120/cpython-3.13.9%2B20251120-x86_64-unknown-linux-gnu-install_only_stripped.tar.gz", + "digest": "sha256:1bc9a94ab2425fb6ae58e45f69cab61647759b57895649701390822d976fa923", + "size": 34257108, + "asset_id": 318881608, + "release_metadata_sha256": "705690ab2fd466e90b4c868635c529f3c7fd0cb597d399e350f399e3f3e35e6c", + "repository": "astral-sh/python-build-standalone" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dna-assembly.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dna-assembly.json new file mode 100644 index 0000000..914ef6c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dna-assembly.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v2.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v2.json new file mode 100644 index 0000000..144013d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v2.json @@ -0,0 +1,1513 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-27T14:39:56.007423+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261027-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json", + "config_sha256": "a504cba0ba21c651be807d51a3a6390efcd85c8ac35c71ab35f930ce25ec8f31", + "job_name": "pi-flash-dev34-install-pair-01-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json", + "config_sha256": "4fd74c33dc4f20fcebc8a47c22e55c4dd8df5752ba2735d871fa554945797a9e", + "job_name": "pi-flash-dev34-install-pair-02-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json", + "config_sha256": "f5f93dad4ab114dbcfcb6e33df013da6a0caa1622ba1f221e8ba23fdbc3d83a0", + "job_name": "pi-flash-dev34-install-pair-03-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json", + "config_sha256": "486ca7f2d04944bbb1b1972884f24a44856973565fa5ce706cb12edc8e75719d", + "job_name": "pi-flash-dev34-install-pair-04-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json", + "config_sha256": "610756dbc34469f06888900a001718093001e4d7798f0149cc128bd9629c337c", + "job_name": "pi-flash-dev34-install-pair-05-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json", + "config_sha256": "916c0c8b2ccbb21d7bdb5fc715deda5807073a0206d38f5839577300bcf36bda", + "job_name": "pi-flash-dev34-install-pair-06-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json", + "config_sha256": "7908b54e3be775dbe7591ea3c6cf63195c3b3467051cc3f13ca6d6d260d6f286", + "job_name": "pi-flash-dev34-install-pair-07-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json", + "config_sha256": "0efda2e91ca6b1e5dd657d4793434c4e2520ae44c8b820c2bcef5886a87c2aed", + "job_name": "pi-flash-dev34-install-pair-08-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json", + "config_sha256": "5af119ca876cbd2d58c3a8a47ac3c5c87f091d2d95a9c861d40a9b67de119157", + "job_name": "pi-flash-dev34-install-pair-09-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json", + "config_sha256": "95d0a0bdc45d5b706dc521d26b112cb4d025f2e1dfe5d61cdd8091e1b6873c46", + "job_name": "pi-flash-dev34-install-pair-10-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json", + "config_sha256": "27e40374aa0ffb882a82090e2b2ee374be364573fa4adc7bb722035cc3e4a138", + "job_name": "pi-flash-dev34-install-pair-11-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json", + "config_sha256": "3042316a2c9b3706329697302fd0c2eb7591090df0c33d229c234702d20d35bc", + "job_name": "pi-flash-dev34-install-pair-12-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json", + "config_sha256": "4aff1477dd87925fc861a983976ff3ca7a24bdf6b6511f8eeb1e4d5e4b7b2c57", + "job_name": "pi-flash-dev34-install-pair-13-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json", + "config_sha256": "bd9e29b461e01930be0013b326e1533b590a4adf1b60488e4fcbfa49df542bfa", + "job_name": "pi-flash-dev34-install-pair-14-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json", + "config_sha256": "bc8d3ccd5585caa533bc7b57dba602e7d24fd657ab2d0816930cfcff3ab9976a", + "job_name": "pi-flash-dev34-install-pair-15-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json", + "config_sha256": "9895825cbce46e0699796cb9f6f7ac6fd035d730771179ebc5d8bd1a352159d2", + "job_name": "pi-flash-dev34-install-pair-16-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json", + "config_sha256": "ee3c012d87e9abc9568fdcc70554f7c6f0a43fd60de5243f4a16e5c3c3a1e44e", + "job_name": "pi-flash-dev34-install-pair-17-20261027" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json", + "config_sha256": "f855275727a19942a078a183c0d17bd778cdc6537990ae0d0d0eaaa4f590cfc2", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json", + "config_sha256": "2e222dec329f537a4ee32a0dcc2099e032073cfd51245a2dceaab24d326321c8", + "job_name": "pi-flash-dev34-build-cython-ext-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json", + "config_sha256": "327cfbecb3e66adce9d1108773a88772e91991ae6dc5c2eff32ec312e0661d3a", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json", + "config_sha256": "a4b3b3256ef06f7f00f30bce5dcbd844fb0098c2f4260ed8009c98731db5bcaf", + "job_name": "pi-flash-dev34-cobol-modernization-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json", + "config_sha256": "61283b4f874bbdb16d0c30c099097b06ad10570e695057349c7ecf62e715239c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json", + "config_sha256": "4ab832bc4db0f7bd4b7d074e3e5069690ee5c545220efe59330a720a89d6f7af", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json", + "config_sha256": "f3dee54b5e729eb956bf6245776a5fd13e2b6e0560fe90a56ab7dc0cdc7ba08b", + "job_name": "pi-flash-dev34-git-multibranch-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json", + "config_sha256": "e114fbae8dd5c433ec07c1e51fffc74afcf3cbe02017a16b43b2b0f0ce1ad725", + "job_name": "pi-flash-dev34-kv-store-grpc-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json", + "config_sha256": "172c60e3defaba1c7247ab270774921a944705a66cf06faea8a4b63e8b32181d", + "job_name": "pi-flash-dev34-log-summary-date-ranges-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json", + "config_sha256": "5da2e84447b06cb1ae8cb799d848c0ba5482751dd4338206f30e8abaf2d5ebba", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json", + "config_sha256": "b99464639394f22fa5778d33f324fdb30f288c82d197e7f1d4fc60fe7aa1e70d", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json", + "config_sha256": "e9d6eccc6ff6302d9183f1fe16a769b4b8aaa9e904e538304524b8b415ea9c44", + "job_name": "pi-flash-dev34-multi-source-data-merger-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json", + "config_sha256": "f5859eb248a1fd47af3fc79e404198657ca15eadd280e86b9df72f38ffb7c5c7", + "job_name": "pi-flash-dev34-nginx-request-logging-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json", + "config_sha256": "1117a63a125bae6762daf8661d6fb4bcacfb2c64fa9cdd77a7feca557282002a", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json", + "config_sha256": "0cf340c2600dae97e163356e5317c4a6815e532f157af1b87be25bd68ffb229d", + "job_name": "pi-flash-dev34-pypi-server-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json", + "config_sha256": "bc8d94cf359c3e574bab1525ff5d2dbc1e9370fa24b76e425b5cdea509b17b81", + "job_name": "pi-flash-dev34-pytorch-model-cli-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json", + "config_sha256": "9468602451b76e319ff1a7511b0af518ae82c00ce66627ecea648e5067e4cefa", + "job_name": "pi-flash-dev34-pytorch-model-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json", + "config_sha256": "797c9b8274cb3275047772436c0923a58db408fe8b1a54de61b6e31ae7a1bfe4", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json", + "config_sha256": "c99965db28ea35888799fadbb4516b4744405993a941f0fc3a41db18ed49d6b0", + "job_name": "pi-flash-dev34-query-optimize-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json", + "config_sha256": "6d2761a3f62abe8de938f9d9c82981acdad4f29c009d295ef43affb9a736c69a", + "job_name": "pi-flash-dev34-regex-log-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json", + "config_sha256": "74e8b02c6626d5d32d6878a89bf9ae6f58c5742adde13603eb1fa191b36bd055", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json", + "config_sha256": "7f9924b834c402fa5de3d4237d03757e577f3bbfa8a1700321be3647e1a82a1d", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json", + "config_sha256": "995c3a4547aa6a43e2da83cd4e116be734286f3e418b54ee7853d2ccbc6e943d", + "job_name": "pi-flash-dev34-write-compressor-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json", + "config_sha256": "ef7378048fb266873970df2b58cb5e8a7f24c2bf46d3dfc29a274878c0c78c50", + "job_name": "pi-flash-dev34-constraints-scheduling-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json", + "config_sha256": "9bbb388e05d922ba833d326fab8c163650330e2103b8cee08c20bc201b4913db", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json", + "config_sha256": "21312c25c60db17eff072db349e11edcb8685813ebecd72b75f746c31a3f3f57", + "job_name": "pi-flash-dev34-dna-assembly-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json", + "config_sha256": "8dfa0b6550412d19a642e0e0bc1ea94e346c826278f350d35dd50342b0691c2a", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json", + "config_sha256": "d40220ece6f7e8b4d17a5a7480abd930faecd6c9e87b201fee20cf4276d51116", + "job_name": "pi-flash-dev34-path-tracing-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json", + "config_sha256": "796feeffd8803d35c0945a67851fcf42b8bc91429a008aab2fcdb681b7490f79", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json", + "config_sha256": "c586de89b868e1419eb2ed328425301cd02ea62e216579d7003df0026e80032e", + "job_name": "pi-flash-dev34-caffe-cifar-10-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json", + "config_sha256": "9638b54f9b08d0e411cecf5d1a13f60af1c8d2050044b7a3e4581a2fbbf63e27", + "job_name": "pi-flash-dev34-circuit-fibsqrt-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json", + "config_sha256": "2ecd4945403c2a5e06c1674bddc09ae374dd45aadb1f9a28256ef191118e7fd9", + "job_name": "pi-flash-dev34-mteb-leaderboard-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json", + "config_sha256": "fb41b5c6e14656a537a9da50179331c9919dc637335b17f1329a656073869c06", + "job_name": "pi-flash-dev34-regex-chess-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json", + "config_sha256": "05b45814dc881e8c0fc6b3ebdee217754b73393487b27779855d13ef742a6b85", + "job_name": "pi-flash-dev34-build-pov-ray-20261027" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "639709c03a7b0c3eb6aa78b4e87e6184b49b4b102853a31e33dd832d759180c0", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "8842e7ad313171cdcd99bbad795558952d5e0242f9d2f8bf8605ea757e3c96ef", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json": "ee8266facfe5c85d8ae02dd409d6fcc002728b1d81683740bfa6dcb7ea8ab068" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "ready_for_native_preflight: project volume has about 69 GiB free; Docker volume has about 20 GiB free", + "runtime_environment": "UV_PROJECT_ENVIRONMENT=/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/jobs/harbor-runtime-venv", + "supersedes": "experiment.json", + "amendment": "Storage capacity restored after repository relocation. Run native installation preflight in consecutive two-task batches. Restore each original image by its registered repository digest and verify its identity before use; evict only image references not present in the initial Docker inventory after each batch, without force or container deletion. Agent distribution, runtime source, model settings, tasks, budgets, ordering, and verifier policy are unchanged.", + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360 + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v3.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v3.json new file mode 100644 index 0000000..6c50abb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v3.json @@ -0,0 +1,1514 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-27T14:44:33.190045+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json", + "config_sha256": "a504cba0ba21c651be807d51a3a6390efcd85c8ac35c71ab35f930ce25ec8f31", + "job_name": "pi-flash-dev34-install-pair-01-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json", + "config_sha256": "4fd74c33dc4f20fcebc8a47c22e55c4dd8df5752ba2735d871fa554945797a9e", + "job_name": "pi-flash-dev34-install-pair-02-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json", + "config_sha256": "f5f93dad4ab114dbcfcb6e33df013da6a0caa1622ba1f221e8ba23fdbc3d83a0", + "job_name": "pi-flash-dev34-install-pair-03-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json", + "config_sha256": "486ca7f2d04944bbb1b1972884f24a44856973565fa5ce706cb12edc8e75719d", + "job_name": "pi-flash-dev34-install-pair-04-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json", + "config_sha256": "610756dbc34469f06888900a001718093001e4d7798f0149cc128bd9629c337c", + "job_name": "pi-flash-dev34-install-pair-05-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json", + "config_sha256": "916c0c8b2ccbb21d7bdb5fc715deda5807073a0206d38f5839577300bcf36bda", + "job_name": "pi-flash-dev34-install-pair-06-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json", + "config_sha256": "7908b54e3be775dbe7591ea3c6cf63195c3b3467051cc3f13ca6d6d260d6f286", + "job_name": "pi-flash-dev34-install-pair-07-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json", + "config_sha256": "0efda2e91ca6b1e5dd657d4793434c4e2520ae44c8b820c2bcef5886a87c2aed", + "job_name": "pi-flash-dev34-install-pair-08-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json", + "config_sha256": "5af119ca876cbd2d58c3a8a47ac3c5c87f091d2d95a9c861d40a9b67de119157", + "job_name": "pi-flash-dev34-install-pair-09-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json", + "config_sha256": "95d0a0bdc45d5b706dc521d26b112cb4d025f2e1dfe5d61cdd8091e1b6873c46", + "job_name": "pi-flash-dev34-install-pair-10-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json", + "config_sha256": "27e40374aa0ffb882a82090e2b2ee374be364573fa4adc7bb722035cc3e4a138", + "job_name": "pi-flash-dev34-install-pair-11-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json", + "config_sha256": "3042316a2c9b3706329697302fd0c2eb7591090df0c33d229c234702d20d35bc", + "job_name": "pi-flash-dev34-install-pair-12-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json", + "config_sha256": "4aff1477dd87925fc861a983976ff3ca7a24bdf6b6511f8eeb1e4d5e4b7b2c57", + "job_name": "pi-flash-dev34-install-pair-13-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json", + "config_sha256": "bd9e29b461e01930be0013b326e1533b590a4adf1b60488e4fcbfa49df542bfa", + "job_name": "pi-flash-dev34-install-pair-14-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json", + "config_sha256": "bc8d3ccd5585caa533bc7b57dba602e7d24fd657ab2d0816930cfcff3ab9976a", + "job_name": "pi-flash-dev34-install-pair-15-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json", + "config_sha256": "9895825cbce46e0699796cb9f6f7ac6fd035d730771179ebc5d8bd1a352159d2", + "job_name": "pi-flash-dev34-install-pair-16-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json", + "config_sha256": "ee3c012d87e9abc9568fdcc70554f7c6f0a43fd60de5243f4a16e5c3c3a1e44e", + "job_name": "pi-flash-dev34-install-pair-17-20261027" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json", + "config_sha256": "f855275727a19942a078a183c0d17bd778cdc6537990ae0d0d0eaaa4f590cfc2", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json", + "config_sha256": "2e222dec329f537a4ee32a0dcc2099e032073cfd51245a2dceaab24d326321c8", + "job_name": "pi-flash-dev34-build-cython-ext-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json", + "config_sha256": "327cfbecb3e66adce9d1108773a88772e91991ae6dc5c2eff32ec312e0661d3a", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json", + "config_sha256": "a4b3b3256ef06f7f00f30bce5dcbd844fb0098c2f4260ed8009c98731db5bcaf", + "job_name": "pi-flash-dev34-cobol-modernization-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json", + "config_sha256": "61283b4f874bbdb16d0c30c099097b06ad10570e695057349c7ecf62e715239c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json", + "config_sha256": "4ab832bc4db0f7bd4b7d074e3e5069690ee5c545220efe59330a720a89d6f7af", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json", + "config_sha256": "f3dee54b5e729eb956bf6245776a5fd13e2b6e0560fe90a56ab7dc0cdc7ba08b", + "job_name": "pi-flash-dev34-git-multibranch-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json", + "config_sha256": "e114fbae8dd5c433ec07c1e51fffc74afcf3cbe02017a16b43b2b0f0ce1ad725", + "job_name": "pi-flash-dev34-kv-store-grpc-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json", + "config_sha256": "172c60e3defaba1c7247ab270774921a944705a66cf06faea8a4b63e8b32181d", + "job_name": "pi-flash-dev34-log-summary-date-ranges-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json", + "config_sha256": "5da2e84447b06cb1ae8cb799d848c0ba5482751dd4338206f30e8abaf2d5ebba", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json", + "config_sha256": "b99464639394f22fa5778d33f324fdb30f288c82d197e7f1d4fc60fe7aa1e70d", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json", + "config_sha256": "e9d6eccc6ff6302d9183f1fe16a769b4b8aaa9e904e538304524b8b415ea9c44", + "job_name": "pi-flash-dev34-multi-source-data-merger-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json", + "config_sha256": "f5859eb248a1fd47af3fc79e404198657ca15eadd280e86b9df72f38ffb7c5c7", + "job_name": "pi-flash-dev34-nginx-request-logging-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json", + "config_sha256": "1117a63a125bae6762daf8661d6fb4bcacfb2c64fa9cdd77a7feca557282002a", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json", + "config_sha256": "0cf340c2600dae97e163356e5317c4a6815e532f157af1b87be25bd68ffb229d", + "job_name": "pi-flash-dev34-pypi-server-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json", + "config_sha256": "bc8d94cf359c3e574bab1525ff5d2dbc1e9370fa24b76e425b5cdea509b17b81", + "job_name": "pi-flash-dev34-pytorch-model-cli-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json", + "config_sha256": "9468602451b76e319ff1a7511b0af518ae82c00ce66627ecea648e5067e4cefa", + "job_name": "pi-flash-dev34-pytorch-model-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json", + "config_sha256": "797c9b8274cb3275047772436c0923a58db408fe8b1a54de61b6e31ae7a1bfe4", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json", + "config_sha256": "c99965db28ea35888799fadbb4516b4744405993a941f0fc3a41db18ed49d6b0", + "job_name": "pi-flash-dev34-query-optimize-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json", + "config_sha256": "6d2761a3f62abe8de938f9d9c82981acdad4f29c009d295ef43affb9a736c69a", + "job_name": "pi-flash-dev34-regex-log-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json", + "config_sha256": "74e8b02c6626d5d32d6878a89bf9ae6f58c5742adde13603eb1fa191b36bd055", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json", + "config_sha256": "7f9924b834c402fa5de3d4237d03757e577f3bbfa8a1700321be3647e1a82a1d", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json", + "config_sha256": "995c3a4547aa6a43e2da83cd4e116be734286f3e418b54ee7853d2ccbc6e943d", + "job_name": "pi-flash-dev34-write-compressor-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json", + "config_sha256": "ef7378048fb266873970df2b58cb5e8a7f24c2bf46d3dfc29a274878c0c78c50", + "job_name": "pi-flash-dev34-constraints-scheduling-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json", + "config_sha256": "9bbb388e05d922ba833d326fab8c163650330e2103b8cee08c20bc201b4913db", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json", + "config_sha256": "21312c25c60db17eff072db349e11edcb8685813ebecd72b75f746c31a3f3f57", + "job_name": "pi-flash-dev34-dna-assembly-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json", + "config_sha256": "8dfa0b6550412d19a642e0e0bc1ea94e346c826278f350d35dd50342b0691c2a", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json", + "config_sha256": "d40220ece6f7e8b4d17a5a7480abd930faecd6c9e87b201fee20cf4276d51116", + "job_name": "pi-flash-dev34-path-tracing-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json", + "config_sha256": "796feeffd8803d35c0945a67851fcf42b8bc91429a008aab2fcdb681b7490f79", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json", + "config_sha256": "c586de89b868e1419eb2ed328425301cd02ea62e216579d7003df0026e80032e", + "job_name": "pi-flash-dev34-caffe-cifar-10-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json", + "config_sha256": "9638b54f9b08d0e411cecf5d1a13f60af1c8d2050044b7a3e4581a2fbbf63e27", + "job_name": "pi-flash-dev34-circuit-fibsqrt-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json", + "config_sha256": "2ecd4945403c2a5e06c1674bddc09ae374dd45aadb1f9a28256ef191118e7fd9", + "job_name": "pi-flash-dev34-mteb-leaderboard-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json", + "config_sha256": "fb41b5c6e14656a537a9da50179331c9919dc637335b17f1329a656073869c06", + "job_name": "pi-flash-dev34-regex-chess-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json", + "config_sha256": "05b45814dc881e8c0fc6b3ebdee217754b73393487b27779855d13ef742a6b85", + "job_name": "pi-flash-dev34-build-pov-ray-20261027" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "639709c03a7b0c3eb6aa78b4e87e6184b49b4b102853a31e33dd832d759180c0", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "bc337c1f8c6a4887ccad778e60815261826eab105aac35e3dfcf43855d1f0f64", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json": "ee8266facfe5c85d8ae02dd409d6fcc002728b1d81683740bfa6dcb7ea8ab068" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "ready_for_native_preflight: project volume has about 69 GiB free; Docker volume has about 20 GiB free", + "runtime_environment": "UV_PROJECT_ENVIRONMENT=/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/jobs/harbor-runtime-venv", + "supersedes": "experiment-v2.json", + "amendment": "Storage capacity restored after repository relocation. Run native installation preflight in consecutive two-task batches. Restore each original image by its registered repository digest and verify its identity before use; evict only image references not present in the initial Docker inventory after each batch, without force or container deletion. Agent distribution, runtime source, model settings, tasks, budgets, ordering, and verifier policy are unchanged. Replace stalled direct Docker pulls with the existing verified ranged-download image-cache utility. Preserve the v2 launch and image-download records. No native job, model call, or official test ran in v2.", + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v4.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v4.json new file mode 100644 index 0000000..d5af61a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v4.json @@ -0,0 +1,1581 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-28T13:48:16.389369+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json", + "config_sha256": "a504cba0ba21c651be807d51a3a6390efcd85c8ac35c71ab35f930ce25ec8f31", + "job_name": "pi-flash-dev34-install-pair-01-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json", + "config_sha256": "4fd74c33dc4f20fcebc8a47c22e55c4dd8df5752ba2735d871fa554945797a9e", + "job_name": "pi-flash-dev34-install-pair-02-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json", + "config_sha256": "f5f93dad4ab114dbcfcb6e33df013da6a0caa1622ba1f221e8ba23fdbc3d83a0", + "job_name": "pi-flash-dev34-install-pair-03-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json", + "config_sha256": "486ca7f2d04944bbb1b1972884f24a44856973565fa5ce706cb12edc8e75719d", + "job_name": "pi-flash-dev34-install-pair-04-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json", + "config_sha256": "610756dbc34469f06888900a001718093001e4d7798f0149cc128bd9629c337c", + "job_name": "pi-flash-dev34-install-pair-05-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json", + "config_sha256": "916c0c8b2ccbb21d7bdb5fc715deda5807073a0206d38f5839577300bcf36bda", + "job_name": "pi-flash-dev34-install-pair-06-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json", + "config_sha256": "7908b54e3be775dbe7591ea3c6cf63195c3b3467051cc3f13ca6d6d260d6f286", + "job_name": "pi-flash-dev34-install-pair-07-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json", + "config_sha256": "0efda2e91ca6b1e5dd657d4793434c4e2520ae44c8b820c2bcef5886a87c2aed", + "job_name": "pi-flash-dev34-install-pair-08-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json", + "config_sha256": "5af119ca876cbd2d58c3a8a47ac3c5c87f091d2d95a9c861d40a9b67de119157", + "job_name": "pi-flash-dev34-install-pair-09-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json", + "config_sha256": "95d0a0bdc45d5b706dc521d26b112cb4d025f2e1dfe5d61cdd8091e1b6873c46", + "job_name": "pi-flash-dev34-install-pair-10-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json", + "config_sha256": "27e40374aa0ffb882a82090e2b2ee374be364573fa4adc7bb722035cc3e4a138", + "job_name": "pi-flash-dev34-install-pair-11-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json", + "config_sha256": "3042316a2c9b3706329697302fd0c2eb7591090df0c33d229c234702d20d35bc", + "job_name": "pi-flash-dev34-install-pair-12-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json", + "config_sha256": "4aff1477dd87925fc861a983976ff3ca7a24bdf6b6511f8eeb1e4d5e4b7b2c57", + "job_name": "pi-flash-dev34-install-pair-13-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json", + "config_sha256": "bd9e29b461e01930be0013b326e1533b590a4adf1b60488e4fcbfa49df542bfa", + "job_name": "pi-flash-dev34-install-pair-14-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json", + "config_sha256": "bc8d3ccd5585caa533bc7b57dba602e7d24fd657ab2d0816930cfcff3ab9976a", + "job_name": "pi-flash-dev34-install-pair-15-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json", + "config_sha256": "9895825cbce46e0699796cb9f6f7ac6fd035d730771179ebc5d8bd1a352159d2", + "job_name": "pi-flash-dev34-install-pair-16-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json", + "config_sha256": "ee3c012d87e9abc9568fdcc70554f7c6f0a43fd60de5243f4a16e5c3c3a1e44e", + "job_name": "pi-flash-dev34-install-pair-17-20261027" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json", + "config_sha256": "f855275727a19942a078a183c0d17bd778cdc6537990ae0d0d0eaaa4f590cfc2", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json", + "config_sha256": "2e222dec329f537a4ee32a0dcc2099e032073cfd51245a2dceaab24d326321c8", + "job_name": "pi-flash-dev34-build-cython-ext-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json", + "config_sha256": "327cfbecb3e66adce9d1108773a88772e91991ae6dc5c2eff32ec312e0661d3a", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json", + "config_sha256": "a4b3b3256ef06f7f00f30bce5dcbd844fb0098c2f4260ed8009c98731db5bcaf", + "job_name": "pi-flash-dev34-cobol-modernization-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json", + "config_sha256": "61283b4f874bbdb16d0c30c099097b06ad10570e695057349c7ecf62e715239c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json", + "config_sha256": "4ab832bc4db0f7bd4b7d074e3e5069690ee5c545220efe59330a720a89d6f7af", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json", + "config_sha256": "f3dee54b5e729eb956bf6245776a5fd13e2b6e0560fe90a56ab7dc0cdc7ba08b", + "job_name": "pi-flash-dev34-git-multibranch-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json", + "config_sha256": "e114fbae8dd5c433ec07c1e51fffc74afcf3cbe02017a16b43b2b0f0ce1ad725", + "job_name": "pi-flash-dev34-kv-store-grpc-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json", + "config_sha256": "172c60e3defaba1c7247ab270774921a944705a66cf06faea8a4b63e8b32181d", + "job_name": "pi-flash-dev34-log-summary-date-ranges-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json", + "config_sha256": "5da2e84447b06cb1ae8cb799d848c0ba5482751dd4338206f30e8abaf2d5ebba", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json", + "config_sha256": "b99464639394f22fa5778d33f324fdb30f288c82d197e7f1d4fc60fe7aa1e70d", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json", + "config_sha256": "e9d6eccc6ff6302d9183f1fe16a769b4b8aaa9e904e538304524b8b415ea9c44", + "job_name": "pi-flash-dev34-multi-source-data-merger-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json", + "config_sha256": "f5859eb248a1fd47af3fc79e404198657ca15eadd280e86b9df72f38ffb7c5c7", + "job_name": "pi-flash-dev34-nginx-request-logging-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json", + "config_sha256": "1117a63a125bae6762daf8661d6fb4bcacfb2c64fa9cdd77a7feca557282002a", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json", + "config_sha256": "0cf340c2600dae97e163356e5317c4a6815e532f157af1b87be25bd68ffb229d", + "job_name": "pi-flash-dev34-pypi-server-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json", + "config_sha256": "bc8d94cf359c3e574bab1525ff5d2dbc1e9370fa24b76e425b5cdea509b17b81", + "job_name": "pi-flash-dev34-pytorch-model-cli-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json", + "config_sha256": "9468602451b76e319ff1a7511b0af518ae82c00ce66627ecea648e5067e4cefa", + "job_name": "pi-flash-dev34-pytorch-model-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json", + "config_sha256": "797c9b8274cb3275047772436c0923a58db408fe8b1a54de61b6e31ae7a1bfe4", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json", + "config_sha256": "c99965db28ea35888799fadbb4516b4744405993a941f0fc3a41db18ed49d6b0", + "job_name": "pi-flash-dev34-query-optimize-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json", + "config_sha256": "6d2761a3f62abe8de938f9d9c82981acdad4f29c009d295ef43affb9a736c69a", + "job_name": "pi-flash-dev34-regex-log-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json", + "config_sha256": "74e8b02c6626d5d32d6878a89bf9ae6f58c5742adde13603eb1fa191b36bd055", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json", + "config_sha256": "7f9924b834c402fa5de3d4237d03757e577f3bbfa8a1700321be3647e1a82a1d", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json", + "config_sha256": "995c3a4547aa6a43e2da83cd4e116be734286f3e418b54ee7853d2ccbc6e943d", + "job_name": "pi-flash-dev34-write-compressor-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json", + "config_sha256": "ef7378048fb266873970df2b58cb5e8a7f24c2bf46d3dfc29a274878c0c78c50", + "job_name": "pi-flash-dev34-constraints-scheduling-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json", + "config_sha256": "9bbb388e05d922ba833d326fab8c163650330e2103b8cee08c20bc201b4913db", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json", + "config_sha256": "21312c25c60db17eff072db349e11edcb8685813ebecd72b75f746c31a3f3f57", + "job_name": "pi-flash-dev34-dna-assembly-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json", + "config_sha256": "8dfa0b6550412d19a642e0e0bc1ea94e346c826278f350d35dd50342b0691c2a", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json", + "config_sha256": "d40220ece6f7e8b4d17a5a7480abd930faecd6c9e87b201fee20cf4276d51116", + "job_name": "pi-flash-dev34-path-tracing-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json", + "config_sha256": "796feeffd8803d35c0945a67851fcf42b8bc91429a008aab2fcdb681b7490f79", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json", + "config_sha256": "c586de89b868e1419eb2ed328425301cd02ea62e216579d7003df0026e80032e", + "job_name": "pi-flash-dev34-caffe-cifar-10-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json", + "config_sha256": "9638b54f9b08d0e411cecf5d1a13f60af1c8d2050044b7a3e4581a2fbbf63e27", + "job_name": "pi-flash-dev34-circuit-fibsqrt-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json", + "config_sha256": "2ecd4945403c2a5e06c1674bddc09ae374dd45aadb1f9a28256ef191118e7fd9", + "job_name": "pi-flash-dev34-mteb-leaderboard-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json", + "config_sha256": "fb41b5c6e14656a537a9da50179331c9919dc637335b17f1329a656073869c06", + "job_name": "pi-flash-dev34-regex-chess-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json", + "config_sha256": "05b45814dc881e8c0fc6b3ebdee217754b73393487b27779855d13ef742a6b85", + "job_name": "pi-flash-dev34-build-pov-ray-20261027" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "8f76391304c9b16db4d984d272b6fff4d883f2b0cbf9cebd6909eb1ebf30d098", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "74de1f60cf9433da8ce9488e2eae9a8ff7081015f3a0aebac2ac3ddee6a0637b", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json": "ee8266facfe5c85d8ae02dd409d6fcc002728b1d81683740bfa6dcb7ea8ab068", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/agent/bootstrap.json": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/manifest.json": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/config.json": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/agent/bootstrap.json": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/agent/bootstrap.json": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/manifest.json": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/config.json": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/agent/bootstrap.json": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/agent/bootstrap.json": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/manifest.json": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/config.json": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/agent/bootstrap.json": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/agent/bootstrap.json": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/manifest.json": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/config.json": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/manifest.json": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/config.json": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/manifest.json": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/config.json": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/agent/bootstrap.json": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/manifest.json": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/config.json": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/manifest.json": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/config.json": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json": "159ae6f4c3253468f249ff44d2c9da07992827d153416e40a28956b08d63e43b", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/initial-docker-images.json": "036c56a8e4ce32953540f670d8ea9b30829ea1e560bd3e35b04dd4bccec1ad5a" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "16 of 34 installation checks passed; 18 pending; zero model outputs. QEMU verifier preflight remains pending.", + "runtime_environment": "UV_PROJECT_ENVIRONMENT=/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/jobs/harbor-runtime-venv", + "supersedes": "experiment-v3.json", + "amendment": "Resume after a registry HTTP error during QEMU image restoration, before its native installation job. Preserve the failed image-transfer evidence. Refresh anonymous registry tokens before expiry and once on HTTP 401; resume existing ranges and retain full digest verification. Reuse 16 strictly validated native installation records from eight completed jobs, with their evidence bound by hash. Run the remaining 18 fresh installation checks, then the same fixed 34-task model queue. No model calls or native verifier executions have occurred.", + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + }, + "reused_preflight_trials": "preflight-resume-evidence.json" +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v5.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v5.json new file mode 100644 index 0000000..521c62a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v5.json @@ -0,0 +1,1648 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-29T13:25:44.227545+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261029-transport-v5-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json", + "config_sha256": "a504cba0ba21c651be807d51a3a6390efcd85c8ac35c71ab35f930ce25ec8f31", + "job_name": "pi-flash-dev34-install-pair-01-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json", + "config_sha256": "4fd74c33dc4f20fcebc8a47c22e55c4dd8df5752ba2735d871fa554945797a9e", + "job_name": "pi-flash-dev34-install-pair-02-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json", + "config_sha256": "f5f93dad4ab114dbcfcb6e33df013da6a0caa1622ba1f221e8ba23fdbc3d83a0", + "job_name": "pi-flash-dev34-install-pair-03-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json", + "config_sha256": "486ca7f2d04944bbb1b1972884f24a44856973565fa5ce706cb12edc8e75719d", + "job_name": "pi-flash-dev34-install-pair-04-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json", + "config_sha256": "610756dbc34469f06888900a001718093001e4d7798f0149cc128bd9629c337c", + "job_name": "pi-flash-dev34-install-pair-05-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json", + "config_sha256": "916c0c8b2ccbb21d7bdb5fc715deda5807073a0206d38f5839577300bcf36bda", + "job_name": "pi-flash-dev34-install-pair-06-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json", + "config_sha256": "7908b54e3be775dbe7591ea3c6cf63195c3b3467051cc3f13ca6d6d260d6f286", + "job_name": "pi-flash-dev34-install-pair-07-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json", + "config_sha256": "0efda2e91ca6b1e5dd657d4793434c4e2520ae44c8b820c2bcef5886a87c2aed", + "job_name": "pi-flash-dev34-install-pair-08-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json", + "config_sha256": "5af119ca876cbd2d58c3a8a47ac3c5c87f091d2d95a9c861d40a9b67de119157", + "job_name": "pi-flash-dev34-install-pair-09-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json", + "config_sha256": "95d0a0bdc45d5b706dc521d26b112cb4d025f2e1dfe5d61cdd8091e1b6873c46", + "job_name": "pi-flash-dev34-install-pair-10-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json", + "config_sha256": "27e40374aa0ffb882a82090e2b2ee374be364573fa4adc7bb722035cc3e4a138", + "job_name": "pi-flash-dev34-install-pair-11-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json", + "config_sha256": "3042316a2c9b3706329697302fd0c2eb7591090df0c33d229c234702d20d35bc", + "job_name": "pi-flash-dev34-install-pair-12-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json", + "config_sha256": "4aff1477dd87925fc861a983976ff3ca7a24bdf6b6511f8eeb1e4d5e4b7b2c57", + "job_name": "pi-flash-dev34-install-pair-13-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json", + "config_sha256": "bd9e29b461e01930be0013b326e1533b590a4adf1b60488e4fcbfa49df542bfa", + "job_name": "pi-flash-dev34-install-pair-14-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json", + "config_sha256": "bc8d3ccd5585caa533bc7b57dba602e7d24fd657ab2d0816930cfcff3ab9976a", + "job_name": "pi-flash-dev34-install-pair-15-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json", + "config_sha256": "9895825cbce46e0699796cb9f6f7ac6fd035d730771179ebc5d8bd1a352159d2", + "job_name": "pi-flash-dev34-install-pair-16-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json", + "config_sha256": "ee3c012d87e9abc9568fdcc70554f7c6f0a43fd60de5243f4a16e5c3c3a1e44e", + "job_name": "pi-flash-dev34-install-pair-17-20261027" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json", + "config_sha256": "f855275727a19942a078a183c0d17bd778cdc6537990ae0d0d0eaaa4f590cfc2", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json", + "config_sha256": "2e222dec329f537a4ee32a0dcc2099e032073cfd51245a2dceaab24d326321c8", + "job_name": "pi-flash-dev34-build-cython-ext-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json", + "config_sha256": "327cfbecb3e66adce9d1108773a88772e91991ae6dc5c2eff32ec312e0661d3a", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json", + "config_sha256": "a4b3b3256ef06f7f00f30bce5dcbd844fb0098c2f4260ed8009c98731db5bcaf", + "job_name": "pi-flash-dev34-cobol-modernization-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json", + "config_sha256": "61283b4f874bbdb16d0c30c099097b06ad10570e695057349c7ecf62e715239c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json", + "config_sha256": "4ab832bc4db0f7bd4b7d074e3e5069690ee5c545220efe59330a720a89d6f7af", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json", + "config_sha256": "f3dee54b5e729eb956bf6245776a5fd13e2b6e0560fe90a56ab7dc0cdc7ba08b", + "job_name": "pi-flash-dev34-git-multibranch-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json", + "config_sha256": "e114fbae8dd5c433ec07c1e51fffc74afcf3cbe02017a16b43b2b0f0ce1ad725", + "job_name": "pi-flash-dev34-kv-store-grpc-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json", + "config_sha256": "172c60e3defaba1c7247ab270774921a944705a66cf06faea8a4b63e8b32181d", + "job_name": "pi-flash-dev34-log-summary-date-ranges-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json", + "config_sha256": "5da2e84447b06cb1ae8cb799d848c0ba5482751dd4338206f30e8abaf2d5ebba", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json", + "config_sha256": "b99464639394f22fa5778d33f324fdb30f288c82d197e7f1d4fc60fe7aa1e70d", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json", + "config_sha256": "e9d6eccc6ff6302d9183f1fe16a769b4b8aaa9e904e538304524b8b415ea9c44", + "job_name": "pi-flash-dev34-multi-source-data-merger-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json", + "config_sha256": "f5859eb248a1fd47af3fc79e404198657ca15eadd280e86b9df72f38ffb7c5c7", + "job_name": "pi-flash-dev34-nginx-request-logging-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json", + "config_sha256": "1117a63a125bae6762daf8661d6fb4bcacfb2c64fa9cdd77a7feca557282002a", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json", + "config_sha256": "0cf340c2600dae97e163356e5317c4a6815e532f157af1b87be25bd68ffb229d", + "job_name": "pi-flash-dev34-pypi-server-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json", + "config_sha256": "bc8d94cf359c3e574bab1525ff5d2dbc1e9370fa24b76e425b5cdea509b17b81", + "job_name": "pi-flash-dev34-pytorch-model-cli-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json", + "config_sha256": "9468602451b76e319ff1a7511b0af518ae82c00ce66627ecea648e5067e4cefa", + "job_name": "pi-flash-dev34-pytorch-model-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json", + "config_sha256": "797c9b8274cb3275047772436c0923a58db408fe8b1a54de61b6e31ae7a1bfe4", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json", + "config_sha256": "c99965db28ea35888799fadbb4516b4744405993a941f0fc3a41db18ed49d6b0", + "job_name": "pi-flash-dev34-query-optimize-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json", + "config_sha256": "6d2761a3f62abe8de938f9d9c82981acdad4f29c009d295ef43affb9a736c69a", + "job_name": "pi-flash-dev34-regex-log-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json", + "config_sha256": "74e8b02c6626d5d32d6878a89bf9ae6f58c5742adde13603eb1fa191b36bd055", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json", + "config_sha256": "7f9924b834c402fa5de3d4237d03757e577f3bbfa8a1700321be3647e1a82a1d", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json", + "config_sha256": "995c3a4547aa6a43e2da83cd4e116be734286f3e418b54ee7853d2ccbc6e943d", + "job_name": "pi-flash-dev34-write-compressor-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json", + "config_sha256": "ef7378048fb266873970df2b58cb5e8a7f24c2bf46d3dfc29a274878c0c78c50", + "job_name": "pi-flash-dev34-constraints-scheduling-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json", + "config_sha256": "9bbb388e05d922ba833d326fab8c163650330e2103b8cee08c20bc201b4913db", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json", + "config_sha256": "21312c25c60db17eff072db349e11edcb8685813ebecd72b75f746c31a3f3f57", + "job_name": "pi-flash-dev34-dna-assembly-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json", + "config_sha256": "8dfa0b6550412d19a642e0e0bc1ea94e346c826278f350d35dd50342b0691c2a", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json", + "config_sha256": "d40220ece6f7e8b4d17a5a7480abd930faecd6c9e87b201fee20cf4276d51116", + "job_name": "pi-flash-dev34-path-tracing-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json", + "config_sha256": "796feeffd8803d35c0945a67851fcf42b8bc91429a008aab2fcdb681b7490f79", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json", + "config_sha256": "c586de89b868e1419eb2ed328425301cd02ea62e216579d7003df0026e80032e", + "job_name": "pi-flash-dev34-caffe-cifar-10-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json", + "config_sha256": "9638b54f9b08d0e411cecf5d1a13f60af1c8d2050044b7a3e4581a2fbbf63e27", + "job_name": "pi-flash-dev34-circuit-fibsqrt-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json", + "config_sha256": "2ecd4945403c2a5e06c1674bddc09ae374dd45aadb1f9a28256ef191118e7fd9", + "job_name": "pi-flash-dev34-mteb-leaderboard-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json", + "config_sha256": "fb41b5c6e14656a537a9da50179331c9919dc637335b17f1329a656073869c06", + "job_name": "pi-flash-dev34-regex-chess-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json", + "config_sha256": "05b45814dc881e8c0fc6b3ebdee217754b73393487b27779855d13ef742a6b85", + "job_name": "pi-flash-dev34-build-pov-ray-20261027" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "bc6baea30fc2f023a8e3dbd262c4d1bbe486530691d4765b7c20d2a7b5faa71a", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "74de1f60cf9433da8ce9488e2eae9a8ff7081015f3a0aebac2ac3ddee6a0637b", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json": "ee8266facfe5c85d8ae02dd409d6fcc002728b1d81683740bfa6dcb7ea8ab068", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/agent/bootstrap.json": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/manifest.json": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/config.json": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/agent/bootstrap.json": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/agent/bootstrap.json": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/manifest.json": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/config.json": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/agent/bootstrap.json": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/agent/bootstrap.json": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/manifest.json": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/config.json": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/agent/bootstrap.json": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/agent/bootstrap.json": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/manifest.json": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/config.json": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/manifest.json": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/config.json": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/manifest.json": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/config.json": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/agent/bootstrap.json": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/manifest.json": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/config.json": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/manifest.json": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/config.json": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json": "159ae6f4c3253468f249ff44d2c9da07992827d153416e40a28956b08d63e43b", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/initial-docker-images.json": "036c56a8e4ce32953540f670d8ea9b30829ea1e560bd3e35b04dd4bccec1ad5a", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/result.json": "237980344951ee2285e28499288d745bc857c1c96806129340725a5429acb0b4", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/manifest.json": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/config.json": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/result.json": "35329f2737d85aa325756e9d62650e63ade91222fda61d370d8e5422d63074c6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/agent/bootstrap.json": "fc48c7dee6de95fe0edcd1631e1b3dbd900f2652d1ad5cbaa1d4a91f777feb53", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-09-20261027.json": "8b7cda88cdd32e5415096cc5fea995f0cc8f9a1d16f2dfea935b623a53e35f58", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/result.json": "783879855d9c2b4b7f592d981231b02d3f8d6ce92f62f2fcdb511556919c0722", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/agent/bootstrap.json": "37f681eae6736199a9acbe0f9d1b1774eb82c7a097799622073de4b74d61c620", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/manifest.json": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/config.json": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/result.json": "00bf033a6dc65fd8557f7a2bd9cd6132508565d33815224057086789d8fe98a5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-10-20261027.json": "614222e8e2c4079b83b7a1c9e7b5ab11055dcbe005e8983295f1f3856392095a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/result.json": "198681d196f12f8a480dcfb2516e75b08f4b77516673f2562d5a82f770a2259a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/manifest.json": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/config.json": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/result.json": "6f06e8262444c01a2bfa76dfdbaf630eb2e9c416b4df66c273f10801aac77950", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-11-20261027.json": "a4707566ddc1a105e1850f07c038f2de8cefadea9e1bff94b85d17cb9c64784f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/result.json": "d60e5bc61d23265f69ebe35ae2077a57c319220a653d09712fc7942394d2b6ff", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/agent/bootstrap.json": "cf3d634a13469c9792d5f85e9e97f6a6fd2770edd21edd2323b1a4a0ff9b88bf", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/manifest.json": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/config.json": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/result.json": "2f117df7e8d0f27aeed1e3a86f14255bbc4faeb3e06ef4464c6d0bb5f6556887", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/agent/bootstrap.json": "a777f4d674a14f3a92030c0d28a0aeea8efc22dac6e5c348302d0ddccc73925e", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-12-20261027.json": "1991305b8e1c61171cdae0b07dea7dc9be1681300a9d56854e5d37364d525ebf", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/result.json": "9869ea4ef71a7e1e8005983d5439985d64d3c52ae4c1db2f0898d5be46f2561d", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/agent/bootstrap.json": "f7e10349b73b17d26271980d4778162d3b278f8e1d4a71d42a4cb0d23b866852", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/manifest.json": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/config.json": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/result.json": "160dab77106b3651c220484d1b884dd8263aecbb5f78be413cec93e8a352d7e3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-13-20261027.json": "55557503afac1f21545f5cdff0f0b5c8f075936ed68edbdab46c4eae04e0b1c6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/result.json": "4cf30715e661bc55ded77f425af33e87b309aad80ba102f28e9e2ccf2cef3148", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/manifest.json": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/config.json": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/result.json": "b6c8d393ebf552037d56d260f1b3035d5903f8b40a32f083682f518a2f37c2bd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/agent/bootstrap.json": "d2f6dec48072dbd7f15f33bdc2361c6fbfd5d345ddd4748d292bac71e13e8e82", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-14-20261027.json": "3bf013957d71092723564930ab2807a2df1ad479d4773855567b743092b873d3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/result.json": "64fd984e34c1d1d95be0f12e4b4e6637f67a3b75374a3bc69390f2f0a6df9854", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/agent/bootstrap.json": "bfe8b063bf0a185c19fb3d6b3eba32ada212d94dfd2b1f082b9a49e957760a0a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/manifest.json": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/config.json": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/result.json": "8ef5cdf5b93a0dc5f491d2651abc187516954f3866ff4df1faa18801f0ba9104", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-15-20261027.json": "f72f78d43cca70f0603c4875767cd982dadc9e6cf574e87bd9a1487cc1c930a3", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-30-evidence.json": "2b32fff7f4245d987a155666b26869af65d8f0b85903795a57b2644062b8d396", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-import-evidence.json": "f627465bb54dd84ac88e02603c46292c2fe2cfdf62f0b128837b5d1f21228d6a", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v5-orchestration/initial-docker-images.json": "315c7ce2a3189d83354b5cc14cea5f653af0795300509caa9138fbfbf6b5ae76" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "30 installation checks passed; four pending; zero model outputs. Native-platform OCI-index import support passed 206 tests and a real Docker digest-preservation probe.", + "runtime_environment": "UV_PROJECT_ENVIRONMENT=/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/jobs/harbor-runtime-venv", + "supersedes": "experiment-v4.json", + "amendment": "Resume after the range cache rejected the MTEB OCI index and a native pull timed out after 600 seconds. Preserve both transfer failures. Extend the reviewed range cache to retain the unchanged parent index and unique Linux amd64 manifest, fetch only original native layers, import that platform explicitly, and verify the original parent image/repository digest. An isolated real-Docker import probe confirmed preservation. Reuse all 30 strictly validated native installation records, including QEMU verifier dependency preflight. Run the remaining four fresh installation checks, then the unchanged 34-task model queue. No model outputs exist yet.", + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + }, + "reused_preflight_trials": "preflight-resume-30-evidence.json" +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v6.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v6.json new file mode 100644 index 0000000..3ebec2f --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v6.json @@ -0,0 +1,1649 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-29T13:36:38.645087+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json", + "config_sha256": "a504cba0ba21c651be807d51a3a6390efcd85c8ac35c71ab35f930ce25ec8f31", + "job_name": "pi-flash-dev34-install-pair-01-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json", + "config_sha256": "4fd74c33dc4f20fcebc8a47c22e55c4dd8df5752ba2735d871fa554945797a9e", + "job_name": "pi-flash-dev34-install-pair-02-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json", + "config_sha256": "f5f93dad4ab114dbcfcb6e33df013da6a0caa1622ba1f221e8ba23fdbc3d83a0", + "job_name": "pi-flash-dev34-install-pair-03-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json", + "config_sha256": "486ca7f2d04944bbb1b1972884f24a44856973565fa5ce706cb12edc8e75719d", + "job_name": "pi-flash-dev34-install-pair-04-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json", + "config_sha256": "610756dbc34469f06888900a001718093001e4d7798f0149cc128bd9629c337c", + "job_name": "pi-flash-dev34-install-pair-05-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json", + "config_sha256": "916c0c8b2ccbb21d7bdb5fc715deda5807073a0206d38f5839577300bcf36bda", + "job_name": "pi-flash-dev34-install-pair-06-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json", + "config_sha256": "7908b54e3be775dbe7591ea3c6cf63195c3b3467051cc3f13ca6d6d260d6f286", + "job_name": "pi-flash-dev34-install-pair-07-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json", + "config_sha256": "0efda2e91ca6b1e5dd657d4793434c4e2520ae44c8b820c2bcef5886a87c2aed", + "job_name": "pi-flash-dev34-install-pair-08-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json", + "config_sha256": "5af119ca876cbd2d58c3a8a47ac3c5c87f091d2d95a9c861d40a9b67de119157", + "job_name": "pi-flash-dev34-install-pair-09-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json", + "config_sha256": "95d0a0bdc45d5b706dc521d26b112cb4d025f2e1dfe5d61cdd8091e1b6873c46", + "job_name": "pi-flash-dev34-install-pair-10-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json", + "config_sha256": "27e40374aa0ffb882a82090e2b2ee374be364573fa4adc7bb722035cc3e4a138", + "job_name": "pi-flash-dev34-install-pair-11-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json", + "config_sha256": "3042316a2c9b3706329697302fd0c2eb7591090df0c33d229c234702d20d35bc", + "job_name": "pi-flash-dev34-install-pair-12-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json", + "config_sha256": "4aff1477dd87925fc861a983976ff3ca7a24bdf6b6511f8eeb1e4d5e4b7b2c57", + "job_name": "pi-flash-dev34-install-pair-13-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json", + "config_sha256": "bd9e29b461e01930be0013b326e1533b590a4adf1b60488e4fcbfa49df542bfa", + "job_name": "pi-flash-dev34-install-pair-14-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json", + "config_sha256": "bc8d3ccd5585caa533bc7b57dba602e7d24fd657ab2d0816930cfcff3ab9976a", + "job_name": "pi-flash-dev34-install-pair-15-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json", + "config_sha256": "9895825cbce46e0699796cb9f6f7ac6fd035d730771179ebc5d8bd1a352159d2", + "job_name": "pi-flash-dev34-install-pair-16-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json", + "config_sha256": "ee3c012d87e9abc9568fdcc70554f7c6f0a43fd60de5243f4a16e5c3c3a1e44e", + "job_name": "pi-flash-dev34-install-pair-17-20261027" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json", + "config_sha256": "f855275727a19942a078a183c0d17bd778cdc6537990ae0d0d0eaaa4f590cfc2", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json", + "config_sha256": "2e222dec329f537a4ee32a0dcc2099e032073cfd51245a2dceaab24d326321c8", + "job_name": "pi-flash-dev34-build-cython-ext-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json", + "config_sha256": "327cfbecb3e66adce9d1108773a88772e91991ae6dc5c2eff32ec312e0661d3a", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json", + "config_sha256": "a4b3b3256ef06f7f00f30bce5dcbd844fb0098c2f4260ed8009c98731db5bcaf", + "job_name": "pi-flash-dev34-cobol-modernization-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json", + "config_sha256": "61283b4f874bbdb16d0c30c099097b06ad10570e695057349c7ecf62e715239c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json", + "config_sha256": "4ab832bc4db0f7bd4b7d074e3e5069690ee5c545220efe59330a720a89d6f7af", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json", + "config_sha256": "f3dee54b5e729eb956bf6245776a5fd13e2b6e0560fe90a56ab7dc0cdc7ba08b", + "job_name": "pi-flash-dev34-git-multibranch-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json", + "config_sha256": "e114fbae8dd5c433ec07c1e51fffc74afcf3cbe02017a16b43b2b0f0ce1ad725", + "job_name": "pi-flash-dev34-kv-store-grpc-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json", + "config_sha256": "172c60e3defaba1c7247ab270774921a944705a66cf06faea8a4b63e8b32181d", + "job_name": "pi-flash-dev34-log-summary-date-ranges-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json", + "config_sha256": "5da2e84447b06cb1ae8cb799d848c0ba5482751dd4338206f30e8abaf2d5ebba", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json", + "config_sha256": "b99464639394f22fa5778d33f324fdb30f288c82d197e7f1d4fc60fe7aa1e70d", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json", + "config_sha256": "e9d6eccc6ff6302d9183f1fe16a769b4b8aaa9e904e538304524b8b415ea9c44", + "job_name": "pi-flash-dev34-multi-source-data-merger-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json", + "config_sha256": "f5859eb248a1fd47af3fc79e404198657ca15eadd280e86b9df72f38ffb7c5c7", + "job_name": "pi-flash-dev34-nginx-request-logging-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json", + "config_sha256": "1117a63a125bae6762daf8661d6fb4bcacfb2c64fa9cdd77a7feca557282002a", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json", + "config_sha256": "0cf340c2600dae97e163356e5317c4a6815e532f157af1b87be25bd68ffb229d", + "job_name": "pi-flash-dev34-pypi-server-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json", + "config_sha256": "bc8d94cf359c3e574bab1525ff5d2dbc1e9370fa24b76e425b5cdea509b17b81", + "job_name": "pi-flash-dev34-pytorch-model-cli-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json", + "config_sha256": "9468602451b76e319ff1a7511b0af518ae82c00ce66627ecea648e5067e4cefa", + "job_name": "pi-flash-dev34-pytorch-model-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json", + "config_sha256": "797c9b8274cb3275047772436c0923a58db408fe8b1a54de61b6e31ae7a1bfe4", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json", + "config_sha256": "c99965db28ea35888799fadbb4516b4744405993a941f0fc3a41db18ed49d6b0", + "job_name": "pi-flash-dev34-query-optimize-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json", + "config_sha256": "6d2761a3f62abe8de938f9d9c82981acdad4f29c009d295ef43affb9a736c69a", + "job_name": "pi-flash-dev34-regex-log-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json", + "config_sha256": "74e8b02c6626d5d32d6878a89bf9ae6f58c5742adde13603eb1fa191b36bd055", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json", + "config_sha256": "7f9924b834c402fa5de3d4237d03757e577f3bbfa8a1700321be3647e1a82a1d", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json", + "config_sha256": "995c3a4547aa6a43e2da83cd4e116be734286f3e418b54ee7853d2ccbc6e943d", + "job_name": "pi-flash-dev34-write-compressor-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json", + "config_sha256": "ef7378048fb266873970df2b58cb5e8a7f24c2bf46d3dfc29a274878c0c78c50", + "job_name": "pi-flash-dev34-constraints-scheduling-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json", + "config_sha256": "9bbb388e05d922ba833d326fab8c163650330e2103b8cee08c20bc201b4913db", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json", + "config_sha256": "21312c25c60db17eff072db349e11edcb8685813ebecd72b75f746c31a3f3f57", + "job_name": "pi-flash-dev34-dna-assembly-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json", + "config_sha256": "8dfa0b6550412d19a642e0e0bc1ea94e346c826278f350d35dd50342b0691c2a", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json", + "config_sha256": "d40220ece6f7e8b4d17a5a7480abd930faecd6c9e87b201fee20cf4276d51116", + "job_name": "pi-flash-dev34-path-tracing-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json", + "config_sha256": "796feeffd8803d35c0945a67851fcf42b8bc91429a008aab2fcdb681b7490f79", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json", + "config_sha256": "c586de89b868e1419eb2ed328425301cd02ea62e216579d7003df0026e80032e", + "job_name": "pi-flash-dev34-caffe-cifar-10-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json", + "config_sha256": "9638b54f9b08d0e411cecf5d1a13f60af1c8d2050044b7a3e4581a2fbbf63e27", + "job_name": "pi-flash-dev34-circuit-fibsqrt-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json", + "config_sha256": "2ecd4945403c2a5e06c1674bddc09ae374dd45aadb1f9a28256ef191118e7fd9", + "job_name": "pi-flash-dev34-mteb-leaderboard-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json", + "config_sha256": "fb41b5c6e14656a537a9da50179331c9919dc637335b17f1329a656073869c06", + "job_name": "pi-flash-dev34-regex-chess-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json", + "config_sha256": "05b45814dc881e8c0fc6b3ebdee217754b73393487b27779855d13ef742a6b85", + "job_name": "pi-flash-dev34-build-pov-ray-20261027" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "11ddde3fe86a6e4bfdefe7c3578794df98d237f701e40f219d76a541911c9f0c", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "74de1f60cf9433da8ce9488e2eae9a8ff7081015f3a0aebac2ac3ddee6a0637b", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json": "ee8266facfe5c85d8ae02dd409d6fcc002728b1d81683740bfa6dcb7ea8ab068", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/agent/bootstrap.json": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/manifest.json": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/config.json": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/agent/bootstrap.json": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/agent/bootstrap.json": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/manifest.json": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/config.json": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/agent/bootstrap.json": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/agent/bootstrap.json": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/manifest.json": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/config.json": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/agent/bootstrap.json": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/agent/bootstrap.json": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/manifest.json": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/config.json": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/manifest.json": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/config.json": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/manifest.json": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/config.json": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/agent/bootstrap.json": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/manifest.json": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/config.json": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/manifest.json": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/config.json": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json": "159ae6f4c3253468f249ff44d2c9da07992827d153416e40a28956b08d63e43b", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/initial-docker-images.json": "036c56a8e4ce32953540f670d8ea9b30829ea1e560bd3e35b04dd4bccec1ad5a", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/result.json": "237980344951ee2285e28499288d745bc857c1c96806129340725a5429acb0b4", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/manifest.json": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/config.json": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/result.json": "35329f2737d85aa325756e9d62650e63ade91222fda61d370d8e5422d63074c6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/agent/bootstrap.json": "fc48c7dee6de95fe0edcd1631e1b3dbd900f2652d1ad5cbaa1d4a91f777feb53", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-09-20261027.json": "8b7cda88cdd32e5415096cc5fea995f0cc8f9a1d16f2dfea935b623a53e35f58", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/result.json": "783879855d9c2b4b7f592d981231b02d3f8d6ce92f62f2fcdb511556919c0722", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/agent/bootstrap.json": "37f681eae6736199a9acbe0f9d1b1774eb82c7a097799622073de4b74d61c620", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/manifest.json": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/config.json": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/result.json": "00bf033a6dc65fd8557f7a2bd9cd6132508565d33815224057086789d8fe98a5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-10-20261027.json": "614222e8e2c4079b83b7a1c9e7b5ab11055dcbe005e8983295f1f3856392095a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/result.json": "198681d196f12f8a480dcfb2516e75b08f4b77516673f2562d5a82f770a2259a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/manifest.json": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/config.json": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/result.json": "6f06e8262444c01a2bfa76dfdbaf630eb2e9c416b4df66c273f10801aac77950", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-11-20261027.json": "a4707566ddc1a105e1850f07c038f2de8cefadea9e1bff94b85d17cb9c64784f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/result.json": "d60e5bc61d23265f69ebe35ae2077a57c319220a653d09712fc7942394d2b6ff", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/agent/bootstrap.json": "cf3d634a13469c9792d5f85e9e97f6a6fd2770edd21edd2323b1a4a0ff9b88bf", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/manifest.json": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/config.json": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/result.json": "2f117df7e8d0f27aeed1e3a86f14255bbc4faeb3e06ef4464c6d0bb5f6556887", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/agent/bootstrap.json": "a777f4d674a14f3a92030c0d28a0aeea8efc22dac6e5c348302d0ddccc73925e", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-12-20261027.json": "1991305b8e1c61171cdae0b07dea7dc9be1681300a9d56854e5d37364d525ebf", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/result.json": "9869ea4ef71a7e1e8005983d5439985d64d3c52ae4c1db2f0898d5be46f2561d", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/agent/bootstrap.json": "f7e10349b73b17d26271980d4778162d3b278f8e1d4a71d42a4cb0d23b866852", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/manifest.json": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/config.json": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/result.json": "160dab77106b3651c220484d1b884dd8263aecbb5f78be413cec93e8a352d7e3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-13-20261027.json": "55557503afac1f21545f5cdff0f0b5c8f075936ed68edbdab46c4eae04e0b1c6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/result.json": "4cf30715e661bc55ded77f425af33e87b309aad80ba102f28e9e2ccf2cef3148", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/manifest.json": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/config.json": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/result.json": "b6c8d393ebf552037d56d260f1b3035d5903f8b40a32f083682f518a2f37c2bd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/agent/bootstrap.json": "d2f6dec48072dbd7f15f33bdc2361c6fbfd5d345ddd4748d292bac71e13e8e82", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-14-20261027.json": "3bf013957d71092723564930ab2807a2df1ad479d4773855567b743092b873d3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/result.json": "64fd984e34c1d1d95be0f12e4b4e6637f67a3b75374a3bc69390f2f0a6df9854", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/agent/bootstrap.json": "bfe8b063bf0a185c19fb3d6b3eba32ada212d94dfd2b1f082b9a49e957760a0a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/manifest.json": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/config.json": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/result.json": "8ef5cdf5b93a0dc5f491d2651abc187516954f3866ff4df1faa18801f0ba9104", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-15-20261027.json": "f72f78d43cca70f0603c4875767cd982dadc9e6cf574e87bd9a1487cc1c930a3", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-30-evidence.json": "2b32fff7f4245d987a155666b26869af65d8f0b85903795a57b2644062b8d396", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-import-evidence.json": "f627465bb54dd84ac88e02603c46292c2fe2cfdf62f0b128837b5d1f21228d6a", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v5-orchestration/initial-docker-images.json": "315c7ce2a3189d83354b5cc14cea5f653af0795300509caa9138fbfbf6b5ae76", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-attestation-import-evidence.json": "9ae22db1e73ba1b8d009b5e7bdba97ee6ed0c25ea17b53797854e5bb1454f5c6" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "30 native installation checks passed; four pending; zero model outputs. Full native-attestation import support passed 210 regression tests and an isolated real-Docker digest-preservation probe.", + "runtime_environment": "UV_PROJECT_ENVIRONMENT=/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/jobs/harbor-runtime-venv", + "supersedes": "experiment-v5.json", + "amendment": "Complete native-platform OCI-index imports by fetching original attestations associated with the selected Linux amd64 manifest. Preserve every metadata manifest, config, and in-toto payload by size/SHA256 without rewriting parent or child manifests. An isolated four-descriptor real-Docker probe confirmed the original parent image/repository digest with the non-native graph omitted. V5 was stopped during image transfer before any new native installation or model work. Preserve all prior transfer evidence and reuse the same 30 validated native installation records, then run the four pending fresh checks and unchanged 34-task model queue.", + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + }, + "reused_preflight_trials": "preflight-resume-30-evidence.json" +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json new file mode 100644 index 0000000..46c96c9 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json @@ -0,0 +1,1751 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-29T15:15:12.966882+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + }, + "pi_supervised": { + "agent": "pi", + "source_root": "jobs/supervised-source", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json", + "config_sha256": "a504cba0ba21c651be807d51a3a6390efcd85c8ac35c71ab35f930ce25ec8f31", + "job_name": "pi-flash-dev34-install-pair-01-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json", + "config_sha256": "4fd74c33dc4f20fcebc8a47c22e55c4dd8df5752ba2735d871fa554945797a9e", + "job_name": "pi-flash-dev34-install-pair-02-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json", + "config_sha256": "f5f93dad4ab114dbcfcb6e33df013da6a0caa1622ba1f221e8ba23fdbc3d83a0", + "job_name": "pi-flash-dev34-install-pair-03-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json", + "config_sha256": "486ca7f2d04944bbb1b1972884f24a44856973565fa5ce706cb12edc8e75719d", + "job_name": "pi-flash-dev34-install-pair-04-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json", + "config_sha256": "610756dbc34469f06888900a001718093001e4d7798f0149cc128bd9629c337c", + "job_name": "pi-flash-dev34-install-pair-05-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json", + "config_sha256": "916c0c8b2ccbb21d7bdb5fc715deda5807073a0206d38f5839577300bcf36bda", + "job_name": "pi-flash-dev34-install-pair-06-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json", + "config_sha256": "7908b54e3be775dbe7591ea3c6cf63195c3b3467051cc3f13ca6d6d260d6f286", + "job_name": "pi-flash-dev34-install-pair-07-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json", + "config_sha256": "0efda2e91ca6b1e5dd657d4793434c4e2520ae44c8b820c2bcef5886a87c2aed", + "job_name": "pi-flash-dev34-install-pair-08-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json", + "config_sha256": "5af119ca876cbd2d58c3a8a47ac3c5c87f091d2d95a9c861d40a9b67de119157", + "job_name": "pi-flash-dev34-install-pair-09-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json", + "config_sha256": "95d0a0bdc45d5b706dc521d26b112cb4d025f2e1dfe5d61cdd8091e1b6873c46", + "job_name": "pi-flash-dev34-install-pair-10-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json", + "config_sha256": "27e40374aa0ffb882a82090e2b2ee374be364573fa4adc7bb722035cc3e4a138", + "job_name": "pi-flash-dev34-install-pair-11-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json", + "config_sha256": "3042316a2c9b3706329697302fd0c2eb7591090df0c33d229c234702d20d35bc", + "job_name": "pi-flash-dev34-install-pair-12-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json", + "config_sha256": "4aff1477dd87925fc861a983976ff3ca7a24bdf6b6511f8eeb1e4d5e4b7b2c57", + "job_name": "pi-flash-dev34-install-pair-13-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json", + "config_sha256": "bd9e29b461e01930be0013b326e1533b590a4adf1b60488e4fcbfa49df542bfa", + "job_name": "pi-flash-dev34-install-pair-14-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json", + "config_sha256": "bc8d3ccd5585caa533bc7b57dba602e7d24fd657ab2d0816930cfcff3ab9976a", + "job_name": "pi-flash-dev34-install-pair-15-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json", + "config_sha256": "9895825cbce46e0699796cb9f6f7ac6fd035d730771179ebc5d8bd1a352159d2", + "job_name": "pi-flash-dev34-install-pair-16-20261027" + }, + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json", + "config_sha256": "ee3c012d87e9abc9568fdcc70554f7c6f0a43fd60de5243f4a16e5c3c3a1e44e", + "job_name": "pi-flash-dev34-install-pair-17-20261027" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-01.json", + "config_sha256": "a95f95f131addabe8a2f3cad276b8007bc216612a461456b67ab3806e5dcf2ff", + "job_name": "pi-flash-dev34-supervised-install-pair-01-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-02.json", + "config_sha256": "8d50f7ba8c6b955074d31d1bf3acc31d3f54c6bb7248a91c14d4de8491041ee1", + "job_name": "pi-flash-dev34-supervised-install-pair-02-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-03.json", + "config_sha256": "49473c5c502c3c30508d69e930a3ba2bb2a25ae7357503b9add36f0daa265441", + "job_name": "pi-flash-dev34-supervised-install-pair-03-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-04.json", + "config_sha256": "e3801362704d64c3176991a1d3284f64edad1173bc1624c2e22805e13ff26597", + "job_name": "pi-flash-dev34-supervised-install-pair-04-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-05.json", + "config_sha256": "0f66883e4a4b15cd76a23431aaba86d09df21a8e1010111793067ec0f7990f8e", + "job_name": "pi-flash-dev34-supervised-install-pair-05-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-06.json", + "config_sha256": "5ad309ef71cd35a5e3c635cd1e5f36bf00b271699f7c297e0a5cf8cf29b947d8", + "job_name": "pi-flash-dev34-supervised-install-pair-06-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-07.json", + "config_sha256": "a05eb5ff02893f7a137622021e02c8b5c965d726d0f7992e19586158f33e8484", + "job_name": "pi-flash-dev34-supervised-install-pair-07-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-08.json", + "config_sha256": "049dcd77597a922a7320dd14eb70e91f8235e0db4003801c1c099f07a96109bd", + "job_name": "pi-flash-dev34-supervised-install-pair-08-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-09.json", + "config_sha256": "e99622ba45c8dafa04912cf73d6d402bd17894ccdb455be23c7eb2665b849f9c", + "job_name": "pi-flash-dev34-supervised-install-pair-09-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-10.json", + "config_sha256": "a58d68d44511d27accef90e732c9a65f670dfb5ff2d6a6b3e00486976820b33b", + "job_name": "pi-flash-dev34-supervised-install-pair-10-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-11.json", + "config_sha256": "a4f937424f52e21445b75d2b2f518f54faf6cf770e3580eabee0233570bbdbcd", + "job_name": "pi-flash-dev34-supervised-install-pair-11-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-12.json", + "config_sha256": "74a3c1df0dc4ee62673b1cbe3aeb102904facc976b0ba5b2ea8601610af6f2f1", + "job_name": "pi-flash-dev34-supervised-install-pair-12-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-13.json", + "config_sha256": "59c64e41911f7b93dc87e4433259ec7527bb4e0515d272a637dfe7f7a12dbea2", + "job_name": "pi-flash-dev34-supervised-install-pair-13-20261029" + }, + { + "arm": "pi_supervised", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-14.json", + "config_sha256": "8821464ee40883b678e11c5323bfe8c7fff0470d53225f0e91d156e1c9e4afca", + "job_name": "pi-flash-dev34-supervised-install-pair-14-20261029" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json", + "config_sha256": "f855275727a19942a078a183c0d17bd778cdc6537990ae0d0d0eaaa4f590cfc2", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json", + "config_sha256": "2e222dec329f537a4ee32a0dcc2099e032073cfd51245a2dceaab24d326321c8", + "job_name": "pi-flash-dev34-build-cython-ext-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json", + "config_sha256": "327cfbecb3e66adce9d1108773a88772e91991ae6dc5c2eff32ec312e0661d3a", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json", + "config_sha256": "a4b3b3256ef06f7f00f30bce5dcbd844fb0098c2f4260ed8009c98731db5bcaf", + "job_name": "pi-flash-dev34-cobol-modernization-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json", + "config_sha256": "61283b4f874bbdb16d0c30c099097b06ad10570e695057349c7ecf62e715239c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json", + "config_sha256": "4ab832bc4db0f7bd4b7d074e3e5069690ee5c545220efe59330a720a89d6f7af", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/git-multibranch.json", + "config_sha256": "e82de786edcde6487f9d29e8efb7faef15ab287deb7f3839177c4b69d9ffdd58", + "job_name": "pi-flash-dev34-git-multibranch-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/kv-store-grpc.json", + "config_sha256": "5838c57a323643038d325a07b4534e2484116e004740218cf0abf419362c7106", + "job_name": "pi-flash-dev34-kv-store-grpc-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/log-summary-date-ranges.json", + "config_sha256": "16dd3d28e730a50932c8e620d5e85f00ef7bcd2a5ad748cdbd5ee98221790c7b", + "job_name": "pi-flash-dev34-log-summary-date-ranges-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/merge-diff-arc-agi-task.json", + "config_sha256": "dd5640217b8f15603600d909ef369067a4f1f1304e4963c98123a33faae402d5", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/model-extraction-relu-logits.json", + "config_sha256": "2f4b9e8f194227a22429fdd2580097ef93d2f771e1ccb4f1758fe2c2f0e63f6a", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/multi-source-data-merger.json", + "config_sha256": "6b060b709b53eb9edad0a1077c6c35657869f90a817a7dad8027864f29970f5c", + "job_name": "pi-flash-dev34-multi-source-data-merger-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/nginx-request-logging.json", + "config_sha256": "3d4100098c401012d9d3a19cfa7a7416839244083366d4567417d31369212a1f", + "job_name": "pi-flash-dev34-nginx-request-logging-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/openssl-selfsigned-cert.json", + "config_sha256": "f54f55e0dea36ebf48ce57a1230aca23190f1e4e6aa2bf0ef766f92a84befc79", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pypi-server.json", + "config_sha256": "f501dbd5174a64073dca552a058a7378395818275c16dfdb794c3a757a3695c4", + "job_name": "pi-flash-dev34-pypi-server-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-cli.json", + "config_sha256": "3fa99b3b6820c51d9b71c1a558e0364ef4d1f752658d8b1bb535b1414dba54b1", + "job_name": "pi-flash-dev34-pytorch-model-cli-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-recovery.json", + "config_sha256": "1bfc4db3fc0704807455c14a6de79f25f6b6744af31aef2178f57d171b6065dd", + "job_name": "pi-flash-dev34-pytorch-model-recovery-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/qemu-alpine-ssh.json", + "config_sha256": "987c521c4a524d4c7c0b69256581ad64f24062699d96bd9b6c42c3c86a85f9d0", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/query-optimize.json", + "config_sha256": "9d6b41321f3dae16302d119602ce3138bb6323608e3a97fd5da86c45ce71e459", + "job_name": "pi-flash-dev34-query-optimize-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-log.json", + "config_sha256": "acc03788d50492d4c65be8bd4deb72e27c85d6a68cd9d2c0e73434d67da3cca4", + "job_name": "pi-flash-dev34-regex-log-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-pipeline-parallelism.json", + "config_sha256": "c31c908e23504632c6771b34e91bb388aa04c85debbdddae2f7d9c3591e75e94", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-tensor-parallelism.json", + "config_sha256": "82c950c43995eddfee067131cadb8d1838fd57cb5182ffb695743cfe4b898096", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/write-compressor.json", + "config_sha256": "c3e947d7ab3802f1b84d96a951c392c47b29a0c9e6725a79474b60d6d571a6b0", + "job_name": "pi-flash-dev34-write-compressor-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/constraints-scheduling.json", + "config_sha256": "e779ac72c7d08fd52b668a7ee9de6b3a294ed3d8fe7d329dad314e70ab15e747", + "job_name": "pi-flash-dev34-constraints-scheduling-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/custom-memory-heap-crash.json", + "config_sha256": "2f444688239d861c33bf095d9f2488633f005dcdd7e34fa8a95571183fdeb9d3", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/dna-assembly.json", + "config_sha256": "bcd769425a951f2ed2cc848b78cac63e198d6e9fefc8c023c9f1c1bbc0f56e8d", + "job_name": "pi-flash-dev34-dna-assembly-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/llm-inference-batching-scheduler.json", + "config_sha256": "2f5d1f8e7fb1caf47fbcf113b4f1bdf7edadd83704134da6c1a5d8714f567f7f", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/path-tracing.json", + "config_sha256": "84572c7d6aae802e24b30d0d6e029bc95b2844db17903b601aaa15672bc0e8ea", + "job_name": "pi-flash-dev34-path-tracing-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/schemelike-metacircular-eval.json", + "config_sha256": "cfe16703517f9b7004bcb33bdfaafcc20e768c23859e3022ed73e3fbe237e075", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/caffe-cifar-10.json", + "config_sha256": "ca1518a872d96dd9fca94c8b72c64fc9513cd5517057701dff5f24266307d873", + "job_name": "pi-flash-dev34-caffe-cifar-10-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/circuit-fibsqrt.json", + "config_sha256": "825deca8696c453cdaa44e96ae2040aac75e052477473f23ca114dd48791b84c", + "job_name": "pi-flash-dev34-circuit-fibsqrt-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/mteb-leaderboard.json", + "config_sha256": "e56ea98e5479aa97b87c32ce311978b3ec817435b90b912e8e913886c6d267ba", + "job_name": "pi-flash-dev34-mteb-leaderboard-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-chess.json", + "config_sha256": "5794ba8bf3a33cb7a23816c6b9e6174741eceecdee9436f260c917cc3e930ff9", + "job_name": "pi-flash-dev34-regex-chess-supervised-20261029" + }, + { + "arm": "pi_supervised", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/build-pov-ray.json", + "config_sha256": "67f7b68ea75069e03a564f72dbd36ce2e8e9f1d9dce4ceef2fd7b712d7ab63e8", + "job_name": "pi-flash-dev34-build-pov-ray-supervised-20261029" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "11ddde3fe86a6e4bfdefe7c3578794df98d237f701e40f219d76a541911c9f0c", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "74de1f60cf9433da8ce9488e2eae9a8ff7081015f3a0aebac2ac3ddee6a0637b", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json": "ee8266facfe5c85d8ae02dd409d6fcc002728b1d81683740bfa6dcb7ea8ab068", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/agent/bootstrap.json": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/manifest.json": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/config.json": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/agent/bootstrap.json": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/agent/bootstrap.json": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/manifest.json": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/config.json": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/agent/bootstrap.json": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/agent/bootstrap.json": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/manifest.json": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/config.json": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/agent/bootstrap.json": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/agent/bootstrap.json": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/manifest.json": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/config.json": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/manifest.json": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/config.json": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/manifest.json": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/config.json": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/agent/bootstrap.json": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/manifest.json": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/config.json": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/agent/bootstrap.json": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/manifest.json": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/config.json": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json": "159ae6f4c3253468f249ff44d2c9da07992827d153416e40a28956b08d63e43b", + "jobs/pi-deepseek-flash-dev34-20261027-transport-v3-orchestration/initial-docker-images.json": "036c56a8e4ce32953540f670d8ea9b30829ea1e560bd3e35b04dd4bccec1ad5a", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-01-20261027.json": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-02-20261027.json": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-03-20261027.json": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-04-20261027.json": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-05-20261027.json": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-06-20261027.json": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-07-20261027.json": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-08-20261027.json": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/result.json": "237980344951ee2285e28499288d745bc857c1c96806129340725a5429acb0b4", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/manifest.json": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/config.json": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/result.json": "35329f2737d85aa325756e9d62650e63ade91222fda61d370d8e5422d63074c6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/agent/bootstrap.json": "fc48c7dee6de95fe0edcd1631e1b3dbd900f2652d1ad5cbaa1d4a91f777feb53", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-09-20261027.json": "8b7cda88cdd32e5415096cc5fea995f0cc8f9a1d16f2dfea935b623a53e35f58", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/result.json": "783879855d9c2b4b7f592d981231b02d3f8d6ce92f62f2fcdb511556919c0722", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/agent/bootstrap.json": "37f681eae6736199a9acbe0f9d1b1774eb82c7a097799622073de4b74d61c620", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/manifest.json": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/config.json": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/result.json": "00bf033a6dc65fd8557f7a2bd9cd6132508565d33815224057086789d8fe98a5", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-10-20261027.json": "614222e8e2c4079b83b7a1c9e7b5ab11055dcbe005e8983295f1f3856392095a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/result.json": "198681d196f12f8a480dcfb2516e75b08f4b77516673f2562d5a82f770a2259a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/manifest.json": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/config.json": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/result.json": "6f06e8262444c01a2bfa76dfdbaf630eb2e9c416b4df66c273f10801aac77950", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-11-20261027.json": "a4707566ddc1a105e1850f07c038f2de8cefadea9e1bff94b85d17cb9c64784f", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/result.json": "d60e5bc61d23265f69ebe35ae2077a57c319220a653d09712fc7942394d2b6ff", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/agent/bootstrap.json": "cf3d634a13469c9792d5f85e9e97f6a6fd2770edd21edd2323b1a4a0ff9b88bf", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/manifest.json": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/config.json": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/result.json": "2f117df7e8d0f27aeed1e3a86f14255bbc4faeb3e06ef4464c6d0bb5f6556887", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/agent/bootstrap.json": "a777f4d674a14f3a92030c0d28a0aeea8efc22dac6e5c348302d0ddccc73925e", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-12-20261027.json": "1991305b8e1c61171cdae0b07dea7dc9be1681300a9d56854e5d37364d525ebf", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/result.json": "9869ea4ef71a7e1e8005983d5439985d64d3c52ae4c1db2f0898d5be46f2561d", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/agent/bootstrap.json": "f7e10349b73b17d26271980d4778162d3b278f8e1d4a71d42a4cb0d23b866852", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/manifest.json": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/config.json": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/result.json": "160dab77106b3651c220484d1b884dd8263aecbb5f78be413cec93e8a352d7e3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/agent/bootstrap.json": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-13-20261027.json": "55557503afac1f21545f5cdff0f0b5c8f075936ed68edbdab46c4eae04e0b1c6", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/result.json": "4cf30715e661bc55ded77f425af33e87b309aad80ba102f28e9e2ccf2cef3148", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/manifest.json": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/config.json": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/result.json": "b6c8d393ebf552037d56d260f1b3035d5903f8b40a32f083682f518a2f37c2bd", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/agent/bootstrap.json": "d2f6dec48072dbd7f15f33bdc2361c6fbfd5d345ddd4748d292bac71e13e8e82", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-14-20261027.json": "3bf013957d71092723564930ab2807a2df1ad479d4773855567b743092b873d3", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/result.json": "64fd984e34c1d1d95be0f12e4b4e6637f67a3b75374a3bc69390f2f0a6df9854", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/agent/bootstrap.json": "bfe8b063bf0a185c19fb3d6b3eba32ada212d94dfd2b1f082b9a49e957760a0a", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/manifest.json": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/config.json": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027-input/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/result.json": "8ef5cdf5b93a0dc5f491d2651abc187516954f3866ff4df1faa18801f0ba9104", + "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/agent/bootstrap.json": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "jobs/pi-deepseek-flash-dev34-20261028-transport-v4-orchestration/pi-flash-dev34-install-pair-15-20261027.json": "f72f78d43cca70f0603c4875767cd982dadc9e6cf574e87bd9a1487cc1c930a3", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-30-evidence.json": "2b32fff7f4245d987a155666b26869af65d8f0b85903795a57b2644062b8d396", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-import-evidence.json": "f627465bb54dd84ac88e02603c46292c2fe2cfdf62f0b128837b5d1f21228d6a", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v5-orchestration/initial-docker-images.json": "315c7ce2a3189d83354b5cc14cea5f653af0795300509caa9138fbfbf6b5ae76", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-attestation-import-evidence.json": "9ae22db1e73ba1b8d009b5e7bdba97ee6ed0c25ea17b53797854e5bb1454f5c6", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-prefix-evidence.json": "9e54e83c7fb756eb2be38077b8b9b61edb1eb89ad00e6f59ac55ecf19ab17e21", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/timeout-boundary-evidence.json": "3f439452962ece02dea3486136c58bfdb6054954824a5fe798abc7e0cd1259cc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/process-supervision-probe-evidence.json": "9a93e14151c219d5fac1ef5e0c94d964816c96967ccdd505b9edfd4d157a45b9", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-adaptive-rejection-sampler-20261027.json": "34930edb4b539f2f797a4467a5ead86bf898fd6079cabf6c63adc4b3023b10ea", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-build-cython-ext-20261027.json": "cfbdf5ee5164bd04e78061de307ea6aacfc4c8b1c44819aebbbfff0f203458c6", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-cancel-async-tasks-20261027.json": "9ef6450ab54bf76f8dcd2903b1d8ea08f1b97f0f8c7096a9252343a3757114e2", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-cobol-modernization-20261027.json": "bf8210929a97395abec3b035c30fa9fc08069d3193c1db8349cbe8bfbbd1e9de", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-db-wal-recovery-20261027.json": "5397db0073e10bedd064ea5d1ae8f665f24738991f3d14c7f4b2103c0dae05df", + "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-fix-code-vulnerability-20261027.json": "54d0fb06804c46365dc92f10e185b17ab26ddbfae9d1d957a6a455f540839c6b" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "Six model attempts ended: three passes, one scored failure, two agent timeouts with invalid/unverified frozen boundaries. No later task started. Corrected source passed 225 tests and an isolated Docker cancellation probe. Fresh installation checks are required for the remaining 28 tasks.", + "runtime_environment": "UV_PROJECT_ENVIRONMENT=/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/jobs/harbor-supervised-runtime-venv", + "supersedes": "experiment-v6.json", + "amendment": "Correct agent cancellation before verification after timestamps proved post-timeout Pi activity during sampler verification. Preserve all six completed model attempts under their original source, without retries or replacements; quarantine timeout boundary exceptions. The 28 unstarted tasks retain the same Pi distribution/model/prompt/tools/budgets under a new frozen adapter supervising detached tools and confirming cancellation before grading. Require quiescence in the verifier and 28 fresh installation/supervisor probes before further model work.", + "batch_images": { + "manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json", + "storage_path": "/var/lib/docker", + "minimum_free_disk_bytes": 16106127360, + "cache_dir": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/.cache/nanopy-harbor/images" + }, + "reused_preflight_trials": "preflight-resume-30-evidence.json", + "source_policy": "The first six model attempts retain runtime-source at d6ed70a. The 28 unstarted tasks use supervised-source at 29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d. Pi distribution, model settings, instructions, budgets, resources, and task revisions are identical; only process supervision and verifier admission changed. No existing output is replaced.", + "timeout_interpretation": "Preserve all native rewards and exceptions. Legacy timeout boundaries are invalid/unverified and are not ordinary scored failures. Corrected-source timeout outcomes require confirmed process stoppage before official tests; no agent attempt is retried. Standard summary classifications remain unchanged and the final audit must identify source versions and boundary status." +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment.json new file mode 100644 index 0000000..7b11cd4 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment.json @@ -0,0 +1,1409 @@ +{ + "schema_version": 1, + "registered_at": "2026-10-24T14:06:54.025652+00:00", + "experiment": "pi-deepseek-flash-dev34-20261024", + "total_model_runs": 34, + "max_concurrent_trials": 2, + "whole_trial_retries": 0, + "minimum_free_disk_bytes": 16106127360, + "log_dir": "jobs/pi-deepseek-flash-dev34-20261024-orchestration", + "arms": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/adaptive-rejection-sampler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "95408217b9c484ffc18a02a91b7706323b6902919c72b24866a32e27a23543e2", + "environment/Dockerfile": "e644a3343e339303230f1441b7383f076d21c69a5a7aa6430b4f8a9d27f15c8b", + "environment/protected.tar.gz.enc": "7fc9bf1347bb97da8dede04b2c58638f225070913e222d76627068aebd1b1df2", + "instruction.md": "e6d28c28acd76bef61817427c74c2a29e3e8d75f535cbd708afcbfd02c54013f", + "solution/solve.sh": "fe443a8082b5183ba1623311ba0ee958888aefa633eff93766b9ae14e9bdd84d", + "task.toml": "6ed7b37f74beb46e9d47b0f5fad53025667c9548baff4e96d467b56d429fab1a", + "tests/test.sh": "38b43560d173cc2b952c3a3e17b8a480216d84e33450515e047bcb0d806b1e0a", + "tests/test_outputs.py": "547dc6e107f034f41703722aeceb6d0236e3fb69116fc3f2fbaba11884de352f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/build-cython-ext:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d5f5113941c3963292bae396baa9369571026f46e59437f98ddcc93475dd7729", + "environment/Dockerfile": "cb6cf421cb9b69d43bd25cd491e49bda982fbb865d47515178d017eda727194e", + "instruction.md": "7e52874e45f88505bd21540229719de6639a401477557d814489981d472f0e34", + "solution/solve.sh": "41bbdde88617070366a70e518eedca466a4ea1acd7e96899b2fb7da988e40abc", + "task.toml": "c472f0e7522c128fb7f03e00b50c8cdf2e5e6474daa1e094e7441f4b54e770b2", + "tests/test.sh": "0156c9c4085258b696ce424cf1ce96cba44d1263c5d73bdf54083fd80237f915", + "tests/test_outputs.py": "6549f1a8e04b6163ec08877e6f6b9868bf94563f3a3c72ac1fba9c512c31689e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "native_agent_timeout_seconds": 12000, + "verifier_timeout_seconds": 12000.0, + "docker_image": "alexgshaw/build-pov-ray:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "38380f64bc631916e264f80ce4d37feedbc783310001475b367d03989d114fdf", + "environment/Dockerfile": "44f2b5fc7311d8f336daaaca99dd4216b49355e67f50b494ebacf1b0f7c14e92", + "environment/deps/illum1.pov": "5a1f7f53ee0730c4923fcd21cd4a9c34719024d8557920441551ba7d96e75dc4", + "instruction.md": "56c8ecdd0084a07f8c97986c43af5c37ed03dd5aa994da6735cf59806162d1b2", + "solution/solve.sh": "3da9dbf589656512cd7a7004dc3752d3552a6af939cfe59eff79f91695fa42c7", + "task.toml": "b90a47ec26c1191d3c523cc094ea96b170e42fbfe176ef1f814c57d05390c097", + "tests/reference_illum1.png": "bb084f430bbc9d5b623a656920670dc06cee6d9132a9d550be4a1b99e41ab9e0", + "tests/test.sh": "928d1497ad21aef0c456294e3784f1fddf8493410f23584da89f9e02ec6bd134", + "tests/test_outputs.py": "64d6ce107503714ada346e5606832ee886be9950971cf81523e73d8ba28e8d37" + }, + "prior_nanopy_work_budget_seconds": 11820, + "pi_agent_timeout_seconds": 12000 + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/caffe-cifar-10:20260403", + "resources": { + "cpus": 4, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e918e6852c5c034422b81a6be22d7596fddcee53abe088f56727b2482da32728", + "environment/Dockerfile": "b3b8adc915a1afa035b604273af3bc1aa8b90999d2dfd72763218b34cb87abcd", + "instruction.md": "46907b0ee048f74269c29a15e0e086546b2b1220149597017da8b5ccc05a0cea", + "solution/solve.sh": "771887de1ed557269c7f3160c777519aa9538e1e9b59771b2f6ef38c8952490c", + "task.toml": "2831165ad9d9e8bbc646e6d85c0a2290640dcc4f13749547a2937c7675012a8b", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "e99b9f6284e923388fbc21488b1af99fdde46d4a809d3557b0788dbf1cb6b281" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cancel-async-tasks:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9adf86b1279cc44f83fe16e6d340595aa992240f1d87243d204cae32e453693b", + "environment/Dockerfile": "1ee7bdb4a7536f5f114c11a3ac6ff42790abfea901fcba6b2a8ea8685f19140e", + "instruction.md": "23d22f7039b50682941652c716f96dac314b8585c32c133614cac77dc067f47e", + "solution/solve.sh": "4938ca4ad5b7dcd36b414b42a1abb10c058ecefce7462b678f9bd05c9fae1844", + "task.toml": "14db0675e04408f2133b0067e8dbddf66a1d6e7baf6b8d38bef8ecccfc4e58a8", + "tests/test.py": "f533c0a64db068f582b4733c5b81f2e66b8fa52fcd3efa65752e6bd6d24780aa", + "tests/test.sh": "79600a6b863def31b4daa008297423010cd2d6243e689d67503e118e4dc53397", + "tests/test_outputs.py": "d6f510b51fafc81b4e9b5cc95535aa13491c8753e0e4171db42c45f4de460ee7" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/circuit-fibsqrt:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "201f7c889dd6f73dd83ca078e347272b33720fa622f8c757ea6d5770453f4cbd", + "environment/Dockerfile": "780c23986778f3e5f842567868af30b1135ed095ca9449e1c90b7f0e0483d3d4", + "environment/gates.txt": "032786f94f16605767c62bb61ce4dbba7ab7d3f1046712c975f01c9a04b6ad29", + "environment/tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "environment/tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438", + "instruction.md": "a7f500266a878d04ac9de39fba76220cfad216d8f6b8063bba93bebb3d4ab9ea", + "solution/solve.sh": "a4ffab4183815a0eb7646abc1de1fc67695014c060d7ff16e8bca9ec94385d0b", + "task.toml": "9f5aa1f7ebbbcec90fd109121e9bd509af3cfff6b4cec701bd91947988f40972", + "tests/sim.c": "700645ac6156cdc88ec1e210f7c7d8d9969bb42aec969c410597e8efca796173", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "6117b8edb7d567b30e32b16a9244bec4c131ec01e9065d52f6f53f87e36f0438" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/cobol-modernization:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "c89318cf59efcc7040e3d946022e2616161b6733931a080cdeea7b1918e7ecc4", + "environment/Dockerfile": "8b175c83fc03ec7d16d6101516bc5100230b592f480e435279f4fe051c9280b2", + "environment/data/ACCOUNTS.DAT": "46f63eb29442ebc6716bcd8a0c36f4367910f2783f6b84a307ed9a61ccd1046e", + "environment/data/BOOKS.DAT": "db04da646207cd87f8e9c5b6f0b57d95953899f3eef22de1a81423f06ef4a9fd", + "environment/data/TRANSACTIONS.DAT": "1fa4cc5420ded94d585c563e8196b85166ebbae2bb8924b20014717fb2fb9ee3", + "environment/src/INPUT.DAT": "52fc984f0cf14af1495285d7efe02b9bae13159eea59255560ee938c4f78afee", + "environment/src/program.cbl": "27cb0e56f65f52fa75f346720b37a180b8e034ebe7d7622b979f195087bd6ebb", + "instruction.md": "a87074f1141b7345394cd88053d0debb0dc8f7de95308882c6538ee9a3b359da", + "solution/solve.sh": "7d4a0504e1686c1790bfbc074d4e598f4c631f54c2bc5f3d5fd786574b4810b9", + "task.toml": "4b620211dc3d6b1b989f0210c5ecd2f606974581c8bce27bd78659d48cf0cfbd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "009180a7092583fc8143bb3ffc3957923aa0d61d0aac3a471f76eeb4aae54a88" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "native_agent_timeout_seconds": 1200, + "verifier_timeout_seconds": 1200.0, + "docker_image": "alexgshaw/constraints-scheduling:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "personal-assistant", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d54a690970149e1216bd4fc066b45069c7948c13e3b1ef2a213a74342805f209", + "environment/Dockerfile": "7d9aba072a4a907229728b9739c8cf821fc2c98450babd64452682caf3759c84", + "environment/inputs/alice_calendar.ics": "5a5d45804bffd339c2479d49e76fe817b38244d14d389afe1de583443808ecbc", + "environment/inputs/bob_calendar.ics": "c97e85dbcdd756758f8a80f4db5fd5c1dcacee79298e4d2262f46de5ff680b50", + "environment/inputs/carol_calendar.ics": "32a0f8687637df0a12b8e5c56d49c4192bc3e790856943632d70f237c7c1363b", + "instruction.md": "342def78177f9bc41bea4d11af679bc385a9dda2ac8a1e425be384a5b7f937b6", + "solution/solve.sh": "802d81ec064fad3050bbade95a2f847ac24e0b7f80fa69b0bcee0347dc92fe77", + "task.toml": "7b9842de16d74e8faf8c7c0aa03b0b81145941cb12dd322d81ae23fc48607250", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "1422b345ba3aacdfc4184e962811b6379988435896c50d8cf62903e70990c5d1" + }, + "prior_nanopy_work_budget_seconds": 1020, + "pi_agent_timeout_seconds": 1200 + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/custom-memory-heap-crash:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a06ed23a2c467aa36d0d4a88e5c020b1076cc25b8d483517239658e9c596dde9", + "environment/Dockerfile": "6ee81e5a4fa2540443d7f27d577ba840728fc6aab9f301ab7919103673c78080", + "environment/patches/locale_classes.tcc.patch": "16951f4a4df6bf253886ac75f5a39fdbc328e415194493f63cb7589a364aa817", + "environment/patches/locale_init.cc.patch": "e97f133ceebd86f8881f972e0e45b0ec0a13a9bfa428d8032008707683d54292", + "environment/program/main.cpp": "b94538c76887cc2b2ab739b5ac4731cddd0faa8f4a9892a6265c062bbbeede14", + "environment/program/user.cpp": "92b203786fed1bf0b3aa1675c9cd2d82b2651170675064da8ffee73b98ef14f9", + "environment/program/user.h": "f43d155995a56c22b9cd1417daef7c3b5a976cdc662a98eef0feb438ae1eee77", + "instruction.md": "53c68ae14a3c3af9a97c40bfd9bdc30a5d98b37eaa2f69f763475745f70f108d", + "solution/solve.sh": "b0b1d10bea1cd5be14a82b35d522df616df1e67028f90d2a6bef90aaa98511a1", + "task.toml": "ed474be1983e212913aa8866ebb66216acbe36aef2339ac375d232aa46617481", + "tests/test.sh": "70a2b55f0c5757f6b123a216dfd202be4a646680bced28dddcd31fd8959a058f", + "tests/test_outputs.py": "4e1cb33f777fa7fb5e1536f9bea9eace6deccd515c67d4f87570adbe692914ba" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/db-wal-recovery:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "file-operations", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "600762031a1a02ecc3b46d70f7683f3ff930b47c8a3d0e27cc31b146158c3126", + "environment/Dockerfile": "63a260531c62f4ae72060db17f295c0aa03cfc6321f8162c324b7aa6b3f81302", + "environment/main.db": "68cb288eb29e76d814b4e7ee6887b37c415675e644f88173bc191348e290d119", + "environment/main.db-wal.encrypted": "68753bdce26196a53194260d001544cc6bb8f9f38e2f2f6826d09798e31e6e26", + "instruction.md": "5a20375162921b8911c67035d1905df7fd335d1bed03044271e608b2c3c93b26", + "solution/solve.sh": "ed8530c94103bf19783a1d41bc63fe4db478cdd9629f22fee25c45a45699f5e8", + "task.toml": "1cdc2c615e259bc10265efbe71b518332070c1c002e226566399e9788f18d0aa", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "88dc5c2a3abba75bca0e0f9136223ae5bb585d5984b8d9eba51a7829c1d2ba49" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/dna-assembly:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "scientific-computing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e7855c23d9822f688eec4dd89ca0f1caf91c6a69e1c8f3c9eb544b4ab2a94821", + "environment/Dockerfile": "26e9e43a98f0b1d132bb638f44b7dc882b0d0ca32d56fa33048adeee00ab600d", + "environment/sequences.fasta": "ebdc361b5fba28eb0a2c7a8a8a444ccb12cc842c8cbc6df5a2c24254f130736f", + "instruction.md": "88998c98beabdd966ab46c04ede822bbc74796696aa1bf87c567a9131210533c", + "solution/solve.sh": "c976cb118c8ef1befd846ba05fd5cadb1b1f529bc76b9f2ea9cb3049e6e11a47", + "task.toml": "7b571544b64d9be1c91526837e5d805c2181afe0e6fd57a7f0676bba36c868d7", + "tests/test.sh": "271c8317c98539e3b49233d017fe51fb0e5fba8114e76830fe698565499b2b1d", + "tests/test_outputs.py": "5c2d0fd2e5f6303263e9750f67acda7db491ef04841832b4fc62bcfb61b08924" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/fix-code-vulnerability:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "87f28dbad183c3a634887696d328ca68a8a0858f132bc48cb7341c22267d38f7", + "environment/Dockerfile": "0917470986224df4daf4854a71302ae81c73ba5dd6e25be1232d29b758a5b3ab", + "instruction.md": "89b1abdf0af19399f720233b5cdf6b2e8e0ff7b6e3fe3e2a3af0d4d298a8641a", + "solution/solve.sh": "7a0131b369d83ae8546ecc3f9b4e23d38240f1bd1f0941e4430fa85b5182ee03", + "task.toml": "9aa2bd14ce9408289d199d0608bc3fa65b8b5e23374c9641020de62197dbf1e2", + "tests/test.sh": "f1611da693cb4cfc5e8482698c6a1858a5c67220ae9db89b50aed66584e9b17f", + "tests/test_outputs.py": "c006a5de95b69bd78055aa26cc055734dbd50718ede25d331f767bfee4ef7f26" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/git-multibranch:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3b9473429381d9eacabab498cb4a8915aa2bca24fa857ef862e5991d3f81f113", + "environment/Dockerfile": "88d9a9f4fef1ae7fddf632799dedf5a2fb6489460a93b610c33dd8b122982821", + "environment/default.conf": "a3a4c5d0ac9cfe79b0510e4225c80fa3988c27fbac13622505886bf66449e04a", + "instruction.md": "54014a38d7526d65faeb6c11d9b7ed3d4a5cb5d973d68d3da3b68b8c0289e125", + "solution/solve.sh": "300ca96b79db8d77f14742109aaf76048aafc4644ced750b1bc5c2761e8d0175", + "task.toml": "4b9981604578a31205257cca36a28d0c89c9d8d1058541330acd11d3f8d65f82", + "tests/test.sh": "ddbde3eea6ffa8fda105cbd86597a53bd409f41e13223900aa27e77a84b6b155", + "tests/test_outputs.py": "f03c6867663b8e8fb7899df51ee4df26411fab62ed491753964b801e591d16c5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/kv-store-grpc:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "51fdc2ad06f0adb1270164f247ea41799b2ee2ad15490e48ddbf5e79d3767aaa", + "environment/Dockerfile": "47c9982789092437f950736110b2bd18899c39b8fd20f3af3c95b848e564a197", + "instruction.md": "f4b3facc199d5b1c78db6e603ffa29ec1b93a77ff7d291e1a92b42e6bf582bed", + "solution/solve.sh": "6bee2938ac9731d1156b0ca02b28c3d5793bf2cbe5f995c863d397093af01cb7", + "task.toml": "6ff0f30a526bfff86a67b0b53439b9965229c0f368167dc1a5d05f34d82299a3", + "tests/test.sh": "c348467a076bf614d0a850e25b2263f85bbf7b4c7a8d344fbd5454a0611a0df8", + "tests/test_outputs.py": "115de2e964bbe689d07930527113422ef8a9e4be4eb958ad84594e3fcd2b1b0e" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "machine-learning", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ecc9dd635357c184742cf72fc1e452a8946e207b183fc9a7eebbcf07133b911f", + "environment/Dockerfile": "10a6193904ee0022ac098504182989893b196dfc74e626a3ca48534a8eacde48", + "environment/task_file/input_data/requests_bucket_1.jsonl": "8b3cf1e2b06074162038064dd2c114abe308acd32cec7d365ee63ee81978f7de", + "environment/task_file/input_data/requests_bucket_2.jsonl": "e15e2d4c4d55f875c42a6042610d6282f6c0657c39d3e0d0f087268252525cb5", + "environment/task_file/scripts/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "environment/task_file/scripts/baseline_packer.py": "547d230d5e6d93197803480cb81cab0ec6ac63fcfa0a24b246f549523f216c4e", + "environment/task_file/scripts/cost_model.py": "0cf417fd5ff0112defcd46970d37a290c0763c5a9092e91cb3ebb86c3f20e6ce", + "instruction.md": "0817374767533fbf51bb93345ee1bba266263c216c6cc55c096e71b4a1d37dfc", + "solution/solve.sh": "cb9efb586d76a893c3027e62029e286eb6c7583dc92dbf1840a3adc236a926d7", + "task.toml": "52aaac942f917e6a91ff658e0272195bd1cd842d41d71e302414c52603cc588b", + "tests/__init__.py": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "tests/cost_model_for_tests.py": "d3e810a99d40327fb8317f6512d879c52c6aeebdf5dcdc0c5e117c7df61c9ab9", + "tests/test.sh": "005f980332355e2fe143d4ff4aac11828adf070594bbe7055399c238f06953ca", + "tests/test_outputs.py": "80b06b7c4ba0db28832b767d48be74bb462d7f5df5798d9cb58c4f1160d96ffa" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/log-summary-date-ranges:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "026000415cb577f3d3d5132902670262f7d5d8f69841903ec06a882292f756b4", + "environment/Dockerfile": "b479f6e987b63a6ec118fa1691b8bb85c437b2a6a0fe63feab5ed7154957fdfc", + "environment/log_generator_deterministic.py": "beb6eff59979e9aa0be63ce8a0f1b0df1178f92871f04ba9f76fa70cf8beb81c", + "instruction.md": "c1d4516ad4ec2238209740eaf01291df9ab48b0028a3fc5df383c6f831453484", + "solution/solve.sh": "9f6a8ee8947bdf1c0f31b530b6194ae998d66841a3b785fa2e246ee840350ab6", + "task.toml": "9a9593673d6805ef3a301ea734e9c4dbaf80da1b37c0fe9d1f2ceea2a8c57e18", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2fc06a6ce1944ad82f84fb437395fa75ed8c0eadd385b92e375c27e5bc9cb2b4" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "debugging", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d6b48ed0890b12eb68f72c12cd02689bd6bfae2191dd2688949cde6fe77f6a94", + "environment/Dockerfile": "4a03ae6f0c60b9c2f45c33f0413b5aa75c9811870254e3ff1968fd0119173fc6", + "environment/task-deps/bundle1.bundle": "a759f9b6ffc1a2eee50778d72a1899ad7d724becab84fa30e7aac71ee9e443a5", + "environment/task-deps/bundle2.bundle": "75e3e3916a01c980a03715de67b16c729e03661d94a694b5f4223d229bd6593a", + "environment/task-deps/examples.json": "5426569e4ea078bbb74039ec712a2d4d35f1f9fa3a5f26bf2a0de4a2c8ef5db8", + "instruction.md": "2d826479873e5ac1c94c1f802763bf85c22bb04624384685ebdf6b153fe0278e", + "solution/solve.sh": "48d4becc41fda5b6063fe96947474ffe778a29d6924d7b2bb235eb61e5ce555d", + "task.toml": "6ced40e06b43b169857509765eb7f07791db558f9437cc7367d2536771c2ca5b", + "tests/test.json": "20f116a6b718bbf4874bbe203d15a5c0f85a1d88cb622e2d7529c7e4fb2d4c28", + "tests/test.sh": "c9052954cd6ce25c4a375a2fb48e765b1efa5b39f7991b93993cc793af8ab9d2", + "tests/test_outputs.py": "2754ba6c5c5f75c9db5ed2c5729d9f5d7e21cb7fdb18aa0c65abdc3e7e85c149" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/model-extraction-relu-logits:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "mathematics", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9465accba6c57e5de9a196eed776a16eeb0852a8afc18fc3f8979e2de71b0c7b", + "environment/Dockerfile": "598b1f80d3b7194e97cbf02c8d245de30e303909e0cbe6465125d1d3179fddb5", + "environment/forward.py": "af1a23efd8e4390d16a29777c28116b2f8623ae3f7b628fe14db9f94dac26f5b", + "instruction.md": "c9e179dd099065a34c362a865b0eb9fab5f7d96be582cf117e7519580c036ebe", + "solution/solve.sh": "4cea5094f074ff21453d4423aa1b415a104ed90caa55a4acbb4f09941a3994ea", + "task.toml": "eb64dfaf8ae88868911ae28855cdc4f767de464001e2704919015b595be6dc29", + "tests/test.sh": "3b2d8f5e0353c2bab3515d6387468a863a4f9f241ee131c73f545f3fdf9e2d52", + "tests/test_outputs.py": "e5189c7eb9942ede55e1b48a80388590b80a486b75497613f257b3373ad9b2c9" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/mteb-leaderboard:20260430", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "ed147a5bc31e9865b16848b2f01d2f9b1e677af57a7c632d7ea2311c0553ca76", + "environment/Dockerfile": "271fdb5b34520c0bf2cb6462365363ec52058f127cd9eea3f5da2589df5c1044", + "instruction.md": "06ea44106eca0405c25bab00f5b3cfeb611c872093f6c0f82ed057839d1d13de", + "solution/solve.sh": "4ed68831d4acf5d80d2d73eec1f21efd0896a6f51f23020f572cfc0af6da0410", + "task.toml": "86574bd1c6c4fefd2e64eee9c2caa53d87c4bcf2e51547912440a0ddbd0badb8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "b28f8fc8b7614801305813c402be52f4be1e75de6658ecdfdf9383aa793892cf" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/multi-source-data-merger:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9bf37162a4c1b88ed93627830bb8512ebae154325d7cb069c883144a0e4c69d6", + "environment/Dockerfile": "c95a40d3d35b473dc15ae1e8784ba7fb1725805c0583fc574dd2764325d75e4f", + "environment/data/source_a/users.json": "75e51e2d3ab0bdba2269d7cf3757576f84f3ee2c13407d2db94289a5958a1778", + "environment/data/source_b/users.csv": "8e87d16dc47d166edd8e56165955bad265b630d53d02762367c816affb5fa760", + "environment/data/source_c/users.parquet": "23d1ea51c4170bdba53febb885a0f495e28de525a52667347b2932136b9a72fe", + "instruction.md": "d1d28bb6e37f7a255a1b7f78aae8591f35c0494f33e4913392424ad954aab944", + "solution/solve.sh": "1b3d0d50e31930d042495d48f4c8e95f1a9c1a979c803b7f7cec651322537e1e", + "task.toml": "ee3964c0ee040f0a0cb121cefefbc15443d127684acb003fa5b61dd42f31c694", + "tests/test.sh": "d5a00e64413c0e1ae452f38a1aa1a47c3ed7b4416bbfccae1a4c053025a336d1", + "tests/test_outputs.py": "a070888930cf829aeca039771260b1386bda990ea805f313468d381ba3608ca5" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/nginx-request-logging:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "d79997586d1c14ed8dbfea050e3580b25500e04c8345acfd3f294152433c795a", + "environment/Dockerfile": "43003466d89ab83484a3fa618e35b58cdb74a002b1e8716ef30f12b3ac8302d8", + "instruction.md": "26a7ac98aced6107f147206790ada77bbf8ed3c25fb360cf323d5fc6889edf99", + "solution/solve.sh": "2454bbbf7c201a9a778165f286373ba2cb991a0eb1d026a8005d57ae71250dd9", + "task.toml": "b7a78d00be13656f75069197100b7b8c4fd268decf6f78929c853b3222a2729d", + "tests/test.sh": "27761195f53cbfe5941fe39b573631f6c926869e64b7c43220377e1e116528e7", + "tests/test_outputs.py": "045cc716c14efde3b0dcff5fc7c85ec5d18bfc6ce66f8b40a418fa2a3a4acda0" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/openssl-selfsigned-cert:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "security", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "5dfa88651890cda06fa2a25babd3b813818767635e88bbabecc8305d5bb3942f", + "environment/Dockerfile": "f590b523038be43afcaf5291d7b9e8ed1333c48d9994e77b8dee2f2c352f0f40", + "instruction.md": "38054a3371f7c2c853478a2f9af0bff858437d5b28a9a07483238f06e566921f", + "solution/solve.sh": "58cf3c844e907745cd81fb4bd75efb3d080e2f0e82b589c56c954ae6e1756510", + "task.toml": "75c8913f0fa15890786c11999cde920b8bfc10d4ce8a14edf1e76985a2ac5800", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "90970bb3c372e8e7fb6bb9ac5f649ba73eb38e1a5ea2e922a9057a72b6b4192d" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "native_agent_timeout_seconds": 1800, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/path-tracing:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "3e4fc952b3f9a860c7ee7837b3f3c686f1283ee7821b014265dba7c94d7fc849", + "environment/Dockerfile": "7aca02e4e20b04ed2269d627a29dd4e7337a02764a9652b641452bf09836a3d5", + "environment/orig.c": "665c1bde0907740890ac114d6b36af1f3fd00449fa287583817f778bb5da200c", + "instruction.md": "9f36e02d636f5dc0946619bc518f2e51c24fa75251da9d1f0ca9eca612cb9572", + "solution/solve.sh": "d107abde101eecee6838da80cc52a3d480c93956fff3a9343a299ea01b3fe9e6", + "task.toml": "cf47013f69bf1b43242da984f231d4daf364fdf44d3845f2eea2ca2cb9308c46", + "tests/test.sh": "a719dbb1ba7ae76df095e9d815941dfa4c55edaf11f173b9c3c3097afa838548", + "tests/test_outputs.py": "572e68b2843191d29592faf42a405861b7958d8e894868b74c5a8886b53eb879" + }, + "prior_nanopy_work_budget_seconds": 1620, + "pi_agent_timeout_seconds": 1800 + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pypi-server:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "906a096a2cc7b9afc738ff5ad5fbd605be65c103f5069c6673915223b9b187e3", + "environment/Dockerfile": "17a4e67580e4f2c6a151d04b9d74eb020666e7eec8e5259c4a5411f9e93858d0", + "instruction.md": "1c537e4a311a8e71155e121f57bfc03f742fbfb99d05314cd03e433996a28a31", + "solution/solve.sh": "1143c0eade4932ec3aeaa5013fb5e23bbcccca15dead294aa03b6af88750775a", + "task.toml": "07c39d7a733c89adadf74bb63a3e906c3694feb6c490d5fc85ac187c003fd447", + "tests/test.sh": "d2b1220e42bcfdcce87907ac5bbd69ac210a3c94926545ca2439636786e7784e", + "tests/test_outputs.py": "82c6351722e9aa984ff1d701bf57df3150ce51466df2b70afa1aa62d2650be4c" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-cli:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "35a093fa5f1592737784ba718c2e3e9ae17ff8b5e53e003fae560ddce091039c", + "environment/Dockerfile": "8fd5c0f21a6914cd440d10c84c9db06fec606f20dc3e0363b17a3b8c220dc4d9", + "environment/task-deps/cJSON.c": "0e7995261640c96db16ed2c91a5d981efa917cf5e72d0f97ec45a1179ae3ad73", + "environment/task-deps/cJSON.h": "0909160541bcc510e0b2d8c7a9cc051fa1459a0f13be4d8e56346a15491f2f2f", + "environment/task-deps/image.png": "f4d5dfefe463939996541404340e7099d5957b56e1df3760d13b5d2de754db53", + "environment/task-deps/lodepng.cpp": "67873c5f6272f779a143a9c81070e8783327c5552d0b9de2aaee2cb779308917", + "environment/task-deps/lodepng.h": "b735b7e0357319973e9488d520d7f270d734a6e9e1bf924fb1d5b8d26d7e97ce", + "environment/task-deps/model.pth": "a32103eef09936e586b1a7dc7e64d279500a38ff30fdd6df2167e28eb954eee6", + "environment/task-deps/model.py": "e4d3a5c728449656d0efde7871ee227619348352769795ed1be1a8b7f835fddc", + "instruction.md": "ce36ee1f6c589f81ee6a6be4967e25c76a2ed697006928023f42dfb9f28a6750", + "solution/solve.sh": "3b34e561c9ef0a53c9948655847846eb4811aaac67b990480aa6aac1b04bfe66", + "task.toml": "fc1c018efbd5d0537ac0741d53d1ebad3886faa51d14cbde8ec953856411bc0d", + "tests/test.sh": "43fb67e537c7adc2df11c2367f47c876cb5d16a7c1ae16d0339c3a3364bb2b06", + "tests/test_outputs.py": "370702c57a4e0c85f70ee37a5c357ae3a5df8480a275bb6263b0376f2f6c093b" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/pytorch-model-recovery:20260430", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "model-training", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "e080308a05703b40edd16f53aea41a0e6b9874750ecee35bb432029d0d44bf53", + "environment/Dockerfile": "8a6b7edbb9e7c241e447f15a71e13e5d043b25169be2fd9802b84f5be3d56eb6", + "environment/dataset.pt": "6db65e4ec1c0760209c2dd997ef582e4a1519ab38e0312339999c460a696960d", + "environment/weights.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080", + "instruction.md": "2bef07dc28be3ab2c35ae389584034c2e4dde92c2c31b3eb6bd829628c8c3f4a", + "solution/solve.sh": "175a7802777b43e0fa4f629d71f7c68fb40706813c342aaddc72108d89bd5b2c", + "task.toml": "851a51204f53482a31f47cdf8833128cf421113b0331011286940fc857c37dfd", + "tests/test.sh": "6005f3a75ccfe00d7bed1005264b4a84a1f8e30cd780f8a97b3a804be7cc7a8e", + "tests/test_outputs.py": "5d4d57c43e6913bf3d0081da764ad36d1b7275e3bb114a05abda529afe563905", + "tests/weights_gtruth.pt": "71471e71c5a311a897362dd5b77413e7ca2ca59abd00a110bdc2cd3f8584d080" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/qemu-alpine-ssh:20251031", + "resources": { + "cpus": 1, + "memory_mb": 4096, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "system-administration", + "verifier_dependency_profile": "qemu-alpine-ssh-v1", + "task_file_sha256": { + "README.md": "72f1c83fa1e3e6381ef85dd82fd109ee18cdeac090311cf3212a0a8cff0a32a5", + "environment/Dockerfile": "015ce0601ff54758f66b33f4952e0d3a577d84c09aed8f3ec5945a570e8d168a", + "instruction.md": "8348c60490cb90bc690cba259dfe98996da4c3294c6d42bba1119b00c64de4de", + "solution/solve.sh": "1cd31186c427b91bb00499d22aa4f0214fc1ca93b40b6e4cfa969611b848db2a", + "task.toml": "10a43ec6ee732eeba852c377f12e23178f7fa5a4868143f3699a15565ffb64c7", + "tests/test.sh": "3b244479776c4ee3639631112799f2cde22cbb80ac10511715e3a8f32909c289", + "tests/test_outputs.py": "83ccb4bdcd259b342212bebefe395f289858b4dc55b2ee7ae0dab238ad2e9718" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 1800.0, + "docker_image": "alexgshaw/query-optimize:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-science", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "a82c53e333a0ac0b0b410b5e26cb8e7305dece31db84d340248ffb13d5a45160", + "environment/Dockerfile": "064818771f200e0517d22068f87229a7a95d0a82b0778c84ca7203bf1926a0b8", + "environment/task-deps/my-sql-query.sql": "43382792f0f438625d643e34abe144febd5669518f4ab1ab550a15c6d9314a21", + "instruction.md": "473e12d320d2732f679fb631c2a4dab4a01deea6775686718ae0c417b8a39447", + "solution/solve.sh": "0d95f3cedb287ff9d47c796cbaf812c300d1222c10fa80c49e8acb0e81eb6978", + "task.toml": "6e88f2a86df699594bcb1bf81df4ed694505f79c9420f671335b02524bbe3868", + "tests/golden.sql": "4da1885d56b3ffa7b7b22b422996d018795611f6963499d551582881b74b84fd", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "2068ae80c57af8d52a7ae510f0666732c057806e556b480970f36f14bae683f6" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "native_agent_timeout_seconds": 3600, + "verifier_timeout_seconds": 3600.0, + "docker_image": "alexgshaw/regex-chess:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "9296cbee592fc996c58d228b51fee303647e5ecfae82d5666d42a40cb30ec355", + "environment/Dockerfile": "2616cd775252303839dfda0ee2f047eb862952b053b0103bd6b2460ef90c11b7", + "environment/check.py": "f9c416ae9ea01dcdd5f6a43170c8434b8c0a0b389a4a30876a2048e277d191c7", + "instruction.md": "a6ed3fcc54fc92d71a84af921942422d299ca89b410dcd92cd05a53d0d06436c", + "solution/solve.sh": "b728bf0ea392a3d244afbd0818d36933c8d3e190e47743c1382cd518d518a2e9", + "task.toml": "07fe52636fcc4bc9fff25d5630af145b97a5331219d719cc8decb4ca67ea21d4", + "tests/test.sh": "8f7f00a1c4dde7fcca6c918c768f2aba432b8c4007c15989716c68091950da87", + "tests/test_outputs.py": "4983c44d21c56dd8191381eff0304afc8b87bb714527ea9036b3319c015a30b4" + }, + "prior_nanopy_work_budget_seconds": 3420, + "pi_agent_timeout_seconds": 3600 + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/regex-log:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "data-processing", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "29ffd98c55dbb6f4844f1b9d8c1769b2155c117ee475e0df6bbc5ef330d239b3", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "4f7ac05e70cf9220ea0f1e5a052c5f908cd0fa884e847d80b0bd51bae2e96f9c", + "solution/solve.sh": "7e670d4f2b4bccb1e4db38f2a173e085ceda028c38167912b466b0a84fcc0999", + "task.toml": "6bc98525852755caf1fe34c174c2f10f4328adb4ce6c1968dcafe1c877e4e160", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "345c3bd09ab6f6fe8c8361a58c0a47bf0a13b3fcb38a5ac7824e44ff855e8f72" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "native_agent_timeout_seconds": 2400, + "verifier_timeout_seconds": 2400.0, + "docker_image": "alexgshaw/schemelike-metacircular-eval:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "095d3ec6dcc0fe0c7a7e92355456dcea6992f91d1f8d397669b8cfe18475f506", + "environment/Dockerfile": "9b3df76222e16aad360fbd34cb0f71a0be8986052c065b10b92c0cdf417b25c7", + "environment/tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "environment/tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "environment/tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "environment/tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "environment/tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "environment/tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "environment/tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "environment/tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "environment/tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "environment/tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "environment/tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "environment/tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "environment/tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "environment/tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "environment/tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "environment/tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "environment/tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "environment/tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "environment/tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "environment/tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "environment/tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "environment/tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "environment/tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "environment/tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "environment/tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "environment/tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "environment/tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "environment/tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "environment/tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "environment/tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "environment/tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "environment/tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "environment/tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "environment/tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "environment/tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "environment/tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "environment/tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "environment/tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "environment/tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "environment/tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "environment/tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "environment/tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "environment/tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "environment/tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "environment/tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "environment/tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "environment/tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "environment/tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "environment/tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "environment/tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "environment/tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "environment/tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "environment/tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "environment/tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "environment/tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "environment/tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "environment/tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "environment/tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "environment/tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "environment/tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "environment/tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "environment/tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "environment/tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "environment/tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "environment/tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "environment/tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed", + "instruction.md": "3af5f9567b62f93683db7518ec6fcb609f83796258aa02166b5a7091c5b69031", + "solution/solve.sh": "e2c5b0ac2bef4801a1891e13c108813381877426c1a7f01bab86b4fc1f712510", + "task.toml": "8dae8eb226183b0ca92331c1106152a7abc93b03b4e5a95291b86a4ddc9b44e0", + "tests/interp.py": "efb8b5cdcf78b73afcf39d59600a1f355ca89b7bac9854bb01f110cc77bea2c1", + "tests/shadow_test/01-factorial.scm": "540e81fac476f04dc510ba107721d606ac980760088b2136889e7d4bf884e5c5", + "tests/shadow_test/02-fibonacci.scm": "628957171e0c4a3ee373635923fcd29e75bdf4bf13a402ac26fe94a5fc1443ac", + "tests/shadow_test/03-list-operations.scm": "fe8f17f59fe4045582fce363d52df95ba7c4de7345542c8bd4993a9f2f396235", + "tests/shadow_test/04-higher-order.scm": "9c2eba8c91bb7b224de56d789207d2eaad98bbe1f8c360d6eca4f014f42f9230", + "tests/shadow_test/05-simple-io.scm": "1eba30db30824fcf6048ea8e49b84e85c866cefce30862fce49b300e4416843c", + "tests/shadow_test/06-interactive-io.scm": "4fbc0e2d6e42c5a9495e549ccb13c02fcaeddf8fdaf8d2cf5de833116f12610a", + "tests/shadow_test/08-progn-sequencing.scm": "3bf8236dc5356d29ab8dbfe1b0f4c519354fbb23c8f3eaf946779c28b80feea3", + "tests/shadow_test/09-mutual-recursion.scm": "3bfa1f77e602c7039f8c8c70315e6d4fff6eed737dab0f0398e5bdf690137c2c", + "tests/shadow_test/10-advanced-features.scm": "b69ef497088fcf0aec388789a17561b783b8cdd5915719915a51c5ba189c31ff", + "tests/shadow_test/accumulator_patterns.scm": "8b9192796a6f9c3155cc5e44d0a0ba6a46d20a0d94e265b69b23ec8af43e4a07", + "tests/shadow_test/binary_tree.scm": "5a3dafc8844969cebb5809f253f4e54de7df480f378da8197f870badef03ee47", + "tests/shadow_test/church_numerals.scm": "31d075785f2f605bac3bd9ac2d99fcf7eaaed6686eb9afeab1d40bf5f35bbf2c", + "tests/shadow_test/closures.scm": "c7d025a376987f5ee22f529847b66eaf53f0af7033acbb4055cedb734168d612", + "tests/shadow_test/continuation_passing.scm": "79e7445c42227846f1adfaf08c17e278aac7a34df77482900b068be4fd248080", + "tests/shadow_test/currying.scm": "690a8cccbb1d249c8f6fcabc0193cecc5d0e4c1af39a0bb67c7e95fcff8e393e", + "tests/shadow_test/filter_operations.scm": "485a0563b81d002e368c5e4b2b7cc98a5306163701053630f416e3b783a2a238", + "tests/shadow_test/fold_operations.scm": "461b33db6b3f5580122d4c6dff0fd6d11c7f044865ffd684ade67d9b296a5a24", + "tests/shadow_test/function_composition.scm": "65ceee00188b12c3549134449e2f938d010c2c97dba090f5b53244abb3579b1f", + "tests/shadow_test/lazy_evaluation.scm": "edd95748de14aadf916c65209b5bcb8759d3e33b7ec935289a5128a656d33bec", + "tests/shadow_test/list_operations.scm": "660963851ecb49639f66557bd88db1e73db323186742145f33cb2e14d3af1103", + "tests/shadow_test/map_operations.scm": "974ac42d892f4b62b9b21ef7844581624dad811b38d35944f1005cc47f249856", + "tests/shadow_test/memoization.scm": "25482c1912fab1235651b1b7acd727f2ea6a55dc5c1919f841b44cfc32107b05", + "tests/shadow_test/mutual_recursion.scm": "2fbc9562e863cb2b325f86419fb2a7588265aaf5576295550722ee4274663712", + "tests/shadow_test/nested_defines.scm": "23ad3b2f936949501466da784511ea2e6215e2b198ca371bef86f6af877ca923", + "tests/shadow_test/oeis_sequences.scm": "1b586df6af2429a54808f7587f60021ba2b2c9bcc2ef2f9cbe44339699853c62", + "tests/shadow_test/oeis_sequences2.scm": "300da0b0dc20b883e01142adea9eb6da1a583a1b969ac0be47aa32c1ab66340e", + "tests/shadow_test/oeis_sequences3.scm": "dea198e874bf518b93c8406a843b64c1586cf4ce780fec8c444f8343e9322c75", + "tests/shadow_test/recursive_structures.scm": "64f1a430bf04a627f5e27bf004805544b60406f8a123d186516774dfcf3846cd", + "tests/shadow_test/test_read.scm": "fda448c22685e2f9af37ff7b986ee5e906d485a2db16f62eec113149f866870f", + "tests/shadow_test/variadic_functions.scm": "ac940ac0273e87ee06280fee192194ab5cdc14c88b61da9cfba92105e66b8f2f", + "tests/shadow_test/y_combinator.scm": "38d25b02d54342ab5610ad03fcef68c5f1d4eef8902295645f6acceb853c9860", + "tests/test/01-factorial.scm": "2623000a742c63d50ae1851854826247caee129c7d1613be6c7131d21f26a6ad", + "tests/test/02-fibonacci.scm": "db769cef446d5da9e9b1abccdf9ca644e239938095a98c21551aee25255a57d4", + "tests/test/03-list-operations.scm": "1a14b4100dedef30010fed62ea9fd295add1b0facb2e8e6e27c8d8eb3d90111b", + "tests/test/04-higher-order.scm": "59cd17bbffad641a37729b17c5bf9178b8bae7289fe89cc7135eb0bfec2b4fd6", + "tests/test/05-simple-io.scm": "3deeb3275df1b771449ebaebfcb9a934f375415e138c5e9f3685516c69a1f7a0", + "tests/test/06-interactive-io.scm": "aa3cae79b79dca90ac9d61a1babf64d6a5bb6559798c7727c11be02d8ddfd216", + "tests/test/08-progn-sequencing.scm": "4c0a4d980cf4223d1d17715d58adbe27dc120eced257a0d9f42de952c094bc5b", + "tests/test/09-mutual-recursion.scm": "939e18f4bd25b5ab0e2a4a767bf7eb6b0d8fb3530d28bf236f06cf27f2d90ee8", + "tests/test/10-advanced-features.scm": "c4cc5fb412df363875768c6a1e645255f6bc8f9ddf988d1ec2f4a266cedb8ec7", + "tests/test/README.md": "1f2174c7f12721f9f1898cda190c2886fb3ef9d35fd085f315da5814657104ae", + "tests/test/accumulator_patterns.scm": "50d246e00577d1f3ab3f0462c2b9618ea63680c33fcca12369cd1707b010cb5f", + "tests/test/binary_tree.scm": "c95f59ccac1ea4a9c846afddc993aa40ac081c0f9e6731544530109b269b67ae", + "tests/test/calculator.scm": "a09213ec8a2ec52486b9346c420ddb7c537576ef71b1cc5c6ec61d13d9139f61", + "tests/test/church_numerals.scm": "f37d858f32a9c5ba8868d094018c92516acadf0ea4e55b7c36f14320cb0c69a6", + "tests/test/closures.scm": "3ecfc35289479288876e20bf7f55e5b8eda84acd35937e89e3d062308a99aeff", + "tests/test/continuation_passing.scm": "01049581e582f7d35ef40ef68e1827d012da619beea57c1c74675a5f245318d7", + "tests/test/currying.scm": "2844d54eee9a94af05a11b4aae06d6e7823d193f17235c8f90bf98926cbca75d", + "tests/test/filter_operations.scm": "f6bc75f6da9ece96c0616e3570e4df59d0f2eb4655375d3a46b0349c86cc5dd8", + "tests/test/fold_operations.scm": "05aab492ad914baecd19cf454ed587e7a031e94b8990500eb1c869c15da6fc0b", + "tests/test/function_composition.scm": "9b56dc78823e7d35adb213744c0f8e95684074d6a12bb98c8c05755788be5b06", + "tests/test/lazy_evaluation.scm": "ea5912a53a35b65bf2bee9483dca90814b8baf52ae3ae50c2d4589f794dca54d", + "tests/test/list_operations.scm": "d7e70621afbbb787cf646e2089bccfbd3681fbb2e70325f3e50f56ccd30dbd9d", + "tests/test/map_operations.scm": "0cebddb3d4a50e4b6f6bfc0e8c12762d3f42609030aef077d8ab4bedda4718d9", + "tests/test/memoization.scm": "cf8202df22407dbba72ca7f660d8ab7cd4ca962b215c36cc491d5c1a474c160b", + "tests/test/mutual_recursion.scm": "a3dc23a96ec1cf05b0be4a4f0b6d3da9e2bad1aa3f7a83c7566f71f980a67fe2", + "tests/test/nested_defines.scm": "21faa0ccaf744cb8c9d283bbfd0c36e195c803c85f511e70a99267c0a85b6216", + "tests/test/oeis_sequences.scm": "d1f77161b367176b3dfab55a34466e511c602c7ae359e05f5a715627b93dca6a", + "tests/test/oeis_sequences2.scm": "5679ef0f99743114d8c06edcdcde1fea9474b71aeb57c2069f1ee33df1970bce", + "tests/test/oeis_sequences3.scm": "5dc8b44158b2648a6e9766f6bcdf21553a07bf487d79e64ebb418511ef3b8186", + "tests/test/recursive_structures.scm": "67838ff5bac26ea359139720adec1d5700176147b0062065c7def9715e8a2aeb", + "tests/test/test_read.scm": "5b9ee381c3cf45b2fc8c2e7b490857fb67f1ac78653b586d7f2848daf6853809", + "tests/test/variadic_functions.scm": "c9b847c50840c6977904e5033a07f6108cc3a54012b02d916e79025fb43e0f6e", + "tests/test/y_combinator.scm": "971144dcb2b04aaa3b34d9279459bd7d2e8256c585d17809465848c2589d36d8", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "fbb84e3c0c2f72ed7de90f54380f860052c16155e7d0c9076cd57c966e9be7ed" + }, + "prior_nanopy_work_budget_seconds": 2220, + "pi_agent_timeout_seconds": 2400 + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-pipeline-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "297ddea07c230dfd246a8ab07337dd7abcd8e5e2a2b2e26590cb0a8af779839c", + "environment/Dockerfile": "c357ea41e4a4013c5c5980b8fef04af6a5ae250ccc6697cf3bad3127401a931f", + "instruction.md": "989bfcf0c54baf6c8cdc391743dbb2f9e858c5779dd85112536875d4d1b2dcd2", + "solution/solve.sh": "787198e4178d88b887697aefdd29819f29435ffdfe356545d92f58a30fb38b70", + "task.toml": "8b4f9f525c08afbbc4eca9168e46238a61ccb903486773e9ca4da64bb7ffe857", + "tests/test.sh": "cc9fd28b6590993fc37e4a99da771cbf04726c8c62fe308e802612241da69dea", + "tests/test_outputs.py": "23574246a407163a8e6581c84287cc2f1eb745efbaad2a317a5a28c39958c64f" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/torch-tensor-parallelism:20251031", + "resources": { + "cpus": 1, + "memory_mb": 8192, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "4c1b3c3be30548f1f94c6f5503b1bc74b006af7f192dc4cd026f411df1a54604", + "environment/Dockerfile": "5f264ff0bf62a49aee3c6398447de5108df141e7a2ad35c6de8ef39fa62049c6", + "instruction.md": "13bde2ab165d72a311e2bf066d5d6c3c8761bf9d3dd39d1b23eda23bc546b910", + "solution/solve.sh": "d083b7cbf9dbb6cc2ff38c8f2acc150f0059c04aea42d6c2183aeaecc5a19385", + "task.toml": "72fd2a5500d1fe8795dcae76cccb1a29fb25894881a7475f2fa519c87d425a8d", + "tests/test.sh": "a7e6c9408b2401e4e7448371f7ddc72f3284caccfb56277d3cd4e98051868cdf", + "tests/test_outputs.py": "4ae4285e15fba1fcf13d7b7c0638ac8c2e2b0797c937abab1fb6b136fd508290" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "native_agent_timeout_seconds": 900, + "verifier_timeout_seconds": 900.0, + "docker_image": "alexgshaw/write-compressor:20251031", + "resources": { + "cpus": 1, + "memory_mb": 2048, + "storage_mb": 10240, + "gpus": 0 + }, + "category": "software-engineering", + "verifier_dependency_profile": "not_configured", + "task_file_sha256": { + "README.md": "603015a28500a28f0838fea2c43a5168613746bfd07737920f4bae40b5a326f7", + "environment/Dockerfile": "9a583d29d46818f6ff2be7c6a0879d03343106e5376fd5a48eb5926b402bb93b", + "environment/data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "environment/decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "environment/main.rs": "fa53d6572fca95903e30d2d953f1e428bb8b63513ff218ebee9bc60a425ab389", + "instruction.md": "c4fc3e73e44deeac95f1cd4a34a2d87e853b22ceec2b2824767f5f087b470bb2", + "solution/solve.sh": "01550b7de7ab1320b1898eae055226bb7c73bee259c75102d2240c974a730409", + "task.toml": "6066a509066433c97c417fea68c4e3e73996f698dcdcbabe07718835eea355a6", + "tests/original-data.txt": "0fc483d0fb31128dfa41ed11adca56c88688888189826ef9daeafd7559b4fa4e", + "tests/original-decomp.c": "7f8d9ef66943f912a217aa48f4ef5efc967b44c2b21d0f56df029b5ab413f62a", + "tests/test.sh": "4770437ea96c3cc84684b4f99d55fb148fcac09f9ea1e8ef49de487716e6c334", + "tests/test_outputs.py": "809f486053e3ffbce852111dee1f63d80ab31297bd33172389e50278bed384e2" + }, + "prior_nanopy_work_budget_seconds": 720, + "pi_agent_timeout_seconds": 900 + } + ], + "preflight": [ + { + "arm": "pi", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/install-only.json", + "config_sha256": "246429e25d0b54aff9db85f2f93435e80e714be666595b2f4e7fa32a14f11795", + "job_name": "pi-flash-dev34-install-20261024" + } + ], + "queue": [ + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/adaptive-rejection-sampler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/adaptive-rejection-sampler.json", + "config_sha256": "e737e23d0d9e761c13d934411a79055c35cf4a7c13502d79717dca7ce790a9a5", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-cython-ext", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-cython-ext.json", + "config_sha256": "da458c548221b2a1066d4aa30ddc778ad6e3cc3c33c8602e2e1a9cb12df6fb23", + "job_name": "pi-flash-dev34-build-cython-ext-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cancel-async-tasks", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cancel-async-tasks.json", + "config_sha256": "cfb498953fe328ee6f005d8783e385616b69c3434fb1c8ecd1aab178bd4c0dce", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/cobol-modernization", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/cobol-modernization.json", + "config_sha256": "96d36cedda0a82b9f07a76459212f7ed0c8488b71092ae684773312c4b3917b6", + "job_name": "pi-flash-dev34-cobol-modernization-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/db-wal-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/db-wal-recovery.json", + "config_sha256": "c7942215a12854d5f9ee54ca7340ba3f9e89190856efab054742bf2fdb4b305c", + "job_name": "pi-flash-dev34-db-wal-recovery-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/fix-code-vulnerability", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/fix-code-vulnerability.json", + "config_sha256": "6bd358f64d0b58b38784faac7b784f678bf5d679f25e44759fc0440e57ab1651", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/git-multibranch", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/git-multibranch.json", + "config_sha256": "ec21d2f29cc528fce2f36897b2f1a82262fbb44a818e7195856d791725d5e928", + "job_name": "pi-flash-dev34-git-multibranch-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/kv-store-grpc", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/kv-store-grpc.json", + "config_sha256": "3251f511205c9f21a6390e904392661352addee3db92d8498c6d1fb90548b4fe", + "job_name": "pi-flash-dev34-kv-store-grpc-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/log-summary-date-ranges", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/log-summary-date-ranges.json", + "config_sha256": "7c859a4ba515eb7a3d4ca332b5a72acb099e4d8146c7412b25be532084068351", + "job_name": "pi-flash-dev34-log-summary-date-ranges-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/merge-diff-arc-agi-task", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/merge-diff-arc-agi-task.json", + "config_sha256": "44fb82b39850f58dfdb7d3d4da903f043dd37e5c83bc48312798612870df1f92", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/model-extraction-relu-logits", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-extraction-relu-logits.json", + "config_sha256": "89451efa0da9e693d359be60cbcfca12ab8148c5d081c6da8acd2fd6f933d3d5", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/multi-source-data-merger", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/multi-source-data-merger.json", + "config_sha256": "c168fada695ee17b8da44d005693b0d701be6cc843dd1a037e4feac9c24621de", + "job_name": "pi-flash-dev34-multi-source-data-merger-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/nginx-request-logging", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/nginx-request-logging.json", + "config_sha256": "6a9adf41ab2bf874c0ea037085d9ccd42573589b7010d5d3a56f0b8fe9b356b4", + "job_name": "pi-flash-dev34-nginx-request-logging-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/openssl-selfsigned-cert", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/openssl-selfsigned-cert.json", + "config_sha256": "f9f0ec331c08d2db293109d7d40012f92249cfc206b0873b3ae9c0230089d813", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pypi-server", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pypi-server.json", + "config_sha256": "e4238419099c1eaf7da9f3bbcf8cc72a8d382e1a27ce2bdc95182c62af110943", + "job_name": "pi-flash-dev34-pypi-server-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-cli", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-cli.json", + "config_sha256": "f84653ea98195696e9377fa5504362b20a7ebab746e4a44eac469133399d1226", + "job_name": "pi-flash-dev34-pytorch-model-cli-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/pytorch-model-recovery", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-recovery.json", + "config_sha256": "ca8d05456eefa36a767610781835c0e3ebfa6a587c12aa814ad4a940d507dd12", + "job_name": "pi-flash-dev34-pytorch-model-recovery-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/qemu-alpine-ssh", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/qemu-alpine-ssh.json", + "config_sha256": "eba0397cf677116056fe2c64cfd3883252288c1263ce2b65a9c51657a8de9f9c", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/query-optimize", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/query-optimize.json", + "config_sha256": "9f33dd57e5beba6e14b9ff9e1e163d1bf9cd6742402b386aaca792965636c482", + "job_name": "pi-flash-dev34-query-optimize-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-log", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-log.json", + "config_sha256": "b150765adf9dd3eda13a9befc40abeaa807479653f8df9eb157e894283e23269", + "job_name": "pi-flash-dev34-regex-log-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-pipeline-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-pipeline-parallelism.json", + "config_sha256": "b8b9f74d9ec270eaa8215b69528e09214d33a3d565a9b7e5301de44342773347", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/torch-tensor-parallelism", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-tensor-parallelism.json", + "config_sha256": "c66e31078c9ac76e056984c16b19338004c203afc6cb700ed6fa219979362737", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/write-compressor", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/write-compressor.json", + "config_sha256": "4c3398cec611bc726de21f2833be71a6d8c5332b37fbed155b72eea43bc73ac5", + "job_name": "pi-flash-dev34-write-compressor-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/constraints-scheduling", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/constraints-scheduling.json", + "config_sha256": "5b7ad367963a6894d17eef2ae698f6f4ba329ac779072abacb5f8f8eb5f13f5b", + "job_name": "pi-flash-dev34-constraints-scheduling-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/custom-memory-heap-crash", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/custom-memory-heap-crash.json", + "config_sha256": "02f32088faca67cd0b81860022c5c03861f41f772058b1c18e72b903439fee59", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/dna-assembly", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dna-assembly.json", + "config_sha256": "1e21697089756a67a11683a935a00b1f4f0373c8d9e5977cd73ea9ad6754db86", + "job_name": "pi-flash-dev34-dna-assembly-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/llm-inference-batching-scheduler", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/llm-inference-batching-scheduler.json", + "config_sha256": "990ffdb0670445825f8ce37707ca3c18625e37608fcd22ebd8f9f8c30b90cbf6", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/path-tracing", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/path-tracing.json", + "config_sha256": "e6aef64f083166c6eb4c506a1f7c1ea01d406f4c6e2a9dd87d5fbff5c24bd92f", + "job_name": "pi-flash-dev34-path-tracing-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/schemelike-metacircular-eval", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/schemelike-metacircular-eval.json", + "config_sha256": "8c25426015609bd39dc1002c9da30aca4c2979731701c2683a320dbeffc6eb20", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/caffe-cifar-10", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/caffe-cifar-10.json", + "config_sha256": "ef0fa08a4a2ffb6f7652dd1b918ab0a1e397e4082a96c8518cb64261e787791a", + "job_name": "pi-flash-dev34-caffe-cifar-10-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/circuit-fibsqrt", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/circuit-fibsqrt.json", + "config_sha256": "141731de25773e5e87c6c4157968f2526820430dac6d153411d961fdfbd017ca", + "job_name": "pi-flash-dev34-circuit-fibsqrt-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/mteb-leaderboard", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/mteb-leaderboard.json", + "config_sha256": "712201f8c47c3995c2e3f553b08c1b3883df7b4cde6078bd6fb450675e92c6ad", + "job_name": "pi-flash-dev34-mteb-leaderboard-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/regex-chess", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-chess.json", + "config_sha256": "f8402e08659eff542793bd3e9f95960f47f659265f1424ec22e2c19d32e1260b", + "job_name": "pi-flash-dev34-regex-chess-20261024" + }, + { + "arm": "pi", + "repeat": 1, + "task": "terminal-bench/build-pov-ray", + "config": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/build-pov-ray.json", + "config_sha256": "2c77147b66971c37fbafa1597046389864cee4d3137940c9fd05f3f33dedafbf", + "job_name": "pi-flash-dev34-build-pov-ray-20261024" + } + ], + "additional_file_sha256": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json": "0e1e2cdbef4ede58b2beed0f4c3655099fb2d6f7e3f6d2ca369879fa24546eb1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json": "47a2c7370d7e5359c401a74efbc4fef63168229f54e164e88a7974e00af94ce1", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json": "77dfda40e45b870a8ce7f07a1e058d416d94a04ab3d27c2f21bc370f22151394", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf": "d0e1ffb6dbd02c54978ba2df386326311a59f6d3774990180f22bcc6adfaedbc", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json": "fd9abd24bcc397d172ac0ad4e3faa875a2da8c482b35651a2ab296e04c3adcc4", + "benchmarks/harbor/scripts/apt_index_proxy.py": "bc15f7adf1d7968d627ceca4683ab86742a7188a1b1955ba1fef37ba48e8a936", + "benchmarks/harbor/scripts/cache_image.py": "639709c03a7b0c3eb6aa78b4e87e6184b49b4b102853a31e33dd832d759180c0", + "benchmarks/harbor/scripts/uv_release_cache.py": "993cc35b1f3a9e8e171b662ab6d4524c5fe059f0d1f0e275a5bb1930111e0ff5", + "benchmarks/harbor/scripts/run_comparison.py": "b5f16a23b4495ab7e4a7028eb697649c0b8b22cbef6b03a4493437ba0f5204c7", + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json": "7f21855951ff5fff62cb16e686d93e4aae726be117dad049641c67956ab00d46" + }, + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation checked 2026-10-24 maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "verifier_policy": "Original official tests and native per-attempt timeouts. Only verifier timeouts retry, at most three total attempts on the same output. No dependency-error recovery or whole-trial retry. Preserve setup errors and verifier errors separately from scored failures.", + "preflight_policy": "Fresh install-only checks for all 34 tasks before any model trial. QEMU has the registered qemu-alpine-ssh-v1 profile; the other 33 explicitly report not_configured. Binary installation success does not certify unsupported verifier dependencies.", + "queue_policy": "Ascending native agent timeout then task name; consecutive pair barriers; at most two task trials at once; exactly one fresh output per task.", + "information_policy": "All 34 tasks are the existing development pool with prior exposure. No outcome-based task replacement. Review only operational metadata while running; freeze all new outputs before inspecting new trajectories or hidden tests. Keep the remaining 55 tasks unopened.", + "status_at_registration": "not_started: insufficient free system disk; no installation preflight or model trial has started", + "runtime_environment": "Use UV_PROJECT_ENVIRONMENT=/tmp/nanopy-pi-dev34-harbor-venv. Do not reuse or modify another worktree virtualenv." +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/fix-code-vulnerability.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/fix-code-vulnerability.json new file mode 100644 index 0000000..94fed72 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/fix-code-vulnerability.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/git-multibranch.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/git-multibranch.json new file mode 100644 index 0000000..d241f16 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/git-multibranch.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json new file mode 100644 index 0000000..c5661be --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/image-digests.json @@ -0,0 +1,312 @@ +{ + "observed_at": "2026-10-24T14:07:09.186521+00:00", + "images": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "image": "alexgshaw/adaptive-rejection-sampler:20251031", + "image_id": "sha256:985bed39e8448bc95f138b5f766b59003a95c4be171c96af8e76448338f3dd8f", + "repo_digests": [ + "alexgshaw/adaptive-rejection-sampler@sha256:985bed39e8448bc95f138b5f766b59003a95c4be171c96af8e76448338f3dd8f" + ], + "size_bytes": 29731756 + }, + { + "task": "terminal-bench/build-cython-ext", + "image": "alexgshaw/build-cython-ext:20251031", + "image_id": "sha256:3612a38fadb89a96f74a1a951fb0b0af734198fd160571eeaba6401593234594", + "repo_digests": [ + "alexgshaw/build-cython-ext@sha256:3612a38fadb89a96f74a1a951fb0b0af734198fd160571eeaba6401593234594" + ], + "size_bytes": 303222492 + }, + { + "task": "terminal-bench/cancel-async-tasks", + "image": "alexgshaw/cancel-async-tasks:20251031", + "image_id": "sha256:84c7fae6b256dcc56a350790e2a9715eefc7dad662a9d8e8a472363aa71ef18d", + "repo_digests": [ + "alexgshaw/cancel-async-tasks@sha256:84c7fae6b256dcc56a350790e2a9715eefc7dad662a9d8e8a472363aa71ef18d" + ], + "size_bytes": 44171676 + }, + { + "task": "terminal-bench/cobol-modernization", + "image": "alexgshaw/cobol-modernization:20251031", + "image_id": "sha256:593ab9df3d83f771e927888b2d9436b380cdf7ec4d0af2a9f0802d541edebba0", + "repo_digests": [ + "alexgshaw/cobol-modernization@sha256:593ab9df3d83f771e927888b2d9436b380cdf7ec4d0af2a9f0802d541edebba0" + ], + "size_bytes": 162020401 + }, + { + "task": "terminal-bench/db-wal-recovery", + "image": "alexgshaw/db-wal-recovery:20251031", + "image_id": "sha256:0e33ea5ec823975d1bd6c3778395c9f94251dd88f571146057bff6adb7e4594e", + "repo_digests": [ + "alexgshaw/db-wal-recovery@sha256:0e33ea5ec823975d1bd6c3778395c9f94251dd88f571146057bff6adb7e4594e" + ], + "size_bytes": 175276145 + }, + { + "task": "terminal-bench/fix-code-vulnerability", + "image": "alexgshaw/fix-code-vulnerability:20251031", + "image_id": "sha256:cac325252991f823713b2d0441502972901dd782bd67f66c03d9b1e410dac5c0", + "repo_digests": [ + "alexgshaw/fix-code-vulnerability@sha256:cac325252991f823713b2d0441502972901dd782bd67f66c03d9b1e410dac5c0" + ], + "size_bytes": 190696453 + }, + { + "task": "terminal-bench/git-multibranch", + "image": "alexgshaw/git-multibranch:20251031", + "image_id": "sha256:5bc449921c138295f6ddd8dca2977319fee2ce523811dbe56cec4ca663385fb5", + "repo_digests": [ + "alexgshaw/git-multibranch@sha256:5bc449921c138295f6ddd8dca2977319fee2ce523811dbe56cec4ca663385fb5" + ], + "size_bytes": 222134454 + }, + { + "task": "terminal-bench/kv-store-grpc", + "image": "alexgshaw/kv-store-grpc:20251031", + "image_id": "sha256:3399400800dcb207634daa42bc1b052e831e285cc9d221eea66c47bc0fc79791", + "repo_digests": [ + "alexgshaw/kv-store-grpc@sha256:3399400800dcb207634daa42bc1b052e831e285cc9d221eea66c47bc0fc79791" + ], + "size_bytes": 44171671 + }, + { + "task": "terminal-bench/log-summary-date-ranges", + "image": "alexgshaw/log-summary-date-ranges:20251031", + "image_id": "sha256:cbeb6ba905c2fec294f16cd5e16e3ea7f2e04d38ac2484d51a11de262aa7dc51", + "repo_digests": [ + "alexgshaw/log-summary-date-ranges@sha256:cbeb6ba905c2fec294f16cd5e16e3ea7f2e04d38ac2484d51a11de262aa7dc51" + ], + "size_bytes": 44866158 + }, + { + "task": "terminal-bench/merge-diff-arc-agi-task", + "image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "image_id": "sha256:bfc2a235f2ceea64ffb1ffc5529c2acebf570bff61f3668f0e5669ef570c17d4", + "repo_digests": [ + "alexgshaw/merge-diff-arc-agi-task@sha256:bfc2a235f2ceea64ffb1ffc5529c2acebf570bff61f3668f0e5669ef570c17d4" + ], + "size_bytes": 29728285 + }, + { + "task": "terminal-bench/model-extraction-relu-logits", + "image": "alexgshaw/model-extraction-relu-logits:20251031", + "image_id": "sha256:52fe1f089f38650f0dc22d7531ee6e01ebd526de1b349aaa2ecb331dca9fabce", + "repo_digests": [ + "alexgshaw/model-extraction-relu-logits@sha256:52fe1f089f38650f0dc22d7531ee6e01ebd526de1b349aaa2ecb331dca9fabce" + ], + "size_bytes": 83698971 + }, + { + "task": "terminal-bench/multi-source-data-merger", + "image": "alexgshaw/multi-source-data-merger:20251031", + "image_id": "sha256:8b32782078ff7383a1b4e5d3cecca8ce287e30f50bb4c0e5db009a18064e666e", + "repo_digests": [ + "alexgshaw/multi-source-data-merger@sha256:8b32782078ff7383a1b4e5d3cecca8ce287e30f50bb4c0e5db009a18064e666e" + ], + "size_bytes": 220880128 + }, + { + "task": "terminal-bench/nginx-request-logging", + "image": "alexgshaw/nginx-request-logging:20251031", + "image_id": "sha256:693e64431a0dbf45952e8b91a298d54fd60247e501c6623fb8fa3ebc07fc3d3d", + "repo_digests": [ + "alexgshaw/nginx-request-logging@sha256:693e64431a0dbf45952e8b91a298d54fd60247e501c6623fb8fa3ebc07fc3d3d" + ], + "size_bytes": 67285778 + }, + { + "task": "terminal-bench/openssl-selfsigned-cert", + "image": "alexgshaw/openssl-selfsigned-cert:20251031", + "image_id": "sha256:4c948a4e630af2435ae0a19108fc0814a946ac2fa29a512469e0fc77b38c8c12", + "repo_digests": [ + "alexgshaw/openssl-selfsigned-cert@sha256:4c948a4e630af2435ae0a19108fc0814a946ac2fa29a512469e0fc77b38c8c12" + ], + "size_bytes": 44173158 + }, + { + "task": "terminal-bench/pypi-server", + "image": "alexgshaw/pypi-server:20251031", + "image_id": "sha256:d18bb30f47c7dcaa3acdbc31ac7413c98eadca2ea16540bbd380f2f063b95276", + "repo_digests": [ + "alexgshaw/pypi-server@sha256:d18bb30f47c7dcaa3acdbc31ac7413c98eadca2ea16540bbd380f2f063b95276" + ], + "size_bytes": 83726049 + }, + { + "task": "terminal-bench/pytorch-model-cli", + "image": "alexgshaw/pytorch-model-cli:20251031", + "image_id": "sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "repo_digests": [ + "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67" + ], + "size_bytes": 44349611 + }, + { + "task": "terminal-bench/pytorch-model-recovery", + "image": "alexgshaw/pytorch-model-recovery:20260430", + "image_id": "sha256:7c2bb52851cef25bdc3939e473496d25fbf11fd0cb7ab58bcf3cefcb751cdc0e", + "repo_digests": [ + "alexgshaw/pytorch-model-recovery@sha256:7c2bb52851cef25bdc3939e473496d25fbf11fd0cb7ab58bcf3cefcb751cdc0e" + ], + "size_bytes": 269661811 + }, + { + "task": "terminal-bench/qemu-alpine-ssh", + "image": "alexgshaw/qemu-alpine-ssh:20251031", + "image_id": "sha256:541e8214a8c4faa7b556531b899d45da21048fa2ca6dd328d8cb6c4cffa8e964", + "repo_digests": [ + "alexgshaw/qemu-alpine-ssh@sha256:541e8214a8c4faa7b556531b899d45da21048fa2ca6dd328d8cb6c4cffa8e964" + ], + "size_bytes": 1308553944 + }, + { + "task": "terminal-bench/query-optimize", + "image": "alexgshaw/query-optimize:20251031", + "image_id": "sha256:28de7da87004eab5b29bd5eaf1823579bc5b73ccea4b80349a5623185770da14", + "repo_digests": [ + "alexgshaw/query-optimize@sha256:28de7da87004eab5b29bd5eaf1823579bc5b73ccea4b80349a5623185770da14" + ], + "size_bytes": 120494963 + }, + { + "task": "terminal-bench/regex-log", + "image": "alexgshaw/regex-log:20251031", + "image_id": "sha256:90101b2e815323a8da20528a1439bebc407eb9761c9c68a3d557730856c878e9", + "repo_digests": [ + "alexgshaw/regex-log@sha256:90101b2e815323a8da20528a1439bebc407eb9761c9c68a3d557730856c878e9" + ], + "size_bytes": 29725514 + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "image": "alexgshaw/torch-pipeline-parallelism:20251031", + "image_id": "sha256:3cb7b39d86f1704b20e668851da0b08b34b3270c01d583f136d65190984a8d1b", + "repo_digests": [ + "alexgshaw/torch-pipeline-parallelism@sha256:3cb7b39d86f1704b20e668851da0b08b34b3270c01d583f136d65190984a8d1b" + ], + "size_bytes": 29725531 + }, + { + "task": "terminal-bench/torch-tensor-parallelism", + "image": "alexgshaw/torch-tensor-parallelism:20251031", + "image_id": "sha256:c50ac169e11465f41b49749fe87e4487f24d31a0c39c0906cd3242cc5499c690", + "repo_digests": [ + "alexgshaw/torch-tensor-parallelism@sha256:c50ac169e11465f41b49749fe87e4487f24d31a0c39c0906cd3242cc5499c690" + ], + "size_bytes": 29725529 + }, + { + "task": "terminal-bench/write-compressor", + "image": "alexgshaw/write-compressor:20251031", + "image_id": "sha256:3618e1f8a997b437c09cd3dbaac736705809c05f6f12f584b65722175970ebe1", + "repo_digests": [ + "alexgshaw/write-compressor@sha256:3618e1f8a997b437c09cd3dbaac736705809c05f6f12f584b65722175970ebe1" + ], + "size_bytes": 291927726 + }, + { + "task": "terminal-bench/constraints-scheduling", + "image": "alexgshaw/constraints-scheduling:20251031", + "image_id": "sha256:567ce5a189f8d11ac461790876e934cc7af38391baf89a78f95f4dafc1fec3b0", + "repo_digests": [ + "alexgshaw/constraints-scheduling@sha256:567ce5a189f8d11ac461790876e934cc7af38391baf89a78f95f4dafc1fec3b0" + ], + "size_bytes": 49852952 + }, + { + "task": "terminal-bench/custom-memory-heap-crash", + "image": "alexgshaw/custom-memory-heap-crash:20251031", + "image_id": "sha256:1895d7d6240d5fb2ec774d6bfdd1057d170f8d199c15c582e4585d3c63802557", + "repo_digests": [ + "alexgshaw/custom-memory-heap-crash@sha256:1895d7d6240d5fb2ec774d6bfdd1057d170f8d199c15c582e4585d3c63802557" + ], + "size_bytes": 1267999268 + }, + { + "task": "terminal-bench/dna-assembly", + "image": "alexgshaw/dna-assembly:20251031", + "image_id": "sha256:d1adf6835f1dd91205ba70e452c699d0aea601010038e5617f370716efb50569", + "repo_digests": [ + "alexgshaw/dna-assembly@sha256:d1adf6835f1dd91205ba70e452c699d0aea601010038e5617f370716efb50569" + ], + "size_bytes": 29727501 + }, + { + "task": "terminal-bench/llm-inference-batching-scheduler", + "image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "image_id": "sha256:19e79aa49be4e55dccca1dde966515ebe6852314b6010bcb9230afb48b996774", + "repo_digests": [ + "alexgshaw/llm-inference-batching-scheduler@sha256:19e79aa49be4e55dccca1dde966515ebe6852314b6010bcb9230afb48b996774" + ], + "size_bytes": 44188631 + }, + { + "task": "terminal-bench/path-tracing", + "image": "alexgshaw/path-tracing:20251031", + "image_id": "sha256:4b44725102cd627dfa895191a5a30d16dae3c5623bb43da6459f79135db6b482", + "repo_digests": [ + "alexgshaw/path-tracing@sha256:4b44725102cd627dfa895191a5a30d16dae3c5623bb43da6459f79135db6b482" + ], + "size_bytes": 399390468 + }, + { + "task": "terminal-bench/schemelike-metacircular-eval", + "image": "alexgshaw/schemelike-metacircular-eval:20251031", + "image_id": "sha256:b485be38bf80d56c1a4cd9da95bbf7ec129dc03f6786b86c0525518a96f6d1db", + "repo_digests": [ + "alexgshaw/schemelike-metacircular-eval@sha256:b485be38bf80d56c1a4cd9da95bbf7ec129dc03f6786b86c0525518a96f6d1db" + ], + "size_bytes": 44188336 + }, + { + "task": "terminal-bench/caffe-cifar-10", + "image": "alexgshaw/caffe-cifar-10:20260403", + "image_id": "sha256:929a6d631b592c82e0d0f5d4a65cf03c946dc554cfc9bb09916e82076e7d3215", + "repo_digests": [ + "alexgshaw/caffe-cifar-10@sha256:929a6d631b592c82e0d0f5d4a65cf03c946dc554cfc9bb09916e82076e7d3215" + ], + "size_bytes": 213171982 + }, + { + "task": "terminal-bench/circuit-fibsqrt", + "image": "alexgshaw/circuit-fibsqrt:20251031", + "image_id": "sha256:29783439f529eaed2145592f15af7a2281161528860392a382c825489030ae3a", + "repo_digests": [ + "alexgshaw/circuit-fibsqrt@sha256:29783439f529eaed2145592f15af7a2281161528860392a382c825489030ae3a" + ], + "size_bytes": 149136738 + }, + { + "task": "terminal-bench/mteb-leaderboard", + "image": "alexgshaw/mteb-leaderboard:20260430", + "image_id": "sha256:a8538f4882ff132e0a20a75be3fffc07164318a7425ad19a6b4bf5268107feb4", + "repo_digests": [ + "alexgshaw/mteb-leaderboard@sha256:a8538f4882ff132e0a20a75be3fffc07164318a7425ad19a6b4bf5268107feb4" + ], + "size_bytes": 663825427 + }, + { + "task": "terminal-bench/regex-chess", + "image": "alexgshaw/regex-chess:20251031", + "image_id": "sha256:e3b6d61d2d3930fbbdc5f7bf091a7f8b7f6125e1951ef8a7dbe165b1ef62942c", + "repo_digests": [ + "alexgshaw/regex-chess@sha256:e3b6d61d2d3930fbbdc5f7bf091a7f8b7f6125e1951ef8a7dbe165b1ef62942c" + ], + "size_bytes": 55625584 + }, + { + "task": "terminal-bench/build-pov-ray", + "image": "alexgshaw/build-pov-ray:20251031", + "image_id": "sha256:874baf49341e2853a15a9184e308c7eb632e97f434359fcff2891c6e5ca1b39b", + "repo_digests": [ + "alexgshaw/build-pov-ray@sha256:874baf49341e2853a15a9184e308c7eb632e97f434359fcff2891c6e5ca1b39b" + ], + "size_bytes": 163066497 + } + ], + "matches_prior_registration": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/install-only.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/install-only.json new file mode 100644 index 0000000..046cd04 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/install-only.json @@ -0,0 +1,165 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "install_only": true, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12" + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1" + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88" + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71" + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/kv-store-grpc.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/kv-store-grpc.json new file mode 100644 index 0000000..b360136 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/kv-store-grpc.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/llm-inference-batching-scheduler.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/llm-inference-batching-scheduler.json new file mode 100644 index 0000000..f1a3025 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/llm-inference-batching-scheduler.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/log-summary-date-ranges.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/log-summary-date-ranges.json new file mode 100644 index 0000000..f85d2c0 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/log-summary-date-ranges.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/merge-diff-arc-agi-task.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/merge-diff-arc-agi-task.json new file mode 100644 index 0000000..8cf8e87 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/merge-diff-arc-agi-task.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-extraction-relu-logits.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-extraction-relu-logits.json new file mode 100644 index 0000000..4412126 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-extraction-relu-logits.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-prefix-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-prefix-evidence.json new file mode 100644 index 0000000..f69b1ff --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/model-prefix-evidence.json @@ -0,0 +1,98 @@ +{ + "verified_at": "2026-10-29T15:06:42.217549+00:00", + "source_plan": "experiment-v6.json", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "policy": "Preserve all six completed model attempts byte-for-byte and never rerun them. Timeout integrity exceptions are recorded separately in timeout-boundary-evidence.json. No new solution or hidden-test content was reviewed.", + "completed_model_attempts": [ + { + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027", + "task": "terminal-bench/adaptive-rejection-sampler", + "state_path": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-adaptive-rejection-sampler-20261027.json", + "state_sha256": "34930edb4b539f2f797a4467a5ead86bf898fd6079cabf6c63adc4b3023b10ea", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/result.json", + "result_sha256": "54f5f469df311a50cc463503f529e0ad7188823891ab31840965ef4c78bfef97", + "events_sha256": "70022bc1980da1fef1d93a9105d5b7da64753d01d526d93cd32184b677e5ec8b", + "manifest_sha256": "1e4792e091786a5affc7b599a1c3a8d996b12e2648c3265e62d5ce858d08af90", + "input_config_sha256": "d80a1afeaf576d6f97179c0a8e28af8643e3f8886ddacba5707054f0b6fe6be4", + "native_exception_type": "AgentTimeoutError", + "native_rewards": { + "reward": 0.0 + } + }, + { + "job_name": "pi-flash-dev34-build-cython-ext-20261027", + "task": "terminal-bench/build-cython-ext", + "state_path": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-build-cython-ext-20261027.json", + "state_sha256": "cfbdf5ee5164bd04e78061de307ea6aacfc4c8b1c44819aebbbfff0f203458c6", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/result.json", + "result_sha256": "3a49d523dae23770753f74c93bdff2e97494138a2cd436cde8cd8f4705bc93b9", + "events_sha256": "997b216ad3a1b32d6186a02fca6575e3008ec53ba9832a55ed272af7a285e039", + "manifest_sha256": "534f51423f4bc9cbeff1a2f09808aa888c2bd073abdb4659a7298f2322febfec", + "input_config_sha256": "67ef98793375cac7ace3a013eb2c920bbb74c8636950d1d493b59c545486e411", + "native_exception_type": null, + "native_rewards": { + "reward": 1.0 + } + }, + { + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027", + "task": "terminal-bench/cancel-async-tasks", + "state_path": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-cancel-async-tasks-20261027.json", + "state_sha256": "9ef6450ab54bf76f8dcd2903b1d8ea08f1b97f0f8c7096a9252343a3757114e2", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/result.json", + "result_sha256": "39339742cbac7e4dc77a7efa1287c5717c9c92f22b57aef56251e8d2e40f8293", + "events_sha256": "dab321388ecd82f38a1b3158bc962da4b029e34c7a7471b3986afd9e3edb4faf", + "manifest_sha256": "40b9662d7f30f8110fb23213eb78e0f2932d87ce8d475f9d5f708f8e20eb45cc", + "input_config_sha256": "b110936d56dd3f287fae2e4de577a17856cefff6eb51d4d0477587f5050fe19f", + "native_exception_type": null, + "native_rewards": { + "reward": 0.0 + } + }, + { + "job_name": "pi-flash-dev34-cobol-modernization-20261027", + "task": "terminal-bench/cobol-modernization", + "state_path": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-cobol-modernization-20261027.json", + "state_sha256": "bf8210929a97395abec3b035c30fa9fc08069d3193c1db8349cbe8bfbbd1e9de", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/result.json", + "result_sha256": "7f9ac5e0d92e065f66c1a58229952a9c48a2a61d016cb6e559b6238ec6b6a1fd", + "events_sha256": "e309667ca4a8a464ee43d8e372fe0f2cab1426cad8cae6377ee0043c53960430", + "manifest_sha256": "29df150affe475379b4d87689a5367d5b4bdba3db50a086ca2eea65741d48abe", + "input_config_sha256": "074e443d06eec327ea4ade6ef74dceb808c14309c37dafd11e6d34120ad16ad5", + "native_exception_type": null, + "native_rewards": { + "reward": 1.0 + } + }, + { + "job_name": "pi-flash-dev34-db-wal-recovery-20261027", + "task": "terminal-bench/db-wal-recovery", + "state_path": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-db-wal-recovery-20261027.json", + "state_sha256": "5397db0073e10bedd064ea5d1ae8f665f24738991f3d14c7f4b2103c0dae05df", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/result.json", + "result_sha256": "1318b9f55b8ca84b5c8067ba5a36ee2d7b953acb550f3e01062d2612090a0755", + "events_sha256": "fb60c26f364cb61eff8742cf53feb87feb9743f73c4afdb06e3314b343aed3cf", + "manifest_sha256": "cafa7c5dbbe442c8498e20b5a83ac803ab3eb45615c5e45fc2719f7bbe001757", + "input_config_sha256": "1372342a226701f3df82b7076fb65492eb0c4ae74f357cfd85d4da1287bdef7c", + "native_exception_type": "AgentTimeoutError", + "native_rewards": { + "reward": 0.0 + } + }, + { + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027", + "task": "terminal-bench/fix-code-vulnerability", + "state_path": "jobs/pi-deepseek-flash-dev34-20261029-transport-v6-orchestration/pi-flash-dev34-fix-code-vulnerability-20261027.json", + "state_sha256": "54d0fb06804c46365dc92f10e185b17ab26ddbfae9d1d957a6a455f540839c6b", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/result.json", + "result_sha256": "f69a07a3a1588d680942a9aac2f9fb821906f9fe2d08be24d6798d5b13f06a1d", + "events_sha256": "3fc3420e80c5381a3d6e7482cc1287ae66d17cba4b1b736984a8c5fad41c4c1b", + "manifest_sha256": "5df76d2c959c00eccbdb95c8fa6d782d5e7fa1c0dbb77fab7d6059f1da0751a6", + "input_config_sha256": "1072978ce9e8e38d9c8b0d783976d320f89575178ca10db995414798d19b1ebe", + "native_exception_type": null, + "native_rewards": { + "reward": 1.0 + } + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/mteb-leaderboard.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/mteb-leaderboard.json new file mode 100644 index 0000000..2132bee --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/mteb-leaderboard.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/multi-source-data-merger.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/multi-source-data-merger.json new file mode 100644 index 0000000..7d2144b --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/multi-source-data-merger.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/nginx-request-logging.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/nginx-request-logging.json new file mode 100644 index 0000000..49100e6 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/nginx-request-logging.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-attestation-import-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-attestation-import-evidence.json new file mode 100644 index 0000000..d94b1b5 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-attestation-import-evidence.json @@ -0,0 +1,9 @@ +{ + "observed_at": "2026-10-29T13:33:31.180831+00:00", + "kind": "isolated four-descriptor index with native attestation and absent ARM image/attestation; actual repository cache writer", + "parent_index_preserved": true, + "repository_digest_preserved": true, + "native_attestation_preserved": true, + "non_native_blobs_not_fetched": true, + "archive_sha256": "1d2a6f1ad9cc8a5f18eb79bfa59e7989d192ff0a159333c49651792980a61cf5" +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-import-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-import-evidence.json new file mode 100644 index 0000000..4e995f6 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/oci-index-import-evidence.json @@ -0,0 +1,16 @@ +{ + "observed_at": "2026-10-29T13:24:10.456460+00:00", + "kind": "isolated synthetic native-platform import, using the repository cache writer", + "root_digest": "sha256:74c3bf04c184bde95a6fc8642ca2fde51e68775085c0627a3762287b7e27b610", + "native_manifest_digest": "sha256:c545cf966677b726472e333f2822a0f8ca2b851b3069609a17b0f9f62753a8ea", + "archive_sha256": "8f9480da60a45b5c41661284adfa3f3f3e427baa077a7852c2ff809539202b47", + "parent_index_preserved": true, + "repository_digest_preserved": true, + "other_platform_metadata_retained": true, + "native_pull_attempt": { + "image": "alexgshaw/mteb-leaderboard@sha256:a8538f4882ff132e0a20a75be3fffc07164318a7425ad19a6b4bf5268107feb4", + "timeout_seconds": 600, + "exit_code": 124, + "native_trials_started": 0 + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/openssl-selfsigned-cert.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/openssl-selfsigned-cert.json new file mode 100644 index 0000000..98b75b7 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/openssl-selfsigned-cert.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json new file mode 100644 index 0000000..d4e4223 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json @@ -0,0 +1,6080 @@ +{ + "schema_version": 1, + "experiment": "pi-deepseek-flash-dev34-20261024", + "frozen_at": "2026-10-30T13:54:30.827662+00:00", + "policy": "All 34 native model trials have ended. File hashes were collected before any new solution trajectory or official hidden-test content review. Raw outputs and original exceptions are preserved; no task output is replayed or replaced.", + "task_count": 34, + "trials": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027", + "trial": "adaptive-rejection-sampler__p4rzaFM", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-adaptive-rejection-sampler-20261027.json" + }, + { + "task": "terminal-bench/build-cython-ext", + "job_name": "pi-flash-dev34-build-cython-ext-20261027", + "trial": "build-cython-ext__PZdLy7W", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-build-cython-ext-20261027.json" + }, + { + "task": "terminal-bench/cancel-async-tasks", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027", + "trial": "cancel-async-tasks__WAq9YLK", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-cancel-async-tasks-20261027.json" + }, + { + "task": "terminal-bench/cobol-modernization", + "job_name": "pi-flash-dev34-cobol-modernization-20261027", + "trial": "cobol-modernization__uGQXih9", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-cobol-modernization-20261027.json" + }, + { + "task": "terminal-bench/db-wal-recovery", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027", + "trial": "db-wal-recovery__J9YieXH", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-db-wal-recovery-20261027.json" + }, + { + "task": "terminal-bench/fix-code-vulnerability", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027", + "trial": "fix-code-vulnerability__4Dqvu3Q", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-fix-code-vulnerability-20261027.json" + }, + { + "task": "terminal-bench/git-multibranch", + "job_name": "pi-flash-dev34-git-multibranch-supervised-20261029", + "trial": "git-multibranch__dh6TaNP", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-git-multibranch-supervised-20261029.json" + }, + { + "task": "terminal-bench/kv-store-grpc", + "job_name": "pi-flash-dev34-kv-store-grpc-supervised-20261029", + "trial": "kv-store-grpc__Q92ErPw", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-kv-store-grpc-supervised-20261029.json" + }, + { + "task": "terminal-bench/log-summary-date-ranges", + "job_name": "pi-flash-dev34-log-summary-date-ranges-supervised-20261029", + "trial": "log-summary-date-ranges__CJ2vodJ", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-log-summary-date-ranges-supervised-20261029.json" + }, + { + "task": "terminal-bench/merge-diff-arc-agi-task", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029", + "trial": "merge-diff-arc-agi-task__iiAEYRk", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029.json" + }, + { + "task": "terminal-bench/model-extraction-relu-logits", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-supervised-20261029", + "trial": "model-extraction-relu-logits__fDX7jio", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029.json" + }, + { + "task": "terminal-bench/multi-source-data-merger", + "job_name": "pi-flash-dev34-multi-source-data-merger-supervised-20261029", + "trial": "multi-source-data-merger__YAw7zpk", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-multi-source-data-merger-supervised-20261029.json" + }, + { + "task": "terminal-bench/nginx-request-logging", + "job_name": "pi-flash-dev34-nginx-request-logging-supervised-20261029", + "trial": "nginx-request-logging__9Nwcxvo", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-nginx-request-logging-supervised-20261029.json" + }, + { + "task": "terminal-bench/openssl-selfsigned-cert", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029", + "trial": "openssl-selfsigned-cert__M9PAeMV", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029.json" + }, + { + "task": "terminal-bench/pypi-server", + "job_name": "pi-flash-dev34-pypi-server-supervised-20261029", + "trial": "pypi-server__9RDKNCX", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pypi-server-supervised-20261029.json" + }, + { + "task": "terminal-bench/pytorch-model-cli", + "job_name": "pi-flash-dev34-pytorch-model-cli-supervised-20261029", + "trial": "pytorch-model-cli__zbEnfLC", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pytorch-model-cli-supervised-20261029.json" + }, + { + "task": "terminal-bench/pytorch-model-recovery", + "job_name": "pi-flash-dev34-pytorch-model-recovery-supervised-20261029", + "trial": "pytorch-model-recovery__sv48aHg", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pytorch-model-recovery-supervised-20261029.json" + }, + { + "task": "terminal-bench/qemu-alpine-ssh", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-supervised-20261029", + "trial": "qemu-alpine-ssh__5gy8Wvz", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029.json" + }, + { + "task": "terminal-bench/query-optimize", + "job_name": "pi-flash-dev34-query-optimize-supervised-20261029", + "trial": "query-optimize__kF6mX9U", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-query-optimize-supervised-20261029.json" + }, + { + "task": "terminal-bench/regex-log", + "job_name": "pi-flash-dev34-regex-log-supervised-20261029", + "trial": "regex-log__oaEBj6S", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-regex-log-supervised-20261029.json" + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029", + "trial": "torch-pipeline-parallelism__4mz37Tn", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029.json" + }, + { + "task": "terminal-bench/torch-tensor-parallelism", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-supervised-20261029", + "trial": "torch-tensor-parallelism__Z79xPBb", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029.json" + }, + { + "task": "terminal-bench/write-compressor", + "job_name": "pi-flash-dev34-write-compressor-supervised-20261029", + "trial": "write-compressor__oh2ej7k", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-write-compressor-supervised-20261029.json" + }, + { + "task": "terminal-bench/constraints-scheduling", + "job_name": "pi-flash-dev34-constraints-scheduling-supervised-20261029", + "trial": "constraints-scheduling__XqZSbiP", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-constraints-scheduling-supervised-20261029.json" + }, + { + "task": "terminal-bench/custom-memory-heap-crash", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-supervised-20261029", + "trial": "custom-memory-heap-crash__UKhh5Pf", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029.json" + }, + { + "task": "terminal-bench/dna-assembly", + "job_name": "pi-flash-dev34-dna-assembly-supervised-20261029", + "trial": "dna-assembly__a33iNAA", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-dna-assembly-supervised-20261029.json" + }, + { + "task": "terminal-bench/llm-inference-batching-scheduler", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029", + "trial": "llm-inference-batching-scheduler__ubFeu2f", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029.json" + }, + { + "task": "terminal-bench/path-tracing", + "job_name": "pi-flash-dev34-path-tracing-supervised-20261029", + "trial": "path-tracing__QdtVmqq", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-path-tracing-supervised-20261029.json" + }, + { + "task": "terminal-bench/schemelike-metacircular-eval", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029", + "trial": "schemelike-metacircular-eval__n3CPpMo", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029.json" + }, + { + "task": "terminal-bench/caffe-cifar-10", + "job_name": "pi-flash-dev34-caffe-cifar-10-supervised-20261029", + "trial": "caffe-cifar-10__tdZV692", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-caffe-cifar-10-supervised-20261029.json" + }, + { + "task": "terminal-bench/circuit-fibsqrt", + "job_name": "pi-flash-dev34-circuit-fibsqrt-supervised-20261029", + "trial": "circuit-fibsqrt__mnDzrnD", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-circuit-fibsqrt-supervised-20261029.json" + }, + { + "task": "terminal-bench/mteb-leaderboard", + "job_name": "pi-flash-dev34-mteb-leaderboard-supervised-20261029", + "trial": "mteb-leaderboard__22sRwKk", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-mteb-leaderboard-supervised-20261029.json" + }, + { + "task": "terminal-bench/regex-chess", + "job_name": "pi-flash-dev34-regex-chess-supervised-20261029", + "trial": "regex-chess__TmQJBXt", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-regex-chess-supervised-20261029.json" + }, + { + "task": "terminal-bench/build-pov-ray", + "job_name": "pi-flash-dev34-build-pov-ray-supervised-20261029", + "trial": "build-pov-ray__oxrWfXD", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-build-pov-ray-supervised-20261029.json" + } + ], + "file_count": 1449, + "files": { + "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json": { + "sha256": "d90107ed15e47981fb3a09e74eb5d3b0e4f72acc96b30308c6d66ec28151128c", + "size": 121842 + }, + "jobs/pi-deepseek-flash-dev34-20261029-launch-v7.json": { + "sha256": "08642a2f5fe25852045f98351ece0d13821f43f38632b7789d4f5ec11129e3c3", + "size": 824 + }, + "jobs/pi-deepseek-flash-dev34-20261029-launch-v7.log": { + "sha256": "ffe0c67dc284892658ffefcdd10355dec39cccfb5c217fd32933d606b7294b5f", + "size": 4303 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/initial-docker-images.json": { + "sha256": "315c7ce2a3189d83354b5cc14cea5f653af0795300509caa9138fbfbf6b5ae76", + "size": 311 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-adaptive-rejection-sampler-20261027.json": { + "sha256": "34930edb4b539f2f797a4467a5ead86bf898fd6079cabf6c63adc4b3023b10ea", + "size": 28854 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-build-cython-ext-20261027.json": { + "sha256": "cfbdf5ee5164bd04e78061de307ea6aacfc4c8b1c44819aebbbfff0f203458c6", + "size": 9075 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-build-pov-ray-supervised-20261029.json": { + "sha256": "75d961a798cadabad48d1a8cfa8bfb2e62367577fe4d72dbf397de494ee30f40", + "size": 10506 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-caffe-cifar-10-supervised-20261029.json": { + "sha256": "270a61c8748438af9b9befb5c09a5d9f0997e081f03ac4686b17f7c7fb117f3b", + "size": 10505 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-cancel-async-tasks-20261027.json": { + "sha256": "9ef6450ab54bf76f8dcd2903b1d8ea08f1b97f0f8c7096a9252343a3757114e2", + "size": 26949 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-circuit-fibsqrt-supervised-20261029.json": { + "sha256": "45ac350b2543eba99624725e345f089ea27ed78cd6b92ab3283f32b2a1fb4395", + "size": 20206 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-cobol-modernization-20261027.json": { + "sha256": "bf8210929a97395abec3b035c30fa9fc08069d3193c1db8349cbe8bfbbd1e9de", + "size": 20131 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-constraints-scheduling-supervised-20261029.json": { + "sha256": "08ecd814263acc11dd7949a7dfa661fe52abe88ad05aaeb87a87a766af67f225", + "size": 23526 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029.json": { + "sha256": "f4c9d218107f8773cb76566f390627615b1f470574aa9b3820163559497f7b98", + "size": 7640 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-db-wal-recovery-20261027.json": { + "sha256": "5397db0073e10bedd064ea5d1ae8f665f24738991f3d14c7f4b2103c0dae05df", + "size": 15794 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-dna-assembly-supervised-20261029.json": { + "sha256": "1873e8224d4e8c7bde9395c333bf17512b261f49ee0cd162cae5af236b3cf4be", + "size": 28857 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-fix-code-vulnerability-20261027.json": { + "sha256": "54d0fb06804c46365dc92f10e185b17ab26ddbfae9d1d957a6a455f540839c6b", + "size": 5359 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-git-multibranch-supervised-20261029.json": { + "sha256": "0d1f0e9027859e872a174869f6e91e4d23523edf4c7face715bba7a6d80ed1a7", + "size": 7567 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-01-20261027.json": { + "sha256": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "size": 35774 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-02-20261027.json": { + "sha256": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "size": 44948 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-03-20261027.json": { + "sha256": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "size": 19014 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-04-20261027.json": { + "sha256": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "size": 32263 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-05-20261027.json": { + "sha256": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "size": 53611 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-06-20261027.json": { + "sha256": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "size": 49573 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-07-20261027.json": { + "sha256": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "size": 44702 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-08-20261027.json": { + "sha256": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "size": 44670 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-09-20261027.json": { + "sha256": "8b7cda88cdd32e5415096cc5fea995f0cc8f9a1d16f2dfea935b623a53e35f58", + "size": 36595 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-10-20261027.json": { + "sha256": "614222e8e2c4079b83b7a1c9e7b5ab11055dcbe005e8983295f1f3856392095a", + "size": 46009 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-11-20261027.json": { + "sha256": "a4707566ddc1a105e1850f07c038f2de8cefadea9e1bff94b85d17cb9c64784f", + "size": 55467 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-12-20261027.json": { + "sha256": "1991305b8e1c61171cdae0b07dea7dc9be1681300a9d56854e5d37364d525ebf", + "size": 45515 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-13-20261027.json": { + "sha256": "55557503afac1f21545f5cdff0f0b5c8f075936ed68edbdab46c4eae04e0b1c6", + "size": 34127 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-14-20261027.json": { + "sha256": "3bf013957d71092723564930ab2807a2df1ad479d4773855567b743092b873d3", + "size": 37195 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-15-20261027.json": { + "sha256": "f72f78d43cca70f0603c4875767cd982dadc9e6cf574e87bd9a1487cc1c930a3", + "size": 35243 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-16-20261027.json": { + "sha256": "048e1d88c8fc8c8ee23555190c45be1ba729fb663a4517ac4e8485dd7b20ed6f", + "size": 22825 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-17-20261027.json": { + "sha256": "2e0930dd3ebcf157cdc1e588183f6da79974f25f84ce574150dd1516673a94c5", + "size": 35207 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-kv-store-grpc-supervised-20261029.json": { + "sha256": "a6d6b9e556e19860124c521ea2c2952fa4c741462ebe137ab250ef215fcb78f8", + "size": 27015 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029.json": { + "sha256": "6f78a578edad8c3770b44f174c024c47b16e58cd826ef5c58543e372397bcd10", + "size": 27168 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-log-summary-date-ranges-supervised-20261029.json": { + "sha256": "bd878d2d9baa843f111491387253877a072247e3d55e6a9034ae6ade8cb9f0d3", + "size": 27096 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029.json": { + "sha256": "37628119bccb8072357888b40e19831b8d513eb00ddaac94eb9cf1ce56f7de59", + "size": 28943 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029.json": { + "sha256": "797873d88fdf9ce8593a3fab63c6c6a324fd5cc9ba7b3d53e22f95c681e42201", + "size": 27135 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-mteb-leaderboard-supervised-20261029.json": { + "sha256": "df32257dfbe9ae1521ca7419c8f544a1d397ad99db5edee4a54148d108f0f6d3", + "size": 4958 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-multi-source-data-merger-supervised-20261029.json": { + "sha256": "7ab2c11c03c82c6889420bddebd180c87c5a783fbd634eed02a66b1e0670d321", + "size": 24888 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-nginx-request-logging-supervised-20261029.json": { + "sha256": "95bf85db7c85706dffcf339b6bf96b50c7fbc6c9aa71c2579b02fc57927d3724", + "size": 20021 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029.json": { + "sha256": "0711f1c9e830d3a0a503f14fbd50103d509d795d3d523624a3095cebe9221ccd", + "size": 27096 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-path-tracing-supervised-20261029.json": { + "sha256": "1fdba69639d44fd9bc6001ca9b2606d36ce914283fde4b58324e2309e3dcf24c", + "size": 12445 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pypi-server-supervised-20261029.json": { + "sha256": "c741db4688153549cab7dd5a44ae8c0937c1eee1b7b56a8f6c4bd259d25b0800", + "size": 19943 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pytorch-model-cli-supervised-20261029.json": { + "sha256": "f3460efe4dbde62a6ce0c37ed017c9397b898693ec7f8d70ea5e489a5a46d96b", + "size": 27047 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pytorch-model-recovery-supervised-20261029.json": { + "sha256": "63b2802619b642b83e11f348bfa11c12b24336af79470f8b25e5702b7f82b219", + "size": 27086 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029.json": { + "sha256": "d2a5da4c3027d49140b7a44682904090d46b951d4e37b6287eb277c5940211fb", + "size": 11882 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-query-optimize-supervised-20261029.json": { + "sha256": "1db8be4f223bb1dfda7bb1659a38d8c318ee3e2bf642865439f53a88ac3fde6b", + "size": 19468 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-regex-chess-supervised-20261029.json": { + "sha256": "9e0c59b3f8fd07dd66e90e850dba3088536a5cea06952bc000a3ddd494e5141f", + "size": 27001 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-regex-log-supervised-20261029.json": { + "sha256": "891ce20156dd1094cbe9ab183472954b6221c8080725596c1b972e2ea9c11943", + "size": 28832 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029.json": { + "sha256": "f848883cba5e0da7c0c670fca073b3c227421fe0b02b3ae983be047ec87ffc59", + "size": 27136 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-01-20261029.json": { + "sha256": "8060da0de104029b46eba44267afec694726284d89476f7abe754faae4bf7f9e", + "size": 32419 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-02-20261029.json": { + "sha256": "2de6ab97ee9092fe976382131234e7afa84cf0ca29d01b13a5f6efc58e2edf06", + "size": 53767 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-03-20261029.json": { + "sha256": "b28ea5056094ce8e4eb031d4d45113d4aca58dbfe99b89d4ed6e94f2460d5ce0", + "size": 49715 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-04-20261029.json": { + "sha256": "60ae309218d6ecd76872fc920bb0cc01d61e04abd0f6c8d51e82532374ed33d8", + "size": 44858 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-05-20261029.json": { + "sha256": "995ec6c0cc508fcb7c9ce03be67b02080d5ab23831ea86d2e0331241b8296dc3", + "size": 44826 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-06-20261029.json": { + "sha256": "46fc0f78d291a7720c74e2520ed4e0d308caa927706ff8f0d824a244e4e7d631", + "size": 36751 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-07-20261029.json": { + "sha256": "8dc70c6f7325f4ecf6775ef4b5e6034c7d5bfe1448ee400a44c21d1efe236a6f", + "size": 46165 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-08-20261029.json": { + "sha256": "26aaa244f6c3fec349309d86a26076767c21f5e9173e3e7fd03c5e8c61689f6d", + "size": 55623 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-09-20261029.json": { + "sha256": "cb362b17996b94369c5a55bb82f739d95f122b61bd085bfcea7bf81511edb670", + "size": 45671 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-10-20261029.json": { + "sha256": "16712c120dd524292deb9594bf1f17caf6267e8b76eae0bc73ea14bea4725979", + "size": 34283 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-11-20261029.json": { + "sha256": "3fb349acb927128e9d756099b3eb722be63a6a733e8234ee118ec502c7511f92", + "size": 37351 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-12-20261029.json": { + "sha256": "74582c1f70521d13bbf070399f4c021f3e8453d3d146b72c83ae7a9b154cd2e6", + "size": 35399 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-13-20261029.json": { + "sha256": "a47addd1295f69e5af895bf1c53e7907f718fa4f222e06cabfdcda37659014d2", + "size": 22981 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-14-20261029.json": { + "sha256": "410c8c2d6c46d4a405450fc4561aeb2515c468188b5e6bfdedede2c3e0af58c4", + "size": 35363 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029.json": { + "sha256": "11834fc6225effecb7ef86d4632c0fe26c4dcbcee80a37bd9d291399eb48eaf0", + "size": 28967 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029.json": { + "sha256": "1284cd0c44e0f59eb910b2377cf608a29e771d2237d95730084d918100d31f92", + "size": 28947 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-write-compressor-supervised-20261029.json": { + "sha256": "853e86d06a5efa20da1f0cd4eafd6796d66f64f4f4cd439e58f9df59629f242a", + "size": 24370 + }, + "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/registration.json": { + "sha256": "fffc66ed65abaa2beb271ba106a8d850e7dab326db29b7e14f0aff3f093fbb05", + "size": 125441 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/config.json": { + "sha256": "d80a1afeaf576d6f97179c0a8e28af8643e3f8886ddacba5707054f0b6fe6be4", + "size": 3342 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/harbor.log": { + "sha256": "bc15e047cd66dfc2518a69ef2d83a5552cd728b70f4ff4a969ddecf76288c5e2", + "size": 1685 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/manifest.json": { + "sha256": "1e4792e091786a5affc7b599a1c3a8d996b12e2648c3265e62d5ce858d08af90", + "size": 36640 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "b6083866c9ba816ff97725e2ecdec54aea515a41017dc300fdafab4eca66ee98", + "size": 562 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "1026cd8774c3fd2289412f58f51168c0dc015af803ae7e0140a1e52c12b57873", + "size": 18740 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "7d52686d775f4fca93b53e1186fc81f2e76e560c63c4e89e6a914f0dbf961d3e", + "size": 11208 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "a309b5d6056ee313d9f96bbace8b796c016b2c6d9c16c8505c5591940eb65332", + "size": 13917 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "bd271b956489874cef6190e621c90fec759da03785355a76beb7f477d176a9e8", + "size": 12046 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/pi.py": { + "sha256": "7938611299cfab6be41474b7f8ecfc73f97833ba279d1255de7814cd1786f9bb", + "size": 9341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/source/harbor_adapter/verifier.py": { + "sha256": "44bf247282437a0092bd7e649fd7fa9c776b6d34a0c14ffedcc10b37a47608ef", + "size": 7921 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/summary.json": { + "sha256": "50d538a73792b5dab26d955c404dc7158280460194a419e81b6cf076f6c623c7", + "size": 26490 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/agent/bootstrap.json": { + "sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "size": 23095 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/agent/pi.txt": { + "sha256": "70022bc1980da1fef1d93a9105d5b7da64753d01d526d93cd32184b677e5ec8b", + "size": 778404 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/agent/pi/sessions/2026-10-29T13-54-34-411Z_01a187f0-e72a-7695-acac-2efab1ba3f2f.jsonl": { + "sha256": "041e9bd2496176a64c1b6d0fa3c5e10f826993a95abf5860200f1223290d9ccf", + "size": 278880 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/config.json": { + "sha256": "04e38be7d72d0eae06341de69743782a18b110605778718ce671f4472958fbb3", + "size": 1774 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/exception.txt": { + "sha256": "527d38e997e3422a37557b2a283ac30751000049407fa6f6ab49691ed81cdf86", + "size": 5500 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/lock.json": { + "sha256": "c9dc20eabeb71642f2a13e0ac25d2b2c659d669597ea97e5e2a11e92a0c2846b", + "size": 2355 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/result.json": { + "sha256": "54f5f469df311a50cc463503f529e0ad7188823891ab31840965ef4c78bfef97", + "size": 44177 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/trial.log": { + "sha256": "95b9734f2192a80c47cec3a6a50e96959613a53c6427923d5b355e442ebbfeb1", + "size": 3989 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/attempts/1/test-stdout.txt": { + "sha256": "80cdef0fa19603e6d650150a3d54f9cd160088d94b662b2ebfb141fbb99f962c", + "size": 1545 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/retry-summary.json": { + "sha256": "324b277fcaed5439105f721d4bfe5620fbb10bd122bd6bc7967dc93ee10575be", + "size": 234 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/test-stdout.txt": { + "sha256": "80cdef0fa19603e6d650150a3d54f9cd160088d94b662b2ebfb141fbb99f962c", + "size": 1545 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/config.json": { + "sha256": "6e5cfdda10f3afcfa38e5e9b451c151899d2a30e9b1e8d4ab97cec2f743c0870", + "size": 1800 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/job.log": { + "sha256": "66202ea122236f7a68de0d61ab463b7541eadcf23807966649f876cffb025849", + "size": 4176 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/lock.json": { + "sha256": "3df576f743fe98064ccec0dc2d2b152f59a9d3741123bf9e4f5671479700a73b", + "size": 3509 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/result.json": { + "sha256": "76248e150f11568625257b4c55805a5eaea44fd73953a908a9cd223f374d4db0", + "size": 1283 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/config.json": { + "sha256": "67ef98793375cac7ace3a013eb2c920bbb74c8636950d1d493b59c545486e411", + "size": 3312 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/harbor.log": { + "sha256": "41b6b0000a849cf2d8bfe89333bafee671d3714c498cfb99b73aac991065e997", + "size": 1328 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/manifest.json": { + "sha256": "534f51423f4bc9cbeff1a2f09808aa888c2bd073abdb4659a7298f2322febfec", + "size": 36640 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "a6b3da20a5380d958743429e5b86be229363e0326151f748c9f6c253288d210c", + "size": 552 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "1b76a1c49a51dff9e25df0e85e42a48c1bf32043d8cd75e67dda8f69ec8cf4ba", + "size": 18730 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "4cba800a0f57cae067cd6c3d0c7761d9c73da5dc09c75d012e78f16898e10ab8", + "size": 11198 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "2096785b71a46f4ea0cbd466b971a643cb0dd20d201771ce96ee11ef3ea21416", + "size": 13907 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "632d753ded1383f22c3c70fdd9f51fc3d63886d81cd30499497c2658c52164ef", + "size": 12036 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/pi.py": { + "sha256": "7938611299cfab6be41474b7f8ecfc73f97833ba279d1255de7814cd1786f9bb", + "size": 9341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/source/harbor_adapter/verifier.py": { + "sha256": "44bf247282437a0092bd7e649fd7fa9c776b6d34a0c14ffedcc10b37a47608ef", + "size": 7921 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/summary.json": { + "sha256": "605442a48230b92c1b85ae8d0218fd3167f25eda716d1da2ee7285711f6d5782", + "size": 7425 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/agent/bootstrap.json": { + "sha256": "e8874a2f7988d95eaf863a9e88bd62528a3bd88d7faab57a118d7e34848ca886", + "size": 6022 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/agent/pi.txt": { + "sha256": "997b216ad3a1b32d6186a02fca6575e3008ec53ba9832a55ed272af7a285e039", + "size": 1948323 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/agent/pi/sessions/2026-10-29T13-54-29-746Z_01a187f0-d4f2-73b0-aa24-790b0e4e0077.jsonl": { + "sha256": "c3da27e60d1e0350904cd5d7785bc5676f0e760fe4006414b74f2c851e7769d3", + "size": 318248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/config.json": { + "sha256": "9f0c9053275f6a6fb9d276f7f51f169d97438e4abe3f19e0be8cd09c003ae7ac", + "size": 1734 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/lock.json": { + "sha256": "e3a9c5acf10144a5eb8a755f0b8bc72477d32cefdf91baf9628007284643c968", + "size": 2335 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/result.json": { + "sha256": "3a49d523dae23770753f74c93bdff2e97494138a2cd436cde8cd8f4705bc93b9", + "size": 14230 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/trial.log": { + "sha256": "a9a51c3f2a223fa9d5356bab30e81e46b929367ef329bdacf465269b4ecb92a8", + "size": 3465 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/attempts/1/ctrf.json": { + "sha256": "f2ddaf25087864646191882836c8927edb48368ef5917da0734cef1cc92a3061", + "size": 4259 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/attempts/1/test-stdout.txt": { + "sha256": "e924d3527cb36dddb6f692b12a65f27c1dc6746635442c45de16565f8d111d23", + "size": 3526 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/ctrf.json": { + "sha256": "f2ddaf25087864646191882836c8927edb48368ef5917da0734cef1cc92a3061", + "size": 4259 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/retry-summary.json": { + "sha256": "161bd68190a23c89e95ab137b70e79a2e77f6c38e3d8f27339abc61c35311911", + "size": 233 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/test-stdout.txt": { + "sha256": "e924d3527cb36dddb6f692b12a65f27c1dc6746635442c45de16565f8d111d23", + "size": 3526 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/config.json": { + "sha256": "e2e05634a72228d865a958b525ad86ef2a98d60e3b09912a551bca3984342414", + "size": 1770 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/job.log": { + "sha256": "a9a51c3f2a223fa9d5356bab30e81e46b929367ef329bdacf465269b4ecb92a8", + "size": 3465 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/lock.json": { + "sha256": "70c7988f052feda63c4db207b1719a10da1c1bebb22b645fcfc4a875e30b99a2", + "size": 3489 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/result.json": { + "sha256": "39a44aaba48805f0584461dc07f0881b84db88c01346e3853611d9e21da37ce7", + "size": 1142 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/config.json": { + "sha256": "b110936d56dd3f287fae2e4de577a17856cefff6eb51d4d0477587f5050fe19f", + "size": 3318 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/harbor.log": { + "sha256": "e2c25e5da88efb8b94de1b25628182913bbb40a7582a4fc49dab6f47bcd28f52", + "size": 1331 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/manifest.json": { + "sha256": "40b9662d7f30f8110fb23213eb78e0f2932d87ce8d475f9d5f708f8e20eb45cc", + "size": 36640 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "53e52635a7cf7f102cb72a4030e8bfbfa2a4c758288876a896ab0aafdcf30623", + "size": 554 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "6f8e24211e0ef02fb21207855b636c3e547328816ccb6aa45b486737634f7d66", + "size": 18732 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "f505462c358d8ae4b184c49ec6511ea3db76ec6dec2343ee3a49df8ae9288de8", + "size": 11200 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "49e649a28ba4eb29661526bef33d8b64342f2140d0202d6169de431580756f9b", + "size": 13909 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "b464a70a763cae1d84ca268fb0aef1740ccf93b33fd1fa0a2f52d0037c8cbc35", + "size": 12038 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/pi.py": { + "sha256": "7938611299cfab6be41474b7f8ecfc73f97833ba279d1255de7814cd1786f9bb", + "size": 9341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/source/harbor_adapter/verifier.py": { + "sha256": "44bf247282437a0092bd7e649fd7fa9c776b6d34a0c14ffedcc10b37a47608ef", + "size": 7921 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/summary.json": { + "sha256": "09cfbaa809768fa3109814ccab583e7665436cbae7cb4ea4f8665ceae61c6da0", + "size": 24671 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/agent/bootstrap.json": { + "sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "size": 21415 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/agent/pi.txt": { + "sha256": "dab321388ecd82f38a1b3158bc962da4b029e34c7a7471b3986afd9e3edb4faf", + "size": 267614 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/agent/pi/sessions/2026-10-29T14-17-47-195Z_01a18806-27bb-73b5-8697-a61b5301839c.jsonl": { + "sha256": "03667891e37cb38981159e20dd047e67aa60c39b3b0710e08a8a32aa6fba9d6b", + "size": 71045 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/config.json": { + "sha256": "b7b1486fe15b997d9f4781acfa686bd88b261cd86052d4d5b5f864f1e27741db", + "size": 1742 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/lock.json": { + "sha256": "8510f43bddf6396aeeb7eca351f58fa42c236fd95f9ae3fe0fa246a4a378dc12", + "size": 2339 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/result.json": { + "sha256": "39339742cbac7e4dc77a7efa1287c5717c9c92f22b57aef56251e8d2e40f8293", + "size": 36141 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/trial.log": { + "sha256": "fef64595cb417dfc39fee083c117efbbca9f4b059a2fed1a63adb436b81f2860", + "size": 2384 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/attempts/1/ctrf.json": { + "sha256": "d7d4de63a16006df19dbb11748973e09ea72c3a4f50fbae56129e8a07a43b544", + "size": 4080 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/attempts/1/test-stdout.txt": { + "sha256": "ad0e96184b6cf24fb5127c1c2691789290ba56a033e98b2b2ea8992b8854878f", + "size": 3210 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/ctrf.json": { + "sha256": "d7d4de63a16006df19dbb11748973e09ea72c3a4f50fbae56129e8a07a43b544", + "size": 4080 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/retry-summary.json": { + "sha256": "5a4ed94af6cf95b6d7d1b056a2e685d698444e8f0424db11e4639384445953c3", + "size": 234 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/test-stdout.txt": { + "sha256": "ad0e96184b6cf24fb5127c1c2691789290ba56a033e98b2b2ea8992b8854878f", + "size": 3210 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/config.json": { + "sha256": "613133ce4b93b41307129752b3791e7dd8a5596c68413cee04c3137f6326c722", + "size": 1776 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/job.log": { + "sha256": "fef64595cb417dfc39fee083c117efbbca9f4b059a2fed1a63adb436b81f2860", + "size": 2384 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/lock.json": { + "sha256": "b3d4781ade94dc32a8f8c7d415ccfa1967bab229c417cd4b217c24dddac293ab", + "size": 3493 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/result.json": { + "sha256": "095539a438da3e813f24ea1d367a72edb61404b6996864ccd8d9e5c1fff8fb7d", + "size": 1132 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/config.json": { + "sha256": "074e443d06eec327ea4ade6ef74dceb808c14309c37dafd11e6d34120ad16ad5", + "size": 3321 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/harbor.log": { + "sha256": "44bf354ffbee2236566baf82f612411aa4409c11b6097969dfcdf34d478f9801", + "size": 1333 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/manifest.json": { + "sha256": "29df150affe475379b4d87689a5367d5b4bdba3db50a086ca2eea65741d48abe", + "size": 36640 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "fdbb267203db21fb69bede2f62e3dab578c0ea1125fe13e5ecb0e4029aeb36c7", + "size": 555 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "11c041e62657133fbdb5e76df744157a37f7ac2a90e1766499f7a60435105ed3", + "size": 18733 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "5c303f31c3fb579586423650e7456b257ef783ce2d30fdb29af2872342466a65", + "size": 11201 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "6683e10d09f74aad2ceb17ca57338568fb52ae69d344787a66242e930b68ec0e", + "size": 13910 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "913c54ea644e3f5c030ebc4f02be351e958913f5d0a12e18056ebca551948401", + "size": 12039 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/pi.py": { + "sha256": "7938611299cfab6be41474b7f8ecfc73f97833ba279d1255de7814cd1786f9bb", + "size": 9341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/source/harbor_adapter/verifier.py": { + "sha256": "44bf247282437a0092bd7e649fd7fa9c776b6d34a0c14ffedcc10b37a47608ef", + "size": 7921 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/summary.json": { + "sha256": "4156e27ccb3b2b82f1ba58a533b5bd5f0fdd21a6aaebbe7257effb478ff60a70", + "size": 18085 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/agent/bootstrap.json": { + "sha256": "bcfa3a0d9c4be3798cb32bcf016009e48b02df7e31360f67504ac46696df2865", + "size": 15542 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/agent/pi.txt": { + "sha256": "e309667ca4a8a464ee43d8e372fe0f2cab1426cad8cae6377ee0043c53960430", + "size": 1416981 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/agent/pi/sessions/2026-10-29T14-17-44-474Z_01a18806-1d1a-73a1-a56c-11d55b695325.jsonl": { + "sha256": "5060f84c21c8c58513f17ce5c4e6c9b84bbc401ca716ab5eafa3d82033fe1763", + "size": 305679 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/config.json": { + "sha256": "4b921a6ba7849ea1923093d1aa4867fbb41b27a22aec21ae7be1a2d368016ad7", + "size": 1746 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/lock.json": { + "sha256": "a56eb9ef0da010839167f50bcc3342c443d317fb8ae7e08523e31be3d130e991", + "size": 2341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/result.json": { + "sha256": "7f9ac5e0d92e065f66c1a58229952a9c48a2a61d016cb6e559b6238ec6b6a1fd", + "size": 27764 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/trial.log": { + "sha256": "c985fa72058aab09f1593243fcd7920ce075aae3a07c1ac9006fda75e0d5cbf8", + "size": 3058 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/attempts/1/ctrf.json": { + "sha256": "a07f0977e9fd8ca18bdf4ed38fe98a28ddac53c41492bf5069706210e0af3a7d", + "size": 1449 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/attempts/1/test-stdout.txt": { + "sha256": "db03a19762d0964bfaad857e3e7a8927a22808260d567dcbbb209218684a9a59", + "size": 1490 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/ctrf.json": { + "sha256": "a07f0977e9fd8ca18bdf4ed38fe98a28ddac53c41492bf5069706210e0af3a7d", + "size": 1449 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/retry-summary.json": { + "sha256": "de8a4c3cd24a67a0bc8f059c044c0dc9be08eab11beb943d9d480d01d79ec1a6", + "size": 233 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/test-stdout.txt": { + "sha256": "db03a19762d0964bfaad857e3e7a8927a22808260d567dcbbb209218684a9a59", + "size": 1490 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/config.json": { + "sha256": "31db80dc3c3df49163c4a103e59639959dd0c18b29cea7b564a2b72bb8c9ed18", + "size": 1779 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/job.log": { + "sha256": "c985fa72058aab09f1593243fcd7920ce075aae3a07c1ac9006fda75e0d5cbf8", + "size": 3058 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/lock.json": { + "sha256": "a9e88891b849bc4cd7133d78f3b03fa22ba44dd4200c6f11a8cb81c807ca62a1", + "size": 3495 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/result.json": { + "sha256": "e2de5904b73cf60dd1568f7ef052e9f5533ba1eb981ece74419bcfdd2a66864a", + "size": 1136 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/config.json": { + "sha256": "1372342a226701f3df82b7076fb65492eb0c4ae74f357cfd85d4da1287bdef7c", + "size": 3309 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/harbor.log": { + "sha256": "62616ff63673746af2abe85ff76538949d343cc7c0b124847be13a2a6929cecd", + "size": 1662 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/manifest.json": { + "sha256": "cafa7c5dbbe442c8498e20b5a83ac803ab3eb45615c5e45fc2719f7bbe001757", + "size": 36640 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "d40fa49623223d567ca2a067679b8aa8a35a609bc9b8581a17cb380c6206866f", + "size": 551 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "7dc86c34280bb20d66d5b0923457fd23babc3580127444dc6a481424bbdd5b12", + "size": 18729 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "089c416e280d2029c3f120827be688ebfabb046f27f6c7c93e6cefdad08d50fa", + "size": 11197 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "9a78d661546fce0be58c0badc6c8399497f84005b07a8a34da4a5c1d3b183bdd", + "size": 13906 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "6564c35eac78a6ae60c8725197156bfba847567bd8a88b5f1f331cf122d0065a", + "size": 12035 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/pi.py": { + "sha256": "7938611299cfab6be41474b7f8ecfc73f97833ba279d1255de7814cd1786f9bb", + "size": 9341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/source/harbor_adapter/verifier.py": { + "sha256": "44bf247282437a0092bd7e649fd7fa9c776b6d34a0c14ffedcc10b37a47608ef", + "size": 7921 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/summary.json": { + "sha256": "e49eeb22824b5c176ebee456ef2c84bc4b0f2995c9a0c4414ae34cc9909bb4b5", + "size": 13930 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/config.json": { + "sha256": "dd329d9559a669f68b69796a472db83d4da270d080ad981455265847b525826a", + "size": 1767 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/bootstrap.json": { + "sha256": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "size": 11860 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/pi.txt": { + "sha256": "fb60c26f364cb61eff8742cf53feb87feb9743f73c4afdb06e3314b343aed3cf", + "size": 2036515 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/pi/sessions/2026-10-29T14-26-49-590Z_01a1880e-6e76-7275-a097-a20814e7b807.jsonl": { + "sha256": "89ec446a2e0e4b7178bf6b639fdf896b0b87e644ecb3296b8c144fdbc56daad7", + "size": 414455 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/config.json": { + "sha256": "d88e651783b0968ce4006bc3c70dc044f9d827ab8ed1cd722c1f1dfc81d33410", + "size": 1730 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/exception.txt": { + "sha256": "6a5cb4f29ec06f509e619de5984b7bc2140e9cc85dcf4d79bdca61453f1a09c9", + "size": 5489 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/lock.json": { + "sha256": "ba1a5fa0c84f525d0d790daf8dee121e439c2164cab8ea32f9207c9afd12305c", + "size": 2333 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/result.json": { + "sha256": "1318b9f55b8ca84b5c8067ba5a36ee2d7b953acb550f3e01062d2612090a0755", + "size": 28257 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/trial.log": { + "sha256": "3e1593af96b2b2624d7c6ebc7f831960961b03fe8b783d921f96374682e5cb8e", + "size": 1920 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/attempts/1/ctrf.json": { + "sha256": "c4c4d54d2b1e4ff51fb4d8e3f1543b28d301214f7f1f8577055e54387f8f23d4", + "size": 6523 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/attempts/1/test-stdout.txt": { + "sha256": "56b743023c01a907fc584d908e57dbc2cfc001e4db337071149ef956176fdd0c", + "size": 5221 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/ctrf.json": { + "sha256": "c4c4d54d2b1e4ff51fb4d8e3f1543b28d301214f7f1f8577055e54387f8f23d4", + "size": 6523 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/retry-summary.json": { + "sha256": "8183af77239c947125f446f27893ba20d07a521819592bde99f1bd0ef978e2e1", + "size": 233 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/test-stdout.txt": { + "sha256": "56b743023c01a907fc584d908e57dbc2cfc001e4db337071149ef956176fdd0c", + "size": 5221 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/job.log": { + "sha256": "affe3b965be31e131177369f1622ab3269d32566eba0d0c2860cd179edbe67b2", + "size": 2107 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/lock.json": { + "sha256": "356c83fb29ba3504b94e3fb27a8b944849ca35db6bdbc6d3974900c558b52927", + "size": 3487 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/result.json": { + "sha256": "db3e1a8fef15d7275226cc30d55603495f76d5a0b4fa2cddd2f2504830de1d0a", + "size": 1273 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/config.json": { + "sha256": "1072978ce9e8e38d9c8b0d783976d320f89575178ca10db995414798d19b1ebe", + "size": 3330 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/harbor.log": { + "sha256": "b5b5f9507db1105144592c92ada89e307fec753108e104dd93f31763e9c4ced1", + "size": 1339 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/manifest.json": { + "sha256": "5df76d2c959c00eccbdb95c8fa6d782d5e7fa1c0dbb77fab7d6059f1da0751a6", + "size": 36640 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "2c791de1cd7f09c854bf49aaadd250093508ee1ae60b91b0d280de7bf2ed8c67", + "size": 558 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "1c12c40ee1c5300f28e0e98dd1f6cfd20a835afbe3bcfce5a6b41063416f3e32", + "size": 18736 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "8b11b088a2a68025405f57a2787c9057876eac4132bde7ab6b5acac0a11fc043", + "size": 11204 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "28f5830614df30462ad0007593373cb6a8bd69490ccf825d566a860001c19920", + "size": 13913 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "9e35f294d9cd3961b33c000c56d1cc81a9d8c5660f50c30aeb382f133cd2002f", + "size": 12042 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/pi.py": { + "sha256": "7938611299cfab6be41474b7f8ecfc73f97833ba279d1255de7814cd1786f9bb", + "size": 9341 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/source/harbor_adapter/verifier.py": { + "sha256": "44bf247282437a0092bd7e649fd7fa9c776b6d34a0c14ffedcc10b37a47608ef", + "size": 7921 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/summary.json": { + "sha256": "3871330e9d916f227d8faee54a94f8521638d2ab0f226a4ea19f59de255b8b44", + "size": 3799 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/config.json": { + "sha256": "54c029f951c83ea073f2da960451168c01afe1888aa69c140bdf7dbfd879424b", + "size": 1788 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/agent/bootstrap.json": { + "sha256": "2241a2a7fb273099c6848428dd504ef84602af1aa1a580dddc4a913f096e44bd", + "size": 2761 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/agent/pi.txt": { + "sha256": "3fc3420e80c5381a3d6e7482cc1287ae66d17cba4b1b736984a8c5fad41c4c1b", + "size": 437715 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/agent/pi/sessions/2026-10-29T14-26-56-000Z_01a1880e-8780-72a7-8402-f12f3afc1870.jsonl": { + "sha256": "08fb7a801ee0d4e845e9f2cfb8b035b5339a78ef7f5267fb57654359c41dd565", + "size": 73715 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/config.json": { + "sha256": "3f0de1b2eba91ba4be88a6e9012cd6b0c6043a7c79edd50f76e2e081d794548f", + "size": 1758 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/lock.json": { + "sha256": "e2f4f166df9c850e11b1bcc3d6fb1b1135e157e1370f2585885c45f757e3c62a", + "size": 2347 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/result.json": { + "sha256": "f69a07a3a1588d680942a9aac2f9fb821906f9fe2d08be24d6798d5b13f06a1d", + "size": 9676 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/trial.log": { + "sha256": "0134df4d98c5a9452aa7d5f64a74029091b8e66a3ae2ff7a7cb394b065204fe1", + "size": 5724 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/attempts/1/ctrf.json": { + "sha256": "605a1eb7542a12d65b937772f774a0dd4d23c2d62c7f2a64a546807c53f614bd", + "size": 2490 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/attempts/1/original-repo-ctrf.json": { + "sha256": "10222dfb73959e65e4061f3d23a9d57e049cc66790e22d21540b66ffd18d17aa", + "size": 136451 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/attempts/1/test-stdout.txt": { + "sha256": "c9cbb5f9e71ab4aa58a3820831cc3c3f83ab08ca508cfa6b5f936482cffd6f56", + "size": 28551 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/ctrf.json": { + "sha256": "605a1eb7542a12d65b937772f774a0dd4d23c2d62c7f2a64a546807c53f614bd", + "size": 2490 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/original-repo-ctrf.json": { + "sha256": "10222dfb73959e65e4061f3d23a9d57e049cc66790e22d21540b66ffd18d17aa", + "size": 136451 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/retry-summary.json": { + "sha256": "8b5af411420fc2067398787182a042ccabfcd900cd28f055c398ad04732b7cc0", + "size": 234 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/test-stdout.txt": { + "sha256": "c9cbb5f9e71ab4aa58a3820831cc3c3f83ab08ca508cfa6b5f936482cffd6f56", + "size": 28551 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/job.log": { + "sha256": "0134df4d98c5a9452aa7d5f64a74029091b8e66a3ae2ff7a7cb394b065204fe1", + "size": 5724 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/lock.json": { + "sha256": "b1553f0eb049ff1b26ac528bd92964eb1180944708c6af1dd8c7033706537064", + "size": 3501 + }, + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/result.json": { + "sha256": "1bc60e1bb1fb4d25337c4987c0f1b26a6ed7e18e94b3250127fd99e26adf8a14", + "size": 1134 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/config.json": { + "sha256": "08ca16ead9d0174e6b0535182398464f61524bfffaf87ebf3aacba7456411aab", + "size": 3371 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/harbor.log": { + "sha256": "d235db2457c75450fa7e34dbbafd223d7347e9b12d232f9b2835e925fafdfea7", + "size": 1353 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/manifest.json": { + "sha256": "c296476ec612eeabb11916fc760674c21e41676e60c1960731c84cd26863173d", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "c8d3e181e52cc2b1b9eb6e237a395a3cbd8e4cc7e275a142d37ba0eb03e625b3", + "size": 563 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "0326b62a237c9c305691bd124df5adc51f10ba6c4158254be75c1ac1eec0e366", + "size": 18741 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "2c6fb8b44b2729ec51ce5d71a4c5861cd4cc86eb717340622654ab8b409db3b7", + "size": 11209 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "dab798c1583b5fe5d98cca1b1dfa770b59cc62878b0eaf467705c3a2c8cc6754", + "size": 16636 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "b21c2432b350f0aaafb370f7f6709332235e190085f1dcad6fa1fb6120bb0b8a", + "size": 13937 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/summary.json": { + "sha256": "cce3de00492e7ec6acd5f733645e9d8303e324ed7404f1dda46c8790240a5d73", + "size": 8772 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/bootstrap.json": { + "sha256": "b23412ff01386d4fbf9f56d0a4b9fc510295fa020e2430ebc634dcaf9fb0968d", + "size": 7240 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/pi.txt": { + "sha256": "5332ecc33747f33cd48d8ab103fdee24ec2bbe1a29f4c3d41b296356819c68be", + "size": 919059 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/pi/sessions/2026-10-29T18-42-12-474Z_01a188f8-3d7a-749f-8e08-de40e1454d0a.jsonl": { + "sha256": "5b26d91b699465179db15bdf5b0ece5565f242acdc4d682c1f59c9e67356f338", + "size": 148756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/config.json": { + "sha256": "1e14825b4ed246a3570e96a49c2f99ed9d3a49ae450b74a21809370601af57c4", + "size": 1796 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/lock.json": { + "sha256": "e124cc1812a896105e8f981a4bc53221737ea4598e62fcc6be85702138dab5cf", + "size": 2389 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/result.json": { + "sha256": "41b225157a865f0a6f8ed41667973a4e850f6e0c952d6cbb2e80cff14403654b", + "size": 16092 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/trial.log": { + "sha256": "c21d59b9a1d96ad41a75528bfbc155ccb5dba884d568f60815cb61a480b21065", + "size": 13756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/attempts/1/ctrf.json": { + "sha256": "07d761f8f96080b884369e844fdfcc058d052889e6cdb66cf1fd5a1b3dc453aa", + "size": 1466 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/attempts/1/test-stdout.txt": { + "sha256": "e1e92e63509e7e0411e2afe54fb5815be89f3bc0f69549be40b9457be525c745", + "size": 2201 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/ctrf.json": { + "sha256": "07d761f8f96080b884369e844fdfcc058d052889e6cdb66cf1fd5a1b3dc453aa", + "size": 1466 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/retry-summary.json": { + "sha256": "08729d1e30969c59bb969a8771ba38e7e6d8ce2a2d5120a6e83452a5cff4d911", + "size": 236 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/test-stdout.txt": { + "sha256": "e1e92e63509e7e0411e2afe54fb5815be89f3bc0f69549be40b9457be525c745", + "size": 2201 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/config.json": { + "sha256": "edae37505c348f05803805fff1d67bd8d368c8e224272d5cf3884b386b92c2f7", + "size": 1835 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/job.log": { + "sha256": "c21d59b9a1d96ad41a75528bfbc155ccb5dba884d568f60815cb61a480b21065", + "size": 13756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/lock.json": { + "sha256": "6072914022196b156e906f60d9ad2f617fd815595a59f57c619a4a3d220508b8", + "size": 3551 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/result.json": { + "sha256": "fc92a0c0c85809882f65b404a88b4c77009f2474126aa71d332e14a674d92fdc", + "size": 1136 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/config.json": { + "sha256": "0fe88d22fc834020c454f69c8157b732d1617ccce74073b116634097e1f9f159", + "size": 3374 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/harbor.log": { + "sha256": "54cc72c25e90db8efdbc5a72167e6a948eee85448dc254f7a90ea152fc066eff", + "size": 1695 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/manifest.json": { + "sha256": "8c862da886aa3b3a6f784b91f50a5ef7034c7d3c5d3c8a6e429982cf74da6687", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "8c6075f2d0d0f228ba84a0d143e01a9c76454bb1b0444c033f76806271f3fc77", + "size": 564 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "c058aa3c2ec015e00cc5fe8a331d1ffdfb2ef850d443ebc71c5a8e042bf24f4a", + "size": 18742 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "ef98fa71c80e6c5786327d86fde4125e19d2f3b4197638567f8c4edba17b2972", + "size": 11210 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "69ffc9bb8a9fe289a21f50d405aeebe8b2838b6b536f4d21867f6dbe692cff23", + "size": 16637 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "4480167360beec73e206189468431babde2a48985cfa760f9e87db6b8d7b7282", + "size": 13938 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/summary.json": { + "sha256": "49c0204f1ffd8b62687d01c3b2fc1e92076d977ec4a0f6be2e8686292363feed", + "size": 8769 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/bootstrap.json": { + "sha256": "b23412ff01386d4fbf9f56d0a4b9fc510295fa020e2430ebc634dcaf9fb0968d", + "size": 7240 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/pi.txt": { + "sha256": "5ebd8135080341ace675a23f03d3385019f77e05191172fdbb3dec37bee16981", + "size": 1276796 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/pi/sessions/2026-10-29T17-27-59-148Z_01a188b4-49ab-742e-817c-9e802e3cadf7.jsonl": { + "sha256": "dfe8494c613b88ff7187df66c38943fb75f6beb1d54bdff9d717e556834bd254", + "size": 298340 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/process-supervision.json": { + "sha256": "01eab71f63ee2e1ff88b2c1f8dc9b5bb768f267c88f23df412a9ea02c4e5fed8", + "size": 76 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/config.json": { + "sha256": "57c53e30eea7da319f08cdc135aabfa8d373ab1207e0012a82b46f77b6716782", + "size": 1800 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/exception.txt": { + "sha256": "dda3b07a4dff43ab0a1b840bf469aa3d324fec8754342014868c03ccf71b4646", + "size": 5612 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/lock.json": { + "sha256": "16d95e57fb4ad46d588987900299bc9c365a1ea0c2fa7be9d9cf625a749595a9", + "size": 2391 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/result.json": { + "sha256": "f7dc6cd62f4f9a06293659a85471067e0d749fab717a0a838811c898ea101a92", + "size": 22098 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/trial.log": { + "sha256": "f1faf9670acc0933065ae726a5ca75d78b281d4d67ac997d0509b32bbc2332ca", + "size": 13299 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/attempts/1/ctrf.json": { + "sha256": "018fab5c38d805e0eac22b47c721730c9cf4bd5b1724d6ce02d83f323cabd7f2", + "size": 2546 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/attempts/1/test-stdout.txt": { + "sha256": "9223c44a71f80eb25e1da5b9a823a3fa84d31408337b7712a45e1ccd3647c4ff", + "size": 2107 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/ctrf.json": { + "sha256": "018fab5c38d805e0eac22b47c721730c9cf4bd5b1724d6ce02d83f323cabd7f2", + "size": 2546 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/retry-summary.json": { + "sha256": "bc74cec712d83796a276dee68ee95df83489dc2ffd76ae380cb3b09529b4eb84", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/test-stdout.txt": { + "sha256": "9223c44a71f80eb25e1da5b9a823a3fa84d31408337b7712a45e1ccd3647c4ff", + "size": 2107 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/config.json": { + "sha256": "770a86eb0193fb8b4a7e0c8746e0004590c9d11a0bc651161220393274411008", + "size": 1838 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/job.log": { + "sha256": "a9fa64568d6209f2effe6caddded2bccf4a374470aa90c65e059c747cf89b447", + "size": 13486 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/lock.json": { + "sha256": "760f51eed10ea5440355ae62d236b29648c515376f75319559ce0ee32fe3ab57", + "size": 3553 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/result.json": { + "sha256": "d834326d9edef8c706f7a58c88427a42729a51fbaa89148064266c8c755c2b4d", + "size": 1263 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/config.json": { + "sha256": "a2e7a359321a123fa1953373ea53b6fc3b87921ec339f606f441e8b77b243714", + "size": 3377 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/harbor.log": { + "sha256": "7385ab36a0cd5cf9e9e8495b357ebcddd11859e8938116bc0d7c270051cb61a3", + "size": 1357 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/manifest.json": { + "sha256": "f4434b014eba820d59dbb4e14e52fb0201c54faab7b3ee4ce51e4000dac19dec", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "c5f50745218c254c8d7716b673c6d52e0236c731462337f81889f2471eaf8c3a", + "size": 565 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "19f26a0678b0f66c6ea925fbc51770f4d3752f5809cc968727e12b78681f9980", + "size": 18743 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "d3a2c22095cb077871ec8ca1522f9bf20eb77d3940a4db8e94ad35e6c3b9d37f", + "size": 11211 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "ee965e0a3ca1c478eab8293877c2bb810364ac93cc59dec66991a69480838faf", + "size": 16638 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "984a2227cf020008c689b7d06f3b0d005b73e162e3f30ebe4c5fdab6bf6b2926", + "size": 13939 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/summary.json": { + "sha256": "593b42d15eb3efc23a086ad1d334d3b3cc88c0c35dbb125c47ec637315ed63c7", + "size": 18124 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/bootstrap.json": { + "sha256": "b76791100f9b0b9298ce9a6ec77dbcc2ed99cd1dfcef15446c54c0f7ebbfc81d", + "size": 15583 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/pi.txt": { + "sha256": "3d4484e8861723c5d2a1255afbbc8f5b2a8d63a45871fd6daf1a189641d7915b", + "size": 465755 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/pi/sessions/2026-10-29T18-29-58-719Z_01a188ed-0b3f-71fa-bf1a-2c8e6ba823a2.jsonl": { + "sha256": "892716738d78bb84f7bb99d94a204ccd19ac503bf8c2365b102d3408a18e4fc5", + "size": 123920 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/config.json": { + "sha256": "0e4d4ec9c6e3f3211561876fab5772bbb5b183f59f579c268297f935a9c91c1e", + "size": 1804 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/lock.json": { + "sha256": "915a97adddeda0e8b3168c159139fc8fbfc8a008196855139679855d82333870", + "size": 2393 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/result.json": { + "sha256": "c8d610ab678b91c24ca6086784d4dfef2330ddb4f6389c961683063082cc47a0", + "size": 28006 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/trial.log": { + "sha256": "cc6378074b39b841821afb16ca6892885511d40d23d2a9d6948141d80f44c328", + "size": 14096 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/attempts/1/ctrf.json": { + "sha256": "effd53996bb3756ed3a1ddc2d7c1753dc720add1033defb14adaa76b22d1681f", + "size": 1438 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/attempts/1/test-stdout.txt": { + "sha256": "7ec35520b3c07ad66f80d9d90716c6cfa1297dfddc469c53a405ce2165ecba91", + "size": 4653 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/ctrf.json": { + "sha256": "effd53996bb3756ed3a1ddc2d7c1753dc720add1033defb14adaa76b22d1681f", + "size": 1438 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/retry-summary.json": { + "sha256": "0dfdd27207b913689972147e73bc1d48356efc199af4a343e27a78de22437f93", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/test-stdout.txt": { + "sha256": "7ec35520b3c07ad66f80d9d90716c6cfa1297dfddc469c53a405ce2165ecba91", + "size": 4653 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/config.json": { + "sha256": "2c0e4f77e8925d58420bd712eb663bfe3bc29289cd186a470477ea4bd5866416", + "size": 1841 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/job.log": { + "sha256": "cc6378074b39b841821afb16ca6892885511d40d23d2a9d6948141d80f44c328", + "size": 14096 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/lock.json": { + "sha256": "a8e8820fcd95641f17137b0f12879285e91c0a223ce6fdacc3ab61f48523e64a", + "size": 3555 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/result.json": { + "sha256": "8883e6a1bbd6183d04c3b3eeb8385e83a08b476c7cd0ef3db35b0a32ee4fd31a", + "size": 1139 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/config.json": { + "sha256": "8696255e70167eb24e3fe0b181f89bd12fe093848db6808ea0bbb5b784919917", + "size": 3398 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/harbor.log": { + "sha256": "9a1b6a8ec94040d8cd10f118002426361dca31452cc8022f140d99e3b5b5e363", + "size": 1371 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/manifest.json": { + "sha256": "0acff64ec34570c28cbc51b872e37818174ebfa341f9c64255eb851b9acc3bff", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "a456cab08265336f6e9061e843a82fcd14b5e3fa038fa62d862032a737623786", + "size": 572 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "49e793616912d213176a0c53c6060396345c34aebc4c3d45b5e2799dc3ca187d", + "size": 18750 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "d39d5cdc2a451354201ff8a34a7cb8ddde61e0f355817d13246ba355e3320e07", + "size": 11218 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "ce83bcc52e7240c4918fe2bef2353d9eefd2ba939102baa0bfd3d5fbfa48540c", + "size": 16645 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "9c7a80ad06e27929ad2f968a6a3f446c52b08e8e84351941629e0297b8f608dc", + "size": 13946 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/summary.json": { + "sha256": "c0a08e7a4032caccddf2f1c91bca361b34c6a0f23a048a02bc16f6d40940a49d", + "size": 21304 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/config.json": { + "sha256": "9997c55e2d9118b74914cf51a56ff2397535c1f875e43e8aa65546dec3a6a00d", + "size": 1862 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/bootstrap.json": { + "sha256": "d874b61c3d815b8791fed422611ab3908ce77c943bbbf631a62343b4848e0708", + "size": 18453 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/pi.txt": { + "sha256": "fa44829a46bc62207103497b888ceeaddb1f52aa1f6edbe44d4b91f93f6579e0", + "size": 120316 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/pi/sessions/2026-10-29T16-59-43-363Z_01a1889a-6983-778a-ae85-cfc9d5b42b8a.jsonl": { + "sha256": "33ace912682d7acba02ad33ece24086f0fc39d55f52754de5af354efd77d7801", + "size": 30495 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/config.json": { + "sha256": "5c318ae5ebfd6259dcce676061a81fc43f37d759520bd8aa2ca281314299aab9", + "size": 1832 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/lock.json": { + "sha256": "e17b822d68afc11bb1a4429fcc492b26e916405dcf16d2013a81bc3a665cafa1", + "size": 2407 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/result.json": { + "sha256": "5facd50971ed4abecd885aacc32a44ded8debd1813010bf628278a3bfdfa9e4b", + "size": 31971 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/trial.log": { + "sha256": "73a38157aa8398d623ab8edf3ecdffe7fec731db741bdf581e2fda22be6e6e31", + "size": 15724 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/attempts/1/ctrf.json": { + "sha256": "8dff70e25efe1a439b8a760c882cad7d3f7494991251038aee815f5d50e064ab", + "size": 1518 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/attempts/1/test-stdout.txt": { + "sha256": "2d642cd113db961ceddee1759cc8daf06cf0c15ecc4a60cc4170ce596011a1c8", + "size": 1741 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/ctrf.json": { + "sha256": "8dff70e25efe1a439b8a760c882cad7d3f7494991251038aee815f5d50e064ab", + "size": 1518 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/retry-summary.json": { + "sha256": "5737855016ac64a38c7947d0a09fafcf499fcd2defeed10c1f13a1274de7381c", + "size": 235 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/test-stdout.txt": { + "sha256": "2d642cd113db961ceddee1759cc8daf06cf0c15ecc4a60cc4170ce596011a1c8", + "size": 1741 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/job.log": { + "sha256": "73a38157aa8398d623ab8edf3ecdffe7fec731db741bdf581e2fda22be6e6e31", + "size": 15724 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/lock.json": { + "sha256": "86d20b4bb7f4835ab5eefaedc6d0642dfb1245461debcefeaba8f5b627a54284", + "size": 3569 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/result.json": { + "sha256": "e1ca99ff4fdad214d3150fced859f41404d25538ef67230a6af79742836e4e3f", + "size": 1143 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/config.json": { + "sha256": "1bc62edf083d5204f4e804592fdf26bf392258ec080fdbd7068726132c5c0a63", + "size": 3404 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/harbor.log": { + "sha256": "b13f15c4f4c9d5cd719191b8a0d059bf34d4f2730662455da4dad6bcd1065a4c", + "size": 1375 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/manifest.json": { + "sha256": "9c0ab477fcc23b995c03b118c97f2ede8d9b2296661da20213850c25803aa41e", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "61562968b9d843d9373d04b1a42fc299acfb3ce2e97f15bdc81424882bd9c82f", + "size": 574 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "4063933dc7f52e78ebc0e6280b780acf678d65922fc6824074a729e77f895630", + "size": 18752 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "fc8c7ec7f50247f96ee1deae77383a98299d7e9462014323ebb1e9b4b1bd3986", + "size": 11220 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "077f35b03a2656e3ca9ff151d2f8d9f6b0962504193d0cd6b364857e1acae399", + "size": 16647 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "4da0f3c0264ae14290f384ebcb36abb1b5c8ee044495e2e48b2c649b22ad2c10", + "size": 13948 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/summary.json": { + "sha256": "3b04f441fb1d65ab06a28cab5b0788bb504ca7c7a01569cc898dc943b15be264", + "size": 5938 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/config.json": { + "sha256": "d974db5aa04b5e2cb04de3a7e594de7acff4ce37461ee566058ae68ca2be666c", + "size": 1868 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/bootstrap.json": { + "sha256": "98cf8aeddd43ad3501c96b3459571091e0a9dcec7703d27a1a258c938fc4edc9", + "size": 4680 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/pi.txt": { + "sha256": "3e6985e0be6984c0c94503d23011886171a069edc3486ddf6f95d24ba821979d", + "size": 763554 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/pi/sessions/2026-10-29T17-06-41-779Z_01a188a0-cbf3-720e-a5f7-f2ac55d6af66.jsonl": { + "sha256": "db5dab4f9e1b081c384a7c7eb74055c3d9940f9793df5836643308212801965b", + "size": 133423 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/config.json": { + "sha256": "6fb057690bc2cabf793a44d43df3d68e306ced1f1c6fff166566b00e9bc5f94e", + "size": 1840 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/lock.json": { + "sha256": "1216027140eb9f03ac79c3cbf7791268cdc33804d29bbc71d140269ef02b1da4", + "size": 2411 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/result.json": { + "sha256": "b93b51681e1e8a69f36f813c5bf39b64f9689c58abe7e6abd64715a5f9893648", + "size": 12587 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/trial.log": { + "sha256": "8470b1ae47449959fff007a94accf7f636736e5d9ee287176b215abf8f4df930", + "size": 14252 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/attempts/1/ctrf.json": { + "sha256": "0056fc87d2eb088ad9130934aa4101fcc87402f5aeba6d05fdbf4aab9a8a9f39", + "size": 2544 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/attempts/1/test-stdout.txt": { + "sha256": "dd49fbab1cf0c35f05a52036e29277dd4d11e830580c728a766bb28eed05c8bb", + "size": 1904 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/ctrf.json": { + "sha256": "0056fc87d2eb088ad9130934aa4101fcc87402f5aeba6d05fdbf4aab9a8a9f39", + "size": 2544 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/retry-summary.json": { + "sha256": "dda00a29b5ee453eaed55618736007ea4014d5ce0183f92d8dfc835360dab645", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/test-stdout.txt": { + "sha256": "dd49fbab1cf0c35f05a52036e29277dd4d11e830580c728a766bb28eed05c8bb", + "size": 1904 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/job.log": { + "sha256": "8470b1ae47449959fff007a94accf7f636736e5d9ee287176b215abf8f4df930", + "size": 14252 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/lock.json": { + "sha256": "6c0f97087f585e1aa0bf4513fca9113d0eef46e77171de97295e27d5b2dbcf27", + "size": 3573 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/result.json": { + "sha256": "a409c9ea95067fdbc9c1bce735a54b62c66e4942b2c7e4f6f2a48077519a7eef", + "size": 1139 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/config.json": { + "sha256": "fd1b2e493cf4983544ca2e6f693412568bb1795c5329b9e2c8f079eccc91623e", + "size": 3368 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/harbor.log": { + "sha256": "583fcaa042aec526502bbd2d9bf2120be7959a379138411c5156653c5ba50d51", + "size": 1351 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/manifest.json": { + "sha256": "fd242f7e3d7dcf272f903e2f651b555522ff8fda7b8465f54f2816d2247d8ec5", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "afbc896b1d7da542ade0f9ade9537c0e0a6c4383a3746ae79c386c21a6e2de14", + "size": 562 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "c79e80a82ca78aa7410b70454e409fbd50980fa6b0484b794a27a91dbb358565", + "size": 18740 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "afead18ed9b74224728aba8e0e0fcc52df9e1227aed7ab03c146a367a975dd43", + "size": 11208 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "94948065f3e80fbe995cd92452767ba68470b1b83d31358a024d46de7ff36421", + "size": 16635 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "a59d908ae03adde93185bc29926a3b03958e2abd50994c0e01eef81904b026cb", + "size": 13936 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/summary.json": { + "sha256": "3194fc12595b697dd2e711dfda2af60988029f7ff5475fe5fb0d2ed0faee1788", + "size": 26513 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/config.json": { + "sha256": "2e9947944ff9a99581f9f9703355672c3dce9d8b23e8429b86d702895b5b356c", + "size": 1832 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/pi.txt": { + "sha256": "e4ceca47f1b1894c95570b5796dd77685869d2bec48cb12de7593f10a7759099", + "size": 1863596 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/pi/sessions/2026-10-29T17-06-58-601Z_01a188a1-0da9-74f7-b08b-427d3e25a283.jsonl": { + "sha256": "947e4b1acad551912170be931337ceb32f148aac77b6529bad7030c092de2529", + "size": 376580 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/config.json": { + "sha256": "e483fb1d3bb39cd3d7aa2d3128756941445d567b7c23012200444cf771b772f6", + "size": 1792 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/lock.json": { + "sha256": "49c4bfd97e78a8633b50ddf82defdb1b1d86e175bcc160af692128d8a8c0fac6", + "size": 2387 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/result.json": { + "sha256": "8b0d37e25e0420b131d1c105b449aa34e921caae9a74fc115271aa0ee1979729", + "size": 38485 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/trial.log": { + "sha256": "2e97c1351871afb0db39c25ba1d0a7f679c28902b3556aa506cfc7585e5eec1c", + "size": 15247 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/attempts/1/ctrf.json": { + "sha256": "98ffbab8681c78be529a1632e8d3019a9cac5fda91d10c3e6e8d776de869b426", + "size": 5874 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/attempts/1/test-stdout.txt": { + "sha256": "edcf5638abea37a344092e747053bc314a2f80c447f5e142f88381371bdb4ec5", + "size": 6505 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/ctrf.json": { + "sha256": "98ffbab8681c78be529a1632e8d3019a9cac5fda91d10c3e6e8d776de869b426", + "size": 5874 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/retry-summary.json": { + "sha256": "c4a0ba2d33e097e744c23315d69866f662116e0360d51e6c3ae9ed5426c5f756", + "size": 235 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/test-stdout.txt": { + "sha256": "edcf5638abea37a344092e747053bc314a2f80c447f5e142f88381371bdb4ec5", + "size": 6505 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/job.log": { + "sha256": "2e97c1351871afb0db39c25ba1d0a7f679c28902b3556aa506cfc7585e5eec1c", + "size": 15247 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/lock.json": { + "sha256": "ef32e1ec7607673f25a6f9a68b95eac0a853f9a38a2ab90632205b69f0fbf88f", + "size": 3549 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/result.json": { + "sha256": "09fc19cb7df3d2459353886b78cb94b6e75de651367765a8df54d05ab91398b6", + "size": 1129 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/config.json": { + "sha256": "d3a7eafa2718d6de07c7e1578c97e2d32564b943c228207b659ac624d3272a18", + "size": 3377 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/harbor.log": { + "sha256": "17e4d38efb5cadadc1da615c02f58a45fbfaece6fc8794f5182c08be9b8dac20", + "size": 1357 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/manifest.json": { + "sha256": "3adde491396731dc6324ea9229eccfbcfd29c206f8fcfe74278d1abf899fb20c", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "c38a2bbd90c6f9c5e306d60323e2b06e45beecd89d4cdf66d550f64c077e1a9b", + "size": 565 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "7a5942cd5553a08c3454caf68ff6709b0f26827e3931929c371ff6a7c3cbcedd", + "size": 18743 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "3303b0b8b92e05d31340bfaf3a4926d134765b875bb039a208ceb5dc4e1b0701", + "size": 11211 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "626273580337c750ef39abaf956fa5a651b226acdb082ce781e7f228d4fd2d67", + "size": 16638 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "7c579576ecd608db09d81dd608ead0318e59c95a4331e1abe82750b0bf854d06", + "size": 13939 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/summary.json": { + "sha256": "29b65ad79ca15f5c988ed76c69352c123d96c7148ea0b36877f9da38749ad9de", + "size": 5919 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/config.json": { + "sha256": "74453837cab96f692819ed9f0446db84b9d76728ad0adfaa07e05de4a30cf872", + "size": 1841 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/bootstrap.json": { + "sha256": "fe13940644c93d5d5dfe12c7d55d0a43083a179072b47f4c917e32ba533f5b3f", + "size": 4680 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/pi.txt": { + "sha256": "918ef5e9fccacf78f64001583e163cd851a1992ff99c8afc590fa25fe6bd5a49", + "size": 1226918 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/pi/sessions/2026-10-29T15-35-34-048Z_01a1884d-5da0-7570-896d-bae897aac6ea.jsonl": { + "sha256": "4b5301b354a02ec96d4f363771731811ffa3b0f53ebd5f787f1fe3959925b51d", + "size": 188236 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/config.json": { + "sha256": "561ff31b1ac99f5d45a04ed04669d151f18472c0e50d4f74b0c3d1144d1ca239", + "size": 1804 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/lock.json": { + "sha256": "6a416940f873407430129e647699f64b01eaf178bc81cde052fa1502c29258fd", + "size": 2393 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/result.json": { + "sha256": "81755b70671e579c34177c69887e976560af8185db0da1b248bccc7c2a49db1b", + "size": 12506 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/trial.log": { + "sha256": "2a1f88b4559a6178dd9dbb3f5b90ad5a518b0dc3f47982c80308a521cbdbf474", + "size": 14338 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/attempts/1/ctrf.json": { + "sha256": "a048e4295aacc22be4569ad39015210ab33f5f9d769ae5c3a24daa94c7ea72ba", + "size": 759 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/attempts/1/test-stdout.txt": { + "sha256": "79b4b15c050bee004a2e5d6ab18d14164da2d11229e144f7b0c75c0e64f2b73e", + "size": 9486 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/ctrf.json": { + "sha256": "a048e4295aacc22be4569ad39015210ab33f5f9d769ae5c3a24daa94c7ea72ba", + "size": 759 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/retry-summary.json": { + "sha256": "05acd31af122723c1f5b403545f619b4c81caaf4ca537f20806e18eac76bf181", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/test-stdout.txt": { + "sha256": "79b4b15c050bee004a2e5d6ab18d14164da2d11229e144f7b0c75c0e64f2b73e", + "size": 9486 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/job.log": { + "sha256": "2a1f88b4559a6178dd9dbb3f5b90ad5a518b0dc3f47982c80308a521cbdbf474", + "size": 14338 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/lock.json": { + "sha256": "6781a617a74971a52e8a3b778c316b4c0e577b32af3fe6f1544b3f2ced6399fd", + "size": 3555 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/result.json": { + "sha256": "e131b208a5ca4ee9105c3b00bf1b9dca813a239c45b7b5dcf7dfc712ffa0c035", + "size": 1130 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/config.json": { + "sha256": "658c2fdac5e4cc78c9d1bc84a345a8210e6e7b2c65fc81843db7c7c645774bde", + "size": 3371 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/harbor.log": { + "sha256": "cedce451bcad0c3208848673d78f60ad085ebd3f95638d6c1455e11e1b374438", + "size": 1353 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/manifest.json": { + "sha256": "754381e108810b32a76e6a76bfcc235890edf0839c7ed15373f2a889aac25be4", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "7294f003e90018beabf74389e2e3767deb4d95c24c8d5452f44b08e5a9d95dd7", + "size": 563 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "6d590d8a6f086bb49e13896fbd9137a83676002c8c51956338fc005f45c4fe81", + "size": 18741 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "cc105d9f509c388f1f7f1dd047750dd127484629ca243418852bb5c15893f6f3", + "size": 11209 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "da4b93874daaae780ed1238c26f8d9645852b6f6a16971c8292748c95031377a", + "size": 16636 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "1421b620b31abd82b89fe4ec8f692aa7ddd5ccabd206acfd307e0ee35cfe5d78", + "size": 13937 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/summary.json": { + "sha256": "8c3dc32a941b4d791626499adc9c4a405e7568ba76ef082d654fc33f0a57f3e7", + "size": 24707 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/config.json": { + "sha256": "37d037ce53f0a9e5e36d02395eaad8a5c40c604bcdf4d0a9a2044861bfc12412", + "size": 1835 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/job.log": { + "sha256": "57e6c931e4456bfbff9318bd8f253831d8e6bc64f2727cac2410f238904624ae", + "size": 14369 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/pi.txt": { + "sha256": "d609be21182b1ab3065b9c6446b488e241734f852842800e8a3542d059c9c0c2", + "size": 106409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/pi/sessions/2026-10-29T15-35-37-712Z_01a1884d-6bf0-737b-adac-4680ed18e3c6.jsonl": { + "sha256": "613b7d61d483c9f783698971771d7288e0b4e3807eedb694156f7bb4f065eaa2", + "size": 21948 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/config.json": { + "sha256": "37acc1733e8d9d6d65ed592b6356b24d1780898a6233f61827f81e6a7bd39c4c", + "size": 1796 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/lock.json": { + "sha256": "11a2cbd0880ce54278963f07a504c7bd199e25cc0a650d4a8c97e8de1f69feeb", + "size": 2389 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/result.json": { + "sha256": "d6ccd8a5b086a93e78f0f2b9469e354c9fb0803057827248e8d23c093a68cd26", + "size": 36374 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/trial.log": { + "sha256": "57e6c931e4456bfbff9318bd8f253831d8e6bc64f2727cac2410f238904624ae", + "size": 14369 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/attempts/1/ctrf.json": { + "sha256": "f7b5b93680f7868c1826d5139ae4b8a83f53b30712415876d08e82637c11238f", + "size": 2877 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/attempts/1/test-stdout.txt": { + "sha256": "23272c762d0632a6e83ec7dcff2e9b33e9c1a7ecb48b8dd907e0e80f18da2e2c", + "size": 4586 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/ctrf.json": { + "sha256": "f7b5b93680f7868c1826d5139ae4b8a83f53b30712415876d08e82637c11238f", + "size": 2877 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/retry-summary.json": { + "sha256": "a10e698b35615d29d9197f68628a013d3b3d05f62c552b05f2b4fcb548ab0d91", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/test-stdout.txt": { + "sha256": "23272c762d0632a6e83ec7dcff2e9b33e9c1a7ecb48b8dd907e0e80f18da2e2c", + "size": 4586 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/lock.json": { + "sha256": "15f91c206e55241786bb72d7cde32abe9060bfdad9c9e57404ae52054ed6c601", + "size": 3551 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/result.json": { + "sha256": "eecc55cafb30986b11900eb18a9b0d1e1be1dd1c668a9a5db780aaaf6e4eaa7c", + "size": 1135 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/config.json": { + "sha256": "26d8838951211379f94d51547dac4b90244f7dc74f8a8179b37fa40d72fd5410", + "size": 3428 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/harbor.log": { + "sha256": "c149db81229b0cf3ad0448ce902ff28326ccae6f0cd84be6e590ffc81692e95a", + "size": 1391 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/manifest.json": { + "sha256": "1d3b21608b6a16219cd05fc58e899f43f8af27f47d6c25ccd5d48a691c549b80", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "721230e24e5ce2c84e55e7d5579d0e7da4374ddb5db480e9e689154fddef4dd4", + "size": 582 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "0c63d5b6bc6df0c4b14c8873a39dff0741887ac1a0d8df48f21884199ba72504", + "size": 18760 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "1c329e1771b025cd49477b47a01a91fae197e762fe26a2ba22792521ff8559ca", + "size": 11228 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "287ab27e191002777a3b5b6bcc8bdf611f146549f03745b80dd1ab76babf3a7c", + "size": 16655 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "cd7a5d688724f344277feafb2f9b00b369716274dd5221a1bd9355b04f0d2c2e", + "size": 13956 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/summary.json": { + "sha256": "eb33d0738c95b855034c64d2f4caf0ec12682454cfc885c631c683fee7abf9e2", + "size": 24746 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/config.json": { + "sha256": "5292a949628ea0ac64c2ed52b434774e07883a289dea50fc66fa4e4348d7bf6b", + "size": 1892 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/job.log": { + "sha256": "72361d0767eea28ba613abcf86da07d6761b1b7208539b7e09208b9363ec460d", + "size": 17626 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/pi.txt": { + "sha256": "4d90e51f4f840f48df1747fa6ae8c7d8ea7f7d2fea399d7c866e0b0c10d0a330", + "size": 809939 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/pi/sessions/2026-10-29T17-21-07-841Z_01a188ae-0301-7023-953b-0d86de4e7501.jsonl": { + "sha256": "7aac057c21d7b8eb93a71f2ff7a6b118a65961002ccdfbbd606b690100394fcb", + "size": 183265 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/config.json": { + "sha256": "39b2a3f2ea0b7db3565bc7ebb8191481c60ae27f380a6b052a364309fceaa055", + "size": 1872 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/lock.json": { + "sha256": "93048d65f92b04405f51dcf8e2d10f77eb5f728aae3437bd333f0861a2ee5e99", + "size": 2427 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/result.json": { + "sha256": "11358254845ba02d43b2d3be41ec8b72a268110b4d94be29244636d7b7cf3a82", + "size": 36539 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/trial.log": { + "sha256": "72361d0767eea28ba613abcf86da07d6761b1b7208539b7e09208b9363ec460d", + "size": 17626 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/attempts/1/ctrf.json": { + "sha256": "da3dda5cf9f2f21e64e75b874169d4cbbf8f0270606ba48ec87df23d60609a97", + "size": 2549 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/attempts/1/test-stdout.txt": { + "sha256": "15f4c60f8c924888b2d5e7008aad87993bb6edbf11dc6b54de9f8ec693d6792e", + "size": 1715 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/ctrf.json": { + "sha256": "da3dda5cf9f2f21e64e75b874169d4cbbf8f0270606ba48ec87df23d60609a97", + "size": 2549 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/retry-summary.json": { + "sha256": "1ddf03e2d3080934b3844b3a63f14466630af002e02da82cd66b7a3c966c8f6e", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/test-stdout.txt": { + "sha256": "15f4c60f8c924888b2d5e7008aad87993bb6edbf11dc6b54de9f8ec693d6792e", + "size": 1715 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/lock.json": { + "sha256": "4a40a75457dadd07b50558261a2427993ba5e3a7ea0ca7167130f47800ed4616", + "size": 3589 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/result.json": { + "sha256": "b3223aca95466df4b5c4d49e8256a202964cbb948a52032f1e617f3a15afc032", + "size": 1147 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/config.json": { + "sha256": "d52dc9abf1c825f913a594f918110770163d343a90716c49b2611df948b7ebc9", + "size": 3401 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/harbor.log": { + "sha256": "b90d70409acbb6ccbed3f7cb4e7f8f9095b12ac8d90def187f586b9f713959c6", + "size": 1373 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/manifest.json": { + "sha256": "1d2c4e828148a59f2117cb757884e056fe87185faa86a7b537b0c6c53482853a", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "1908df5c75b2d42e605d080d50362c62d7d974b2c92fa9b0a0c7b09dfc59a8f9", + "size": 573 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "d322fac5e5d3b090826f86cf7192e894dad1fcfdac57beeec68a2c9ac53a7d3e", + "size": 18751 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "8f6ed9c0fefaf4fe2fa60f6123f6c8e63b052e4c770f234d72db794806240ebf", + "size": 11219 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "3e265c36ef48ad0c52f6277c02e3db20282cf748e607bc5140d1e36df6955cfc", + "size": 16646 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "8fcea6e2ade4aacdb69d4ed141cba5de7ef262f3c4c007f9ae3435bd37334ccb", + "size": 13947 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/summary.json": { + "sha256": "de9991e4e96e23c73409c441cff012986ffe61d49c1dca76d38775d3394ecf35", + "size": 24728 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/config.json": { + "sha256": "f87761df2b4c6232c5991a27bd927c03ddbe308b7e2fa625e96573618bce3757", + "size": 1865 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/job.log": { + "sha256": "07ca46c2fccb0a1d4bad23a0b861d58a3eeeb8ab5548e5885a1021fdd0fa90ee", + "size": 14636 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/lock.json": { + "sha256": "5c8036196755d144f60f909272d5077019fbc9243a644fb65e060a9980add13b", + "size": 3571 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/pi.txt": { + "sha256": "10e3b070ca0349b9a4275bbc3fdca4a1c55e9071494c2655ffd8e6c1959a1b4e", + "size": 171854 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/pi/sessions/2026-10-29T15-38-41-493Z_01a18850-39d5-77dc-a8af-1dbd1e8344ef.jsonl": { + "sha256": "d4b620cedd0503581c6744e036ab51827f4afce72e2d28271cd744522be9848b", + "size": 31819 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/config.json": { + "sha256": "1c25fe6c078e57711c67e3dd184f11b0230162d6a3b0fea31f15666d1047217b", + "size": 1836 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/lock.json": { + "sha256": "cce0132cc4d2b4371d3b6ade69b9e0fd9d138704ae99f18070bc8fcc371e807c", + "size": 2409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/result.json": { + "sha256": "174cbf66d5d2534ac24703f63abcb6b534e28c0feb24c267c9559829bd763260", + "size": 36454 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/trial.log": { + "sha256": "07ca46c2fccb0a1d4bad23a0b861d58a3eeeb8ab5548e5885a1021fdd0fa90ee", + "size": 14636 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/attempts/1/ctrf.json": { + "sha256": "45e58a40f710f53a5e0ad84994529e8aa60a678136905f38b61fb234d1e7d4ae", + "size": 2488 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/attempts/1/test-stdout.txt": { + "sha256": "220818e1eb90f5519d0670fceebf9c0a291fee10cb3cd1f915e5936d27da1aee", + "size": 2787 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/ctrf.json": { + "sha256": "45e58a40f710f53a5e0ad84994529e8aa60a678136905f38b61fb234d1e7d4ae", + "size": 2488 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/retry-summary.json": { + "sha256": "14655e123944a366ec87ec96825f488b6255ad5a2c91da8746be45ce6a3a2410", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/test-stdout.txt": { + "sha256": "220818e1eb90f5519d0670fceebf9c0a291fee10cb3cd1f915e5936d27da1aee", + "size": 2787 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/result.json": { + "sha256": "7e839dbea35f277d5b3d1edd07b2357fe3cb72c60dc2e5d61054a7d274e8c9a0", + "size": 1135 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/config.json": { + "sha256": "1ecaca7f34984ef20966a66f32194a25cf09fc2e2a40c3cda5a0ddbfeedbdecd", + "size": 3401 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/harbor.log": { + "sha256": "60543ea193e1ee629f045726e28a97a51d6a8c904f8894e7b0dcbe2c2a5b848d", + "size": 1373 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/manifest.json": { + "sha256": "34e505ebc87479b6dfe2afcd608430dde357ff39332f7741ae20b4624c7dd3d6", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "bb3c989cb13efcd81a286e0eb4e621113e9dc5e7009dc2e919ec3d783a35726e", + "size": 573 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "9a31a232d28d17775521a0c9ef977c2acb2c56fedb422f7c00698cd871b724f3", + "size": 18751 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "6d0f81d75841627593ae2c2424f028fd0620365e811e2be45fb642895b7a364c", + "size": 11219 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "299b03b455d332530cbc95ff0641cffe1fc4a6d2570533b9d59f8a5721594f28", + "size": 16646 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "5110397f2077337d35b0e93e426cebf718c57cda86349fed32d74c7e7f5435b6", + "size": 13947 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/summary.json": { + "sha256": "236f20ea062837b3c9ac7c9303223d8695e5d9bc91978bc9650d3d105bcda5d3", + "size": 26533 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/config.json": { + "sha256": "3161bfc4545a48e1e80ac6dfef82890b286e7a588142f575a071205476b8c03a", + "size": 1865 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/job.log": { + "sha256": "bdbeb3482ca77fde53d09830dc648f6ff91b517b339d695a92baa49a9ca2cc9c", + "size": 14874 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/lock.json": { + "sha256": "b5653ea7b9776024102efc05dbe9aaa2d44dfe1a4a4a25085eadd2d6cc14a729", + "size": 3571 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/pi.txt": { + "sha256": "ffe0cae91c5440baa645a58ee336f87b1d24db586e4dd97f2e3132e18f48a281", + "size": 284853 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/pi/sessions/2026-10-29T15-38-42-143Z_01a18850-3c5e-7077-beec-004bf7f832bc.jsonl": { + "sha256": "252baada49df27f5100f9a2811800586370bd79ffb0c899281e3eb16bb22ab0f", + "size": 69970 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/config.json": { + "sha256": "f5f0b928341b4c73a018ed92670f2bf9d4516365b5994a73bea309556ceb27ca", + "size": 1836 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/lock.json": { + "sha256": "c62e056f1799ce8db84462afd89a804c4f0101fe6721f7dae2e20273d71b2d82", + "size": 2409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/result.json": { + "sha256": "c5175b21652aacf209836348c5e61f0c3e32c157a591874e2f87c0ee897f7393", + "size": 38582 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/trial.log": { + "sha256": "bdbeb3482ca77fde53d09830dc648f6ff91b517b339d695a92baa49a9ca2cc9c", + "size": 14874 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/attempts/1/ctrf.json": { + "sha256": "cafa9fde8ef217ca8ec9828d59cf5c1a4c336b43e730f5a0d964e13dd599eb02", + "size": 2169 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/attempts/1/test-stdout.txt": { + "sha256": "72e16bf9633de796c88a7a1bc3e611f9365e59a5bb14fb9dbe1ef1ec32b85bef", + "size": 1860 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/ctrf.json": { + "sha256": "cafa9fde8ef217ca8ec9828d59cf5c1a4c336b43e730f5a0d964e13dd599eb02", + "size": 2169 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/retry-summary.json": { + "sha256": "13ec54081d2e0bc96bd9eaca96bb5d5e16654c69e198bb90a8cb83441da073c4", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/test-stdout.txt": { + "sha256": "72e16bf9633de796c88a7a1bc3e611f9365e59a5bb14fb9dbe1ef1ec32b85bef", + "size": 1860 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/result.json": { + "sha256": "0192717743f949278ec9449c7ce19b5014b71530274a80979d4ae15b2e45ace2", + "size": 1138 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/config.json": { + "sha256": "7215b0da05d3e8b922773d56ba4acd479feeedafe630013a88790d923fc8d2bd", + "size": 3416 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/harbor.log": { + "sha256": "8418dde30e15a9cec093aeac498e459dabecbafc109b3bd2ddd84a3d6746445c", + "size": 1383 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/manifest.json": { + "sha256": "5f6340022a2bc1ecf0f9fcb410a3a290e9be9d8382fc442cd99b933ff29ec51a", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "9be7268af7c97f36b608b6e1fbeb888a0a1ba6a5ece1b510a8e1daacccd3c88b", + "size": 578 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "95ef486909cfc48d91bc04f6e5371950d3785b698931bff816718b4f6c9b16e5", + "size": 18756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "de4dd7eb4e48e56b92f18036d9b79d9b5b5598695da4d4e2b43c19f45474e73a", + "size": 11224 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "cfaa8036f3b2bbdacd568b995c3949c55bc44c0263cb03928985cd912111ea38", + "size": 16651 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "6911f6b14f1450bad51f9baa6f6f0c807e1841619ba015bb4a703899c83d0ff3", + "size": 13952 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/summary.json": { + "sha256": "8ce151b0f76b34914bf967f77599fcc58b5f29bffbeb461177d99bbbc35ed8fd", + "size": 24737 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/config.json": { + "sha256": "92657955afa4cea13edc936a1045bfadeedbeafa2286488aa77d4c7b3f70118a", + "size": 1880 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/job.log": { + "sha256": "6f5a89714f8ad9e082e26c4ddd0cae2e154f82db7446cacf35ce61eaab02c3b5", + "size": 13821 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/lock.json": { + "sha256": "b1b2375fbd427a87a1675a5339bb25cc7ec3a07114e6f57d60392d184c8bf7ef", + "size": 3581 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/pi.txt": { + "sha256": "a4aa2df98a59efe892a8fff767669937902579d35c759f909f7f1daf753e703f", + "size": 539432 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/pi/sessions/2026-10-29T15-41-52-509Z_01a18853-23fd-7011-b129-e246bd10dc30.jsonl": { + "sha256": "2940d89f18c166a4d467d8de76a017784e86c1ab56ac0d1617b0f4b7f102377a", + "size": 126186 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/config.json": { + "sha256": "a965ad064bc4530fa27d6304f7e178f0532aeab0847ccfe12682c252f7526844", + "size": 1856 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/lock.json": { + "sha256": "491c8ad79efbc6967000dc5bf9f016fcd62f448579234c70c75536fed154e67d", + "size": 2419 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/result.json": { + "sha256": "58c7bfebd385b768fc80361b9312d4610dcda43bb8fa52a39f356e13af059370", + "size": 36512 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/trial.log": { + "sha256": "6f5a89714f8ad9e082e26c4ddd0cae2e154f82db7446cacf35ce61eaab02c3b5", + "size": 13821 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/attempts/1/ctrf.json": { + "sha256": "e4b2de3d8f83e623fa58e701c2dd72222270a4a4e316b3fa8023912250573377", + "size": 2369 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/attempts/1/test-stdout.txt": { + "sha256": "3eccc41a5675886aaa7462692b3680678831b286d1d0af59c3e5f20fa9a306fd", + "size": 3075 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/ctrf.json": { + "sha256": "e4b2de3d8f83e623fa58e701c2dd72222270a4a4e316b3fa8023912250573377", + "size": 2369 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/retry-summary.json": { + "sha256": "f05f017f4954a9c8c73c4bb0f851ea35a06b71792b5d2b5ba7fb93c5362a108f", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/test-stdout.txt": { + "sha256": "3eccc41a5675886aaa7462692b3680678831b286d1d0af59c3e5f20fa9a306fd", + "size": 3075 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/result.json": { + "sha256": "bd9f522aca5e6f2d7e8ab4c73bb9ad2544fe4fe9075fa4d2bd3ae05188051e15", + "size": 1152 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/config.json": { + "sha256": "ded6f45b014a2be3f6332a78455f198be5fb1beabef225342247038f2b1e9a93", + "size": 3380 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/harbor.log": { + "sha256": "0de98a4427d3d05f8f0139d746ae08282caebcb028c5bedde75e241add801fd4", + "size": 1360 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/manifest.json": { + "sha256": "20a97bbcfb54332d8782d695f5b49f719dcda95077c53094d9b5cb28c389d770", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "a00e51e2e358d6589fa8269be3767b311976b32b7bfaedcdbbcd7ed19e09d0de", + "size": 566 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "2ea2fc97ed291dd5933e8146a51f1ac4a7d9a4f780b1e1764dcc056b0a0b5da8", + "size": 18744 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "7973f76856ff6773621641841c254de05056bbcbe8cbec5245d78ca29062c9b7", + "size": 11212 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "4365b7dffe90b64e2801c6b531b7648fdd04360d6ae6cdcb047406dfb8e294a0", + "size": 16639 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "33bbc04900b29250fb525e82546bc07d1081e1549d19c9dc7fb5c21447a83c09", + "size": 13940 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/summary.json": { + "sha256": "d8db06b24cb2fdc8bffdb560863d228aad71c0f817ebe8511b8e2f6641da7c19", + "size": 3388 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/config.json": { + "sha256": "e6033ae25b93c50b61eca21a8ae8717bdebf508efdf76179f670f216a509afe7", + "size": 1844 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/job.log": { + "sha256": "23fbd572096f0d7341de1963588bc9762a96eb110248666c207e4b531b30196d", + "size": 13633 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/lock.json": { + "sha256": "043c8ce17fea5bbe013f5ba491129c008e945a00437ed05cbd3eb4578c2ada1d", + "size": 3557 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/bootstrap.json": { + "sha256": "55b77aeec36ffddf57314f4522d2d3bf1d598d484badbde6dbadcae955cf7f06", + "size": 2397 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/pi.txt": { + "sha256": "5a2b8ba48a04eb6378ab83bf730add971fc6670cc851e01b13570821daffd0b2", + "size": 21980476 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/pi/sessions/2026-10-29T18-29-52-892Z_01a188ec-f47c-779a-bcfe-942e9c1dc92c.jsonl": { + "sha256": "d0e13345c2ffbe77187300a4f617d55321769d9242c50fcebee4e16f108310cd", + "size": 480736 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/config.json": { + "sha256": "7fdf65c5d064bd384e42cb193bbd0c4bfda6a17158a1a2f80164f1a24f9525fd", + "size": 1808 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/lock.json": { + "sha256": "2c09e994f6548be29f4afa5bf45c94310b09feda78440ddf9be013d87a9a3858", + "size": 2395 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/result.json": { + "sha256": "235ba97614b25344dbcc206e4ccb52774ef1cf7e4dc8dbb9aa5caba6b7318b7e", + "size": 9354 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/trial.log": { + "sha256": "23fbd572096f0d7341de1963588bc9762a96eb110248666c207e4b531b30196d", + "size": 13633 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/attempts/1/ctrf.json": { + "sha256": "bb3c9f675d09d78e5612074bbe1b7c8a1986fb5b53cab949cb8800f3e9bdd50b", + "size": 1095 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/attempts/1/test-stdout.txt": { + "sha256": "1367d9e0635a3e4bf7b7167f5f427711e80ec241ce6f01c9265fa1980328c217", + "size": 1547 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/ctrf.json": { + "sha256": "bb3c9f675d09d78e5612074bbe1b7c8a1986fb5b53cab949cb8800f3e9bdd50b", + "size": 1095 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/retry-summary.json": { + "sha256": "684e04c1382a7d36dd9d4f2986aecd4ca5adcb8d3ab519fb3c7ad47be1998c77", + "size": 235 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/test-stdout.txt": { + "sha256": "1367d9e0635a3e4bf7b7167f5f427711e80ec241ce6f01c9265fa1980328c217", + "size": 1547 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/result.json": { + "sha256": "a41a1cf270b1ee9eaa6bce36e63eae36337c9c33bb767c085db82ae7dcd0da2c", + "size": 1141 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/config.json": { + "sha256": "d4e4f9cc09913c65a63f6732703a66b9a709d865553f58c1828355b528e4a9b0", + "size": 3404 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/harbor.log": { + "sha256": "a6ee37aa39010fa880e7982834a4a4855d6d5f465847806a806ea73de1c844fa", + "size": 1374 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/manifest.json": { + "sha256": "5d6204817bb144202ef40770319b444a95684974dd8bcb6ebba490ff3530efc5", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "0a58abd7732c4441992f645fb53247ae507352661cd6b65305053ce71859ed53", + "size": 574 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "c7a7dfc01c6a186068fcb736ee310292af48fa70fc5853901623b64fffedc27b", + "size": 18752 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "d6fc4357ead15cf4c7eb727d425a27a8951d6ddacf3453bd5a8b5436d20c7054", + "size": 11220 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "b97683b57f2103abd6f8761b9e74474dbde6efce63c5611a6a98003a5cba2b11", + "size": 16647 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "46720494ce6afd454c899b380722cf3c1081bb8668e53887dcd7e67c168a94d1", + "size": 13948 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/summary.json": { + "sha256": "e6100147951a90477afaf82fd06726add0648018c39d5c047165b2f11910ceb1", + "size": 22584 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/config.json": { + "sha256": "2e5482a53b3e592191b7027fe331be219226048207edff462dabab105324c7a6", + "size": 1868 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/job.log": { + "sha256": "f1253016a8e07cba1f658996e5d5ac5262155c7f830d5a59a8bd47f17720a866", + "size": 15157 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/lock.json": { + "sha256": "314e86400e0eb19f6ef045fc1438a766bff0aabdea30c435b5cf8266a70a17a9", + "size": 3573 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/bootstrap.json": { + "sha256": "d7e083e2401f1642b45c8d21d77b5743557e4bddcc254416f8adc88cf296b5a7", + "size": 19520 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/pi.txt": { + "sha256": "e2f6138c06366bd9c0450f547a29ad0ff67968ec62e1286b1bc2db3d6a1355e9", + "size": 142617 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/pi/sessions/2026-10-29T15-41-52-540Z_01a18853-241b-7341-a428-8181ac56c56b.jsonl": { + "sha256": "472c34a4cdce5698b8f1f43bf1dcbc00cab21c80eee75c5807e4240cd0193ff0", + "size": 37819 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/config.json": { + "sha256": "84e0aeeb76d0aed4012a19a3a1546c982b9efa7ed3d1bf22a7444325db6ea151", + "size": 1840 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/lock.json": { + "sha256": "2a24c3f74515530a494ed95138709c1826d679a5f3fdee1183d8171625f46f7c", + "size": 2411 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/result.json": { + "sha256": "f32cb0bb7c0e95039eed52d794d2aa38d0d1ef110c30f48b71a8d4254ead7a7c", + "size": 33796 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/trial.log": { + "sha256": "f1253016a8e07cba1f658996e5d5ac5262155c7f830d5a59a8bd47f17720a866", + "size": 15157 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/attempts/1/ctrf.json": { + "sha256": "af6ad3b9ead232e0578d4dd0026b1ac7f0c53313c175c9dd0034d671f416b77b", + "size": 1465 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/attempts/1/test-stdout.txt": { + "sha256": "9f790b350e552aeb96f0780619d454ea9833de10137949ee439932ebdf6b05e4", + "size": 1644 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/ctrf.json": { + "sha256": "af6ad3b9ead232e0578d4dd0026b1ac7f0c53313c175c9dd0034d671f416b77b", + "size": 1465 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/retry-summary.json": { + "sha256": "88115986fe6bed6908e1ac51079821b9c4d1a99950f688451e3a193dc70a9d8a", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/test-stdout.txt": { + "sha256": "9f790b350e552aeb96f0780619d454ea9833de10137949ee439932ebdf6b05e4", + "size": 1644 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/result.json": { + "sha256": "07c387bab6efca52a6899bfa58e786a9fd8bf6174e982987296f14efc5a5a2eb", + "size": 1145 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/config.json": { + "sha256": "3cb40ad05346d2536b418993ab06a61e3c9dbc5edc05dea08648aba29223d7fb", + "size": 3395 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/harbor.log": { + "sha256": "8624cc27ab538a9801eede267822a6e0743b1c00316b9fa9a0d6ebf0a44d08f9", + "size": 1369 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/manifest.json": { + "sha256": "59669e92c13d4ebcab091ef371b9727603f67805b15a626db2f92dbe721eaa4b", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "5c06a884805494e11c9748243b21a1c84e83b2dcedacf1290096ef36b7d1c9c7", + "size": 571 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "14c92ec85273fc5f7069ae655925c0e0b8bc3b7b313d45739bb40cdc3582bef6", + "size": 18749 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "62064024e8037fc83640afae9103dc92dac1c95c24e71386b2e904724933437f", + "size": 11217 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "1730ba99566d1d52b910a310578cd2ea4ffd95b75ccd702fa96a9a61dbe13726", + "size": 16644 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "3bf9a32957c518afde8591ff80b04447b5f347c9e5e4c18acc5be59294bc7eea", + "size": 13945 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/summary.json": { + "sha256": "aa817fbed1d0700bdbe656d13f1fbd7c023f5fc48cad9254535e9e52fa86b16f", + "size": 17903 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/config.json": { + "sha256": "1bfe5359da52724949293bb3a15826478d0d7ad092ce87640913a3129479e378", + "size": 1859 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/job.log": { + "sha256": "dfa49104f5a851c09bbf98ed1cf1ebf43a7e63321ab56f4c425e2a170799bbd9", + "size": 15175 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/lock.json": { + "sha256": "0c28d6f730bf7aaf8df026fc1d99276111e2b7c48e62e4f8235e1c06e2dca394", + "size": 3567 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/bootstrap.json": { + "sha256": "c6c78961caca216b0c634b0e9a9915fbe60ad34fd6bcb09af1d0267dcaa79390", + "size": 15351 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/pi.txt": { + "sha256": "47f51ffa0024da64287ee079529a42740a661d11b42a07eb8c8ac80435b01c22", + "size": 125310 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/pi/sessions/2026-10-29T15-46-38-250Z_01a18857-802a-73d0-8e8b-fd583560c122.jsonl": { + "sha256": "be7ba096fa35ed31073dfc11b285a5b01dffcbd9f0a4bcc1547730a78ce81542", + "size": 31409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/config.json": { + "sha256": "05750acbd26bf9c3f645e9c705f579abf03d833c00059bba5bfbe8a93546ea96", + "size": 1828 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/lock.json": { + "sha256": "64e1e8f5c3118a63bb0ff5ceb1fdb1764220e05f86363eaf4b74bcdb64f2c038", + "size": 2405 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/result.json": { + "sha256": "5a414d38d37c9abe44c0ad6f71beb0b2fd3d7201c3b5c8c9e35e9735b135c401", + "size": 27814 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/trial.log": { + "sha256": "dfa49104f5a851c09bbf98ed1cf1ebf43a7e63321ab56f4c425e2a170799bbd9", + "size": 15175 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/attempts/1/ctrf.json": { + "sha256": "2d01f8ea22438666ac631b93aaa56bbc1ddc40b4dc7db2eaaa39b1e56243cab7", + "size": 3181 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/attempts/1/test-stdout.txt": { + "sha256": "bc01f8dfdac27e149bf848c8242b2cd96d02f4d434471e5439a2617758b866cd", + "size": 2431 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/ctrf.json": { + "sha256": "2d01f8ea22438666ac631b93aaa56bbc1ddc40b4dc7db2eaaa39b1e56243cab7", + "size": 3181 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/retry-summary.json": { + "sha256": "9a00e7ad31ef5bcd4a73b63df10137508f65ded14a59225950920c52069296bf", + "size": 232 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/test-stdout.txt": { + "sha256": "bc01f8dfdac27e149bf848c8242b2cd96d02f4d434471e5439a2617758b866cd", + "size": 2431 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/result.json": { + "sha256": "62dc37fff9c1e776a1d0e2d15dfdbe3c12a4750bdc50b3aa919609d3d1d0fb6a", + "size": 1132 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/config.json": { + "sha256": "7ae9dba62e399285cec9a720cfdf7b5d4116d749851654cf96f8702d6312c82b", + "size": 3401 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/harbor.log": { + "sha256": "e6d0854cec812cf51ad69af9c2c697107e66171ebf2a85642f1d8dd0f3f556e2", + "size": 1373 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/manifest.json": { + "sha256": "f9cf5bebc092c639a186533abcbb401043e5ec30357efaaa6e4f74be69377316", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "f2e0cba1fbf2ced3cdb4b327bc82411dc1c975197b5b7eea1402b642605ae922", + "size": 573 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "5b97f1040214d5c8ac25051fd64bd58fff5cb33f59d06ad488b8e3a5d5820180", + "size": 18751 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "8c8d0f1d40292e6724d574f51f088f8c8f59b255224b5cff6c2cb5df414103dd", + "size": 11219 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "8baa0a26cadd019e3699a0f2884733172c9b3fbbca2405262ee60086d6861aa7", + "size": 16646 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "1395aafcb13aee17e5dabeacf06e82c56731c2f4e03128454fd17d01e1dbd2f8", + "size": 13947 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/summary.json": { + "sha256": "2c8e9d02d77534674b23b479945c1741423651c6fddd636d020f62258355f75b", + "size": 24728 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/config.json": { + "sha256": "d4d13bedd8e98774577a0d27aae83d94526e2973051439a6ed70d2b8f4b0074d", + "size": 1865 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/job.log": { + "sha256": "85d527cb53b188fab72dcdac70a9ac61f8d1dcb7195ad1ffc82be64059d06893", + "size": 14674 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/lock.json": { + "sha256": "9cce1218352d5d53f443abf3b11e558aac0639599449d4f11ae1ee43b71f9357", + "size": 3571 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/pi.txt": { + "sha256": "79f5b4131e516f91a1d6b61cdf0f2095f4f731a3bb6a14edea8a82b0f51f9f37", + "size": 86878 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/pi/sessions/2026-10-29T15-46-38-953Z_01a18857-82e8-7770-9670-f1ada226b377.jsonl": { + "sha256": "88b698be595fdee00b34d6b10c92868fd6b81f7af25dc76bf6b7d5368b5f1b87", + "size": 22686 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/config.json": { + "sha256": "6069aa5bce7074f75fba8fe5b75731891d079e64757f108ceef7e3c93e3bfd53", + "size": 1836 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/lock.json": { + "sha256": "5691524b59ea5115d0864dce8a95b172889dc2425da3ec9e3121c9e533a7a5c3", + "size": 2409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/result.json": { + "sha256": "21266b563bcbefba84a21e003522a7cccf77cfa1c4620afd7da601a6cf13fa3d", + "size": 36454 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/trial.log": { + "sha256": "85d527cb53b188fab72dcdac70a9ac61f8d1dcb7195ad1ffc82be64059d06893", + "size": 14674 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/attempts/1/ctrf.json": { + "sha256": "6023ea355d07ca15f9554b057d1fd46b9de5a7a8509a600ab29d838c1f259b1b", + "size": 2490 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/attempts/1/test-stdout.txt": { + "sha256": "85d46a4db4417a4f11310d4e0013d82592a515ed996ddec67d02478dc6848573", + "size": 1660 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/ctrf.json": { + "sha256": "6023ea355d07ca15f9554b057d1fd46b9de5a7a8509a600ab29d838c1f259b1b", + "size": 2490 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/retry-summary.json": { + "sha256": "aa1c3309561fff6ca4baf955ed1cd9495e3d10376e1972dd04ac1bd8a19e6f6f", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/test-stdout.txt": { + "sha256": "85d46a4db4417a4f11310d4e0013d82592a515ed996ddec67d02478dc6848573", + "size": 1660 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/result.json": { + "sha256": "a6a22138c357689894f7f326ea97608ac3b4d5f6d5d912aa1d4a5809e433f926", + "size": 1135 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/config.json": { + "sha256": "64764c35b73f57b89e27e49286861defaadb084059eb5f90fd7b77168d964b7c", + "size": 3368 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/harbor.log": { + "sha256": "18019609e8f18e03daba9d6f30962f3c3338696e32c53dd718f1bbe2202d8626", + "size": 1350 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/manifest.json": { + "sha256": "f9e1c5c83d4ab2753389dc88b58249597970df0bea7cfa914ea26a869130b47f", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "20a302f2fdd74a30bd84f06db193e890c67820e89e08f5c57b98e3ceb0d607c7", + "size": 562 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "b98dcf08bbb67e4c0519159d98cc4395e254ad68871de1e0d539e1b2a4f99da8", + "size": 18740 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "78f4fc25b571d43ae0e42cd86d07f6bdaeab69c7c9645b57025b49fdf7604dd2", + "size": 11208 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "2d04254e591531d1827a45ef51a9b53d89b93d443720b2bcda0f7901e862bc7b", + "size": 16635 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "f5702994cb872f3300f51c6eeab42061aeb2fc49e5eaa63622685b241094ea84", + "size": 13936 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/summary.json": { + "sha256": "4881a6aaa459310e678289e3664e5ef021ca35fc6f585c163d132c0cca57b75f", + "size": 10647 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/config.json": { + "sha256": "f2724155b6d9ed178de6b9bc582b0592d384507bd36e328e3a5bfa0454d21c94", + "size": 1832 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/job.log": { + "sha256": "5db6ed78b72cd52adf8263c1c63745c8fd761907718f0dbc452b787c837067a3", + "size": 14085 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/lock.json": { + "sha256": "9d4c977a84fc87b33498526515795974ff76c75b34eb925a222dc66b2f6543ff", + "size": 3549 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/bootstrap.json": { + "sha256": "76aa181eed6c0146913c6ede77fd5bc047864f9c5c5c3f17fb9718cda54b43eb", + "size": 8908 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/pi.txt": { + "sha256": "16d56fdcf52bebc2c87e966dbb178ac66d61c51ff9944108f105c19d2cd1df01", + "size": 1138709 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/pi/sessions/2026-10-29T17-21-04-414Z_01a188ad-f59e-71f6-bae0-5266165a98e7.jsonl": { + "sha256": "b023fe8cf2eabec06ba81a31fd095a2a01b2fab5455eb427a9e060b5b8d94486", + "size": 194026 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/config.json": { + "sha256": "e1e5429b2a04d6a6b27f22c8afbe7322d40f6b51606d2a254bfe4ba16ca24cca", + "size": 1792 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/lock.json": { + "sha256": "e0e6ab636bf5af913c526820236571199d295488005e5cf7e8ecbccc83e94d53", + "size": 2387 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/result.json": { + "sha256": "2148544e350d5478ba70ac6a7e5e1b1e0fb2f17c058bc50550da9216d4dd889f", + "size": 18483 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/trial.log": { + "sha256": "5db6ed78b72cd52adf8263c1c63745c8fd761907718f0dbc452b787c837067a3", + "size": 14085 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/attempts/1/ctrf.json": { + "sha256": "9be4645e7fc33fc2e316fd043967426c38b822306b5218dc55b378d7ce6c8ef9", + "size": 2123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/attempts/1/test-stdout.txt": { + "sha256": "b13f927b31a51ad71b1d2080a2ce2a19aed72df32531a67e077091e69ad4f215", + "size": 1870 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/ctrf.json": { + "sha256": "9be4645e7fc33fc2e316fd043967426c38b822306b5218dc55b378d7ce6c8ef9", + "size": 2123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/retry-summary.json": { + "sha256": "7b09a9f2803c09d83ba55d8336b4ad54f51371be3a16038618b8d138d761823a", + "size": 235 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/test-stdout.txt": { + "sha256": "b13f927b31a51ad71b1d2080a2ce2a19aed72df32531a67e077091e69ad4f215", + "size": 1870 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/result.json": { + "sha256": "ec918d75ffb6b73c1a83e05ac08b66cd20fb1e8485e7966df43ec3b525f2969e", + "size": 1127 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/config.json": { + "sha256": "2cfa8347634c6d4c6b00fbfd664f4fdd56fa169c1536d23335689f55e3cbaa9e", + "size": 3365 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/harbor.log": { + "sha256": "14ba6b6cbd472c57e910d94f601d9ab84fe81394b38ee22b477c0608b9cb3906", + "size": 1348 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/manifest.json": { + "sha256": "28bc2737fa8e588bbba696d401a5a4ccd08a1b121a2498a437ace3a2c5007cf8", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "b6304324d2f10041896f30161abc9f4ba24d306bfe6419a8b190f9d424bf9740", + "size": 561 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "0e05c7b2e499627b6c1a28492042c79a6cb02b89f734c5bda9ae8947afe3a5b9", + "size": 18739 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "9df5075f150ae8d16fe05f3a6a850b8eefc18e88e211be28a4b4ef21198f0fdc", + "size": 11207 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "e32e0e03c269f0ee0e15759db36687acfcab7e6f380a8b9d3ad226423403efc5", + "size": 16634 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "0ac107f05df84c6be98cf2d695f2fcb2e50f5168ba809eb8f24de6eb8c0f2949", + "size": 13935 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/summary.json": { + "sha256": "06d847066d289096dabf8246bab35281dc3bca12d79af925bb6a441d87da7c77", + "size": 17885 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/config.json": { + "sha256": "aca6d8e2c190df6841ae2b58108bac7ec3d733844a524adf77adb66f24150f71", + "size": 1829 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/job.log": { + "sha256": "f1a90145002f35e72248e07f56277b514378c9436c9eff216e52bcb820f08b8d", + "size": 14092 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/lock.json": { + "sha256": "ebfce60981acc8c80f98caa6cfd0f56ddf9620a6e9f2c828d5ed7b392dcc8ebb", + "size": 3547 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/bootstrap.json": { + "sha256": "c6c78961caca216b0c634b0e9a9915fbe60ad34fd6bcb09af1d0267dcaa79390", + "size": 15351 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/pi.txt": { + "sha256": "4479de66aad2c48fb21f66c34b1400e43600b8107c28abf596baab116c9b0dd5", + "size": 208177 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/pi/sessions/2026-10-29T15-48-48-258Z_01a18859-7c02-7596-8d0c-4a8b03b5e3bb.jsonl": { + "sha256": "b21bfc78b399881ff38099f061008eb25d772efcd18ba66859d78674f5f1f83c", + "size": 42934 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/config.json": { + "sha256": "5c6305480e442c30756671dee1f77f80388b49449ed69d4c0e2a18bfb03817d3", + "size": 1788 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/lock.json": { + "sha256": "9bffc595878bd38735399418c5362a87b5536f0a2027044525f85ac3dec365bd", + "size": 2385 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/result.json": { + "sha256": "f30b060cdeb4981cfdda2c6cfe949f9ecdbe54c50695765328d915cf2355187c", + "size": 27725 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/trial.log": { + "sha256": "f1a90145002f35e72248e07f56277b514378c9436c9eff216e52bcb820f08b8d", + "size": 14092 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/attempts/1/ctrf.json": { + "sha256": "eaa9f7177dc6395fd5cb04f741ac3359409d298fa5ec68b457bb035b137e76b0", + "size": 736 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/attempts/1/test-stdout.txt": { + "sha256": "82a48bfca4a38f372614824d1700815033059e51ec667c7bacebc42055fa16e8", + "size": 2323 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/ctrf.json": { + "sha256": "eaa9f7177dc6395fd5cb04f741ac3359409d298fa5ec68b457bb035b137e76b0", + "size": 736 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/retry-summary.json": { + "sha256": "94dd2600f29cbd6354381d18c87889bcdb141825157f9f68e9e033c08acc2b27", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/test-stdout.txt": { + "sha256": "82a48bfca4a38f372614824d1700815033059e51ec667c7bacebc42055fa16e8", + "size": 2323 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/result.json": { + "sha256": "4017c5cabd70bc4b544c104dd405bff5b4f7ceac826c0511dd84cd122da5695c", + "size": 1122 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/config.json": { + "sha256": "fe47808ada65deac4cf331645fc03f15f78e6f17f75b2ca3890685774660ff56", + "size": 3383 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/harbor.log": { + "sha256": "8c3e0348ff6168e0f29aeeb11e2a3d6c2fdbea137c9fed29848733f3ad9b7e2f", + "size": 1362 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/manifest.json": { + "sha256": "f6c80d3321729328c1e4c0e676878561129056e0397aebab5d55828343cf6820", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "5077e4b4e380877b7c6cb93443a0540ea747d63bdc983d24f6a51df7063507b8", + "size": 567 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "b94240a7d41b3c7dcd15fb1e40cb0bee05747d74f945e2cd72b3002ec0f9054f", + "size": 18745 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "ffac8235420ea10642ee7df8615ec50f71075ff7e7eb1e37fec2fa0f5e48586f", + "size": 11213 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "71df1624d8583ef6abee83fbc6a1c7ee3ce211304603a134d6ee432b6f84bdf7", + "size": 16640 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "135549a3985cbdd44c2635ebbc286b6a69feb17e1f9041b2caf75ea3c132243d", + "size": 13941 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/summary.json": { + "sha256": "90ded12397c034683691435a9db1f5b6c0e4c62e4f41c8e51237a9f5c6a3b05b", + "size": 24715 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/config.json": { + "sha256": "de4033a4295a72cab9445292c6a77c269e091a86eae96d963ddf652ab1a24ae4", + "size": 1847 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/job.log": { + "sha256": "62bfa293ad9bf46fd83b8c07539857480da6618cad9565091700595d02022d40", + "size": 13777 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/lock.json": { + "sha256": "4aa7a77cdea3d1ff7ad413e139bc8a36dc8487783234611e7e2a446c2a415a1a", + "size": 3559 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/pi.txt": { + "sha256": "715d2fd6436c0e9714c91c7ec7ac94621a4867d5dead0f327408576d17c18c56", + "size": 771149 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/pi/sessions/2026-10-29T15-48-46-549Z_01a18859-7554-7672-a57c-3b6d455f2aff.jsonl": { + "sha256": "5fba5b34bdd1ae1a9508b5d556127db33b21d5d2de1e39bede6bdf48432e4bb1", + "size": 163017 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/config.json": { + "sha256": "fcd063a9b9fb1c8f7c7ea1d4b1dcd568835715bcf4c386da8a7cf2900a07cd2d", + "size": 1812 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/lock.json": { + "sha256": "1062b81414aed3d381d23ff109c132abc86174ca255827beff8505c97d10e2ca", + "size": 2397 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/result.json": { + "sha256": "cf94167dacab126649a35a465c73ca26a178644c50bdaa178bc080bbb503f923", + "size": 36404 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/trial.log": { + "sha256": "62bfa293ad9bf46fd83b8c07539857480da6618cad9565091700595d02022d40", + "size": 13777 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/attempts/1/ctrf.json": { + "sha256": "cdbfd6c5507922b88c999fe22129c1b9e9298222fc866820e3f22f1965a5963c", + "size": 5268 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/attempts/1/test-stdout.txt": { + "sha256": "00c8cc6ef71fecddec8a437d1538cf0c86105cabdb6c2b1ebba51cc344af0b5c", + "size": 84288 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/ctrf.json": { + "sha256": "cdbfd6c5507922b88c999fe22129c1b9e9298222fc866820e3f22f1965a5963c", + "size": 5268 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/retry-summary.json": { + "sha256": "2e7f7867a1f5270be8b276945139c8596770f463ff53dd5a28e92e0404bfaa58", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/test-stdout.txt": { + "sha256": "00c8cc6ef71fecddec8a437d1538cf0c86105cabdb6c2b1ebba51cc344af0b5c", + "size": 84288 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/result.json": { + "sha256": "8191698f67ab1e47d45d345f6ed8f9f0f39a8465292a9e7b75a6e917a0bc8a2d", + "size": 1132 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/config.json": { + "sha256": "280efb784646e98caa9f841e50f4b0a8d0db2659ca24c0aa019cf79aeca6c4e7", + "size": 3398 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/harbor.log": { + "sha256": "c3be5a30b83e87328adc649c29fc337d7d57332e661a55ddbd9dade117accb59", + "size": 1372 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/manifest.json": { + "sha256": "50fc2d5e30df0c51581bb2434212d44413f5bc77337a4012e67f04ca6269a57b", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "33acf9a371128efdc900d6a34e3da204aac0b8eb6ebe6aafe7f19e4262cb1933", + "size": 572 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "29dd9ea3b24f5f6f819b5bd3913cfee2aefc464f657b9fdd6c75a8dc84196981", + "size": 18750 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "ee60ded50ceee51d531f04e5ef2c6da1c9f2cc1ad8fe71a9aae5a9fb087ac865", + "size": 11218 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "5081b94c5ab11bd26e8f0a829ccb0a0a605b7113f138917d51e67a0f0eb29ba0", + "size": 16645 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "9792b471f767c4b5ab13555dd226ff0d3c51fe86775df00d4da6d6f78504aa32", + "size": 13946 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/summary.json": { + "sha256": "f0166477804a0eaad5d93e8b986cf8e085154584687ea41ad45eccb85c28c7f2", + "size": 24724 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/config.json": { + "sha256": "dc262716a1a2e9742b15a65570190d88152fa1d8aaf2c62e98cd8d5b6aabdeaf", + "size": 1862 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/job.log": { + "sha256": "0bff543ca8a72d136c9c15cc85e67f2386ad280c0bf50238d25edd523e186bf7", + "size": 14624 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/lock.json": { + "sha256": "cce457f0a1c164eda873d2c3d7660da6d5a58b7fcfc31aa45cf42df49ca00e97", + "size": 3569 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/pi.txt": { + "sha256": "b575c5d73a80df3783ea232e0b6b6b37411d0eb8b25794bc61b3878ce77953d8", + "size": 612625 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/pi/sessions/2026-10-29T16-13-36-534Z_01a18870-3196-71ab-9d9a-9e62c3827e58.jsonl": { + "sha256": "19539329eea4e0583588826909634824b3aa409ee03d0d9de4305501704d52a5", + "size": 132503 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/config.json": { + "sha256": "3a45d347f8752067b426e84cac78e8cc3bece2d0146da04baa588b46af2b7bb8", + "size": 1832 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/lock.json": { + "sha256": "05801df997595a17df977e4072cb86afa0261e9d69cfd6ed6cf574c0f563f9d1", + "size": 2407 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/result.json": { + "sha256": "34f7528a3ceb49123694b5c7e75299aa9eb5ffd6b14fc5050864da683b859590", + "size": 36449 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/trial.log": { + "sha256": "0bff543ca8a72d136c9c15cc85e67f2386ad280c0bf50238d25edd523e186bf7", + "size": 14624 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/attempts/1/ctrf.json": { + "sha256": "84250a2a1054785740bc2c61a3469b34bfd1067abffb9ce8d0e4937955bb5f26", + "size": 2138 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/attempts/1/test-stdout.txt": { + "sha256": "24b0e08ac5bea577604b779ea0c8cfb553878e02369fdd8452b3bdf3015f7a30", + "size": 3674 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/ctrf.json": { + "sha256": "84250a2a1054785740bc2c61a3469b34bfd1067abffb9ce8d0e4937955bb5f26", + "size": 2138 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/retry-summary.json": { + "sha256": "50321ba1806cffee7e2ecf9efe3c1a096fb48c6b0e35ea337ea98c5b7e331890", + "size": 232 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/test-stdout.txt": { + "sha256": "24b0e08ac5bea577604b779ea0c8cfb553878e02369fdd8452b3bdf3015f7a30", + "size": 3674 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/result.json": { + "sha256": "573dff0cb5f9c85dcd610ebbabb9c1bf5a98057a569775f6b4533cdb530f9f6b", + "size": 1137 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/config.json": { + "sha256": "a956473611b983bace3ea41d2ce8e932d13378c2646d27ca83ae08fe6836c9f0", + "size": 3377 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/harbor.log": { + "sha256": "036c48c65e81bec81e95e694100361242baee61b9ddbdce3361419a6627de880", + "size": 1357 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/manifest.json": { + "sha256": "f72aaaba509ee887b0cacd7407d9d2504a45f0de57f1d929b4ae456ac49f30a3", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "45cfd4bb17f84a7ad7981791a4556795cae67aae1e5da8f87a227920ec9752d6", + "size": 565 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "f241a613158f86b68583e10e32bba386469ef361dabd47a5ff08464df40ac7ef", + "size": 18743 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "b7153ae7e8a353491e165eec7a07e273455f1e1e69e6ddfa3e4405ffb812e11e", + "size": 11211 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "5af68dd77e01b6acac4279bf0fefa03abc178d6ad8193d672e8ddd4071a71749", + "size": 16638 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "cc56e1dda657d013ab965cab0bc62cbc94c9a76b7e9726f2c3900650e92c63de", + "size": 13939 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/summary.json": { + "sha256": "d88d448879dc0e443787578d0279155e4bec365005c5f8b15228f6f9222f677e", + "size": 10072 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/config.json": { + "sha256": "019f1f01441b1cd58ed5e19a5e835617fd09e9b274c199d1e5305f40c328d71e", + "size": 1841 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/job.log": { + "sha256": "de22f554b1d9a321ad7e2cda94a043ed8ce3aeced434984491a57960b4415d31", + "size": 14204 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/lock.json": { + "sha256": "5def036248c32573a766350b72f92a9f02aa620816c32d5d4cda0fb0718609bf", + "size": 3555 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/bootstrap.json": { + "sha256": "a5579cb66ee1b24253e9fe8d5a86c9496dfd80e628f2564dbce523e3e8868943", + "size": 8346 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/pi.txt": { + "sha256": "ba9ec194f552f7387b69498884733117b56789c23876a95a107da9157379c2ed", + "size": 6992377 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/pi/sessions/2026-10-29T16-13-46-205Z_01a18870-575d-7467-aa5d-c0a2afacaecb.jsonl": { + "sha256": "7df20deca40e0200d23b4439f584e9cd7c8413d49d979c49b6aa35eb7e950df4", + "size": 406119 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/config.json": { + "sha256": "53b675215144ad8af1a780b02c7f0c6ee2c3778f0455896775d2060bea8cbca2", + "size": 1804 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/lock.json": { + "sha256": "305b8fdc75fd22d707d6efe2d7e19b725620cde934052960dbcd19ebcb137e8d", + "size": 2393 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/result.json": { + "sha256": "724a8d49fba4f100da850a5d13b5f2498b02a71f1186ade7c5aadc3a94243e53", + "size": 17851 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/trial.log": { + "sha256": "de22f554b1d9a321ad7e2cda94a043ed8ce3aeced434984491a57960b4415d31", + "size": 14204 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/attempts/1/ctrf.json": { + "sha256": "c703698f083cbeaf74daa5858db2a929aedad23433779c53e7cebc77b19d844c", + "size": 741 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/attempts/1/test-stdout.txt": { + "sha256": "3668beb7a59e1b205f68dffd189b1ef73505a63045da28e40af3fcbde3baacf3", + "size": 1173 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/ctrf.json": { + "sha256": "c703698f083cbeaf74daa5858db2a929aedad23433779c53e7cebc77b19d844c", + "size": 741 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/retry-summary.json": { + "sha256": "c2a8ba6927c83daa1bbc33c260a06ce7c46dbdb719ddfb91c6dd4405631eacbe", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/test-stdout.txt": { + "sha256": "3668beb7a59e1b205f68dffd189b1ef73505a63045da28e40af3fcbde3baacf3", + "size": 1173 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/result.json": { + "sha256": "5cfc6be384843f6e5cc5fa3dd1f9826e4a7b150984c27cf1119cb961fba70a47", + "size": 1131 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/config.json": { + "sha256": "d29a531bc797c7336b15809edfb2932f002856a2d54aa3dbc2f624bad1ac4d89", + "size": 3374 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/harbor.log": { + "sha256": "084197e75478089bddb2e34e17e7ad6ba5b5c95124f7c6d79efea3daf1b84c09", + "size": 1355 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/manifest.json": { + "sha256": "924695f00b049c287263083780d90cfb1dd3d4166eabe10e08e2247d3589dcf1", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "59b7028ee1f7b466952bed2524fad190e14f2aa2348f235e0ef54c7c63607fa9", + "size": 564 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "fb5fbae6e9a6c1415109fa78ba31d897cf0c8f9f7e4f3990f3e9278b5ffdacea", + "size": 18742 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "329e2aa0f973c1a070d00fa00a76596d25ce87dcc4b4cd35cc02025a2e116ed4", + "size": 11210 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "0b047870d0f652abd2acdfd4b3c8753da3d4e0a47e6531f7682a0aaf1ba53c6a", + "size": 16637 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "47b7009f86fcd8f3df9dee6cbed315be82e951d1246f9865458c94cdc276d63d", + "size": 13938 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/summary.json": { + "sha256": "7554dd518914aea9285d9a1755914ee4bf0335572965aaf3c95d88d64be7000b", + "size": 17420 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/config.json": { + "sha256": "3cb6ee626eff128242b01c337e331124ee133f799f33d5b0189d15e6ed51bbd7", + "size": 1838 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/job.log": { + "sha256": "c813249012945b8b101d07c67811e8d2a9c252c91a5e7176da2de5ce9613d415", + "size": 13828 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/lock.json": { + "sha256": "3db88826742e3fa105eccde6128fcaaf9c02bd2455b6f830b4cfc6d778c8a197", + "size": 3553 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/bootstrap.json": { + "sha256": "66da97dfe7a11fa2d21bd5c1fb10d16dc785f336e50de787c2af8c9215e78cf2", + "size": 14964 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/pi.txt": { + "sha256": "93576a3fecc3b6dd625dc2570ca6da7435cf0076f378f17c8bd974ef79db54a1", + "size": 162062 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/pi/sessions/2026-10-29T16-31-03-672Z_01a18880-2bf8-7126-a5d3-65bdc97d446f.jsonl": { + "sha256": "a2a239c58c7ed83a8d175d5f2e83559ea86a3280487c2a0c030aef531c4eda03", + "size": 37433 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/config.json": { + "sha256": "8e7bba7027fcbe924aefacd045309f59b1666e6798b909f44c35f56cb70df611", + "size": 1800 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/lock.json": { + "sha256": "ae270852620ef8fb9a7fe950c6d123bc54b05c25ed6198c55ea018e9cdeafb1c", + "size": 2391 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/result.json": { + "sha256": "a27cf55bc68ca117c24e03562facbf12a81d3695cdfaeb59bcbca8059aab29d2", + "size": 27069 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/trial.log": { + "sha256": "c813249012945b8b101d07c67811e8d2a9c252c91a5e7176da2de5ce9613d415", + "size": 13828 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/attempts/1/ctrf.json": { + "sha256": "7bbdd7dc3aa91336efdbef204c43358c0a61d1b1d064ac61e9fc3074dc3d2857", + "size": 5365 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/attempts/1/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/attempts/1/test-stdout.txt": { + "sha256": "b8caed31f3361102326d883e591ff625ee0f2ce1ae3835eef059c46eed818d06", + "size": 5217 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/ctrf.json": { + "sha256": "7bbdd7dc3aa91336efdbef204c43358c0a61d1b1d064ac61e9fc3074dc3d2857", + "size": 5365 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/retry-summary.json": { + "sha256": "edf26eac53ea6fcf5e34e6c67a3b9f173aa4c227d8532eb4de9edeacc6720056", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/reward.txt": { + "sha256": "9a271f2a916b0b6ee6cecb2426f0b3206ef074578be55d9bc94f6f3fe3ab86aa", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/test-stdout.txt": { + "sha256": "b8caed31f3361102326d883e591ff625ee0f2ce1ae3835eef059c46eed818d06", + "size": 5217 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/result.json": { + "sha256": "1cbb0dd0c1ba8402bf35bf246033af94925366aa6a2d47ebb92cf5bd464acad6", + "size": 1126 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/config.json": { + "sha256": "175bc8df37e279761a8a5cd4aad8f366a2e9517d1f5cd3ac4602836608547666", + "size": 3365 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/harbor.log": { + "sha256": "c1b1c0d688f225c3200888f0b680fa84f62d876cbfea1a478179c0defc047d58", + "size": 1349 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/manifest.json": { + "sha256": "c5a0266fd97b4eb085276f8a94db0526edf5d9bf1a8c0ce5d45d81299921dc27", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "16c3be7f78f8eb70628ee80d8f74e4249db5396fa1133b7a4f441112482d933b", + "size": 561 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "092f86d5774ceb2f965bb820e2c44e5c18a5f0d696aac58e280acd2900092f6e", + "size": 18739 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "b8c8ffb4edd31237e93d421340f2603111e4ba56e0283445d1abe057b9890ca0", + "size": 11207 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "f5df4c6064fec33422016a51755054005f4c063e7e1f915f89ee6f593dade3e2", + "size": 16634 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "42cc8dcb0c611a144512412d91bedf026e26e5d20504b76abc75edeff8da6557", + "size": 13935 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/summary.json": { + "sha256": "f6018918da491b4e7207b549b1708828390f88d4bdb4bd505a7aaf8fe09b8408", + "size": 24705 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/config.json": { + "sha256": "d96dcea21d2a8644a31a01cffd2bd162a5a6c0176551b1071dfa3727c8b4c7cd", + "size": 1829 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/job.log": { + "sha256": "0460d2c1b890400f7128880f1892c179978d1fca56f4601a738ce9ebf7240e65", + "size": 14877 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/lock.json": { + "sha256": "41a5641f59c4bfc865a1e7cba75ee328f633ea31b07130fe54e8765e2cbf4434", + "size": 3547 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/pi.txt": { + "sha256": "41d577a7678652235bc0417169e7d083a6557fb1d58f82ca727e818f8b1f7c11", + "size": 1397907 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/pi/sessions/2026-10-29T18-42-16-926Z_01a188f8-4ede-71ac-80a7-df665cd37dd7.jsonl": { + "sha256": "fe2d78b08988b98a03e2ff1aec9c2399376479c022a9a8214b8096b0c37cfa3b", + "size": 347189 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/config.json": { + "sha256": "2c8d9d99918ba9e86cbf0b92a7b77a9d7e40984cf6e92458214643aabbff4789", + "size": 1788 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/lock.json": { + "sha256": "aaa34c91896a846fb1d6618fd40f329bc0618188bea22a7cc5315f7e772df50b", + "size": 2385 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/result.json": { + "sha256": "8ba32037813aa41478afc3d95bd009ec6dd1ede28cd687503269122575ae1dfd", + "size": 36352 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/trial.log": { + "sha256": "0460d2c1b890400f7128880f1892c179978d1fca56f4601a738ce9ebf7240e65", + "size": 14877 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/attempts/1/ctrf.json": { + "sha256": "99671ff20237bf7d5c5e66728b90857972f22e10cb644e99b092e01f97d7ebf7", + "size": 1772 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/attempts/1/test-stdout.txt": { + "sha256": "5258f81d8857f0b0413236d91def3712b91f8d921b866c5679d0390e344a6b33", + "size": 18271 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/ctrf.json": { + "sha256": "99671ff20237bf7d5c5e66728b90857972f22e10cb644e99b092e01f97d7ebf7", + "size": 1772 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/retry-summary.json": { + "sha256": "70c885ea3232bbf4cc55d881d6f26f4b1bfe37ebb828da774e9a46a0baa38cd2", + "size": 235 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/test-stdout.txt": { + "sha256": "5258f81d8857f0b0413236d91def3712b91f8d921b866c5679d0390e344a6b33", + "size": 18271 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/result.json": { + "sha256": "4dbdce1a7fe664ea011c17aac750deb8688ba030da4077f20d7ca6c28425a130", + "size": 1128 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/config.json": { + "sha256": "78be6b280eeaad2f6a3b4c942696eee1b57075d6181fa8eae3741fdc87fd594c", + "size": 3359 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/harbor.log": { + "sha256": "bed154e522afa80b0c7f34dc1e563a7c1608c27e374dfcf4933acd7bd45be260", + "size": 1344 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/manifest.json": { + "sha256": "e479937407626f9a26ae554d2722705d009edf1b24151bbf228526589309b4f4", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "09597fe15ce220ad7e09cefccc5ac3fc7db64664c32e6a537050a2fa7f299653", + "size": 559 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "a81b19bbe6dbc92c338c1bbf6c62ce559e856bd35dcd0d291de440d533a015de", + "size": 18737 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "154c0ee9b37a02d080eb3ddac814f989f54c21f46606f168de411f5271ecacc8", + "size": 11205 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "5ee5b0ae5f4b14427e1d139f0754064ff59aedadac8835455a393226c78d9e27", + "size": 16632 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "542a0c8f079e774e0edf133c1ddab72f125c01ffe279e46039d1fbf9328c3411", + "size": 13933 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/summary.json": { + "sha256": "ac4c5cb5088c18427b2c0323fc49c00b6db9c3a72aae353bd061b3e280ff1032", + "size": 26506 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/config.json": { + "sha256": "b1568879dbc24e361ae15badeaff8538f600c32b706f08aea97c66d790aaeea0", + "size": 1823 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/job.log": { + "sha256": "50bc5aaf90e0f4646e887833dfb528b18d09a2bce64fb4e7d5ddf75719e22624", + "size": 14282 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/lock.json": { + "sha256": "f084de81affe1ca8f7134fac082b6cbedd500930a4a4c135673820d7452962a9", + "size": 3543 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/pi.txt": { + "sha256": "1b7cc4b4ba2ad4bc78474cae505cdc0020eb6bf53bd11da594ac718a992c1ce6", + "size": 339214 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/pi/sessions/2026-10-29T16-31-09-590Z_01a18880-42dc-7087-b311-18c97d8e57cf.jsonl": { + "sha256": "aa2f97858bfd842bd547fac57a25f69d03c1180b6921e9b7f00df36f7255441e", + "size": 105314 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/config.json": { + "sha256": "be6f75b5585866218c4ef3f96e283d08882f8ba4ac93ec699652f250697184b4", + "size": 1780 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/lock.json": { + "sha256": "bab4442c13e07f9a5244e7f2a044827c7465214be38c6041d1c6c7325ff1de76", + "size": 2381 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/result.json": { + "sha256": "4adf367274178f551869763e8b0c8e2ccc0893d2d39874eedfc50d1b3120acdb", + "size": 38455 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/trial.log": { + "sha256": "50bc5aaf90e0f4646e887833dfb528b18d09a2bce64fb4e7d5ddf75719e22624", + "size": 14282 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/attempts/1/ctrf.json": { + "sha256": "06019f3100c51729080fd39a9e1e9e7137b7b92538b00c20ee498969d4abe174", + "size": 757 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/attempts/1/test-stdout.txt": { + "sha256": "d9a559b38aa032a33b34351b823962968a1437f318bc93b4c14a5e59c87a1e52", + "size": 1561 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/ctrf.json": { + "sha256": "06019f3100c51729080fd39a9e1e9e7137b7b92538b00c20ee498969d4abe174", + "size": 757 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/retry-summary.json": { + "sha256": "949e120f23fba02920eec7d1dcfb6b8d42fb3f113a6a0bacfcf0d236ac860501", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/test-stdout.txt": { + "sha256": "d9a559b38aa032a33b34351b823962968a1437f318bc93b4c14a5e59c87a1e52", + "size": 1561 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/result.json": { + "sha256": "bb6361a5ac33c54fe642bda7fdd00d74f0f32aca6766be95f281d1cb9d6a795a", + "size": 1124 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/config.json": { + "sha256": "340fe0b2c493bb10f391c50f544faeb9571526457c93e825ea479ab4329b9ddd", + "size": 3416 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/harbor.log": { + "sha256": "e2ecb81545effd67ab3388e23c8d0a8f28d7287fbfe28116bb72efc548a4a4da", + "size": 1383 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/manifest.json": { + "sha256": "1d3ba56b4dd728666985cbc4c24742c5e3123b9ba580df8fe57d53a23726f9b7", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "df54a876970b1edaea72ee122bdc4ec6bd5e56dfb6ea4427d04457f2cf5e7617", + "size": 578 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "8ce9455b2be78d52510f40245a70ec09379c9c418b2b1534ab9f8693e3f74c46", + "size": 18756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "c70fe8736c66e3a6e67903a1ea3946bdbe64c299411c5ab8a2779299f58aacf3", + "size": 11224 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "d330503b8c0eecbc7a124e452b5163dd900f09938853423be4a76e68c02172a7", + "size": 16651 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "f9b454382c2637bfd891c3d09485e018943a8bac710141e5a246a9b6f8b3a08d", + "size": 13952 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/summary.json": { + "sha256": "f6111c16966ab27f5a2bfeefaf78b997828a382566670d5b657180f0c3af8fe3", + "size": 24738 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/config.json": { + "sha256": "82da9ec5939940118cc92ee7e2e09592f0b2c94ce985bddab8a2365115826e3e", + "size": 1880 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/job.log": { + "sha256": "628eb34562f3be15a78630675dedbe2bdac4a4f082bc65cb63dc4ff0b32af9f3", + "size": 14468 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/lock.json": { + "sha256": "06e37b83368fc525f6c7fa44617a06a5ada62411210d0888c34129429910c247", + "size": 3581 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/result.json": { + "sha256": "b6cd0064430e2aa501bf4400bfcd168034371025a45eb10045fce39509a72ed1", + "size": 1153 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/bootstrap.json": { + "sha256": "497be2a5990b4fa5f90a27f32fda329638cb7cd6407467a523e6850c131154e9", + "size": 21456 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/pi.txt": { + "sha256": "f8d730262523fa98c2f9c08c5f533a38ffc5e0513dd438c682b42ea11514811f", + "size": 1364832 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/pi/sessions/2026-10-29T17-28-06-541Z_01a188b4-668d-74fd-b538-d231beddecc8.jsonl": { + "sha256": "18f9f7355ae4f7a0bd3d4fd955b563fa43b39a642adb0152dcf71797393ba042", + "size": 293751 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/config.json": { + "sha256": "f385b9b6fc6210145ff0422a480b3f3189b943846939e868ab3902f8804070ff", + "size": 1856 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/lock.json": { + "sha256": "324997b1ad2412455a5cdf52aa721eb68a459c4b229197977b046d188034a79e", + "size": 2419 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/result.json": { + "sha256": "2c5e4caffbc25203b5983b7509ff39a78009cee60b68409fb0055cc139bb5cb3", + "size": 36513 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/trial.log": { + "sha256": "628eb34562f3be15a78630675dedbe2bdac4a4f082bc65cb63dc4ff0b32af9f3", + "size": 14468 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/attempts/1/ctrf.json": { + "sha256": "c1b0309ed52796bd33667a6169e5fb00a0a447f17c2742af8fa9627c61aeb71c", + "size": 740 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/attempts/1/test-stdout.txt": { + "sha256": "d9db5d0a4415bfd14bb840016d2be0137b26ac8b9a6ee4b8f9d95a09f1dd9ed5", + "size": 9680 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/ctrf.json": { + "sha256": "c1b0309ed52796bd33667a6169e5fb00a0a447f17c2742af8fa9627c61aeb71c", + "size": 740 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/retry-summary.json": { + "sha256": "3c8fe969d29212f2db2dcc35671e4ff1e33a758df69d93590c234a5ac1635f7e", + "size": 234 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/test-stdout.txt": { + "sha256": "d9db5d0a4415bfd14bb840016d2be0137b26ac8b9a6ee4b8f9d95a09f1dd9ed5", + "size": 9680 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/config.json": { + "sha256": "092561620bc3ff8bfe2efcec77ed63a99962242248aff40b77f14cc2c13ac15c", + "size": 3410 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/harbor.log": { + "sha256": "289da9ffea9633c281545771d4c7bdf5bc768b92033d78b43913ec1d7942b3cc", + "size": 1379 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/manifest.json": { + "sha256": "37cfd2cf1978f9c3449f3c5f177b8b889994cf071e243a6bbb3e6de1e9fe8b91", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "b96bdca6f329211ec8532883c0344583adbef522040f70e1d15837a65cb90bbe", + "size": 576 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "98f6a8c9f86c8ff6fe4f4bf7823f8af883b134423b540c6bf6a370eb186ef646", + "size": 18754 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "30830020372dbe421f8c6e46c4f37ac0494a0b7fe78c45dc8d3e8811ba2d706e", + "size": 11222 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "cc69d760c0e627cf95f9484530ad5ddff07b6649639b2dcf3468ae4027a39cdb", + "size": 16649 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "b465864f3dffa012eac6002fa27dc475a17f8e235accfba65f9334c32935dc13", + "size": 13950 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/summary.json": { + "sha256": "d248bc91654d7685dc118adde7e420a538ab81141a5e78b5a5d577b9b6be8fcf", + "size": 26539 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/config.json": { + "sha256": "4a75db2cddccf1476e6c2a1c3e35f2ac8a08734091ac67e65d01056ba7cd449c", + "size": 1874 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/job.log": { + "sha256": "7a3b20244a33f3f6de07afdb2ba50f72c707a3542ea9b680ef4f4944ca0d5563", + "size": 15130 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/lock.json": { + "sha256": "6dce5adb28420c7621d1e7b8924e25b74bf98b0233433fc347c1dc3ab9627dde", + "size": 3577 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/result.json": { + "sha256": "22a13cbf2082797ead9f4a537bcc4636312fdf51e9532cb5c7da7e341d0ee9e7", + "size": 1143 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/pi.txt": { + "sha256": "b2bc41b656184603edbc2c894a86215b81fd6fdd5991d0f2f0065fbb7fdf9133", + "size": 1738810 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/pi/sessions/2026-10-29T16-43-19-342Z_01a1888b-65ae-711d-bad1-3cfeedf82e8e.jsonl": { + "sha256": "9a44997b6d8e984722fadf873cbb54f3b82dba2c4989c816f30f9dbdcf09d2fa", + "size": 328280 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/config.json": { + "sha256": "e5d2406b13e54485394fc789bf72b799bda851fa9f290fc68317cf08dc330373", + "size": 1848 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/lock.json": { + "sha256": "2fef410a7bd2a7b4cea02b7215e6724c084edfe9e46c8a4fd3f882174fdec24f", + "size": 2415 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/result.json": { + "sha256": "960ff885e63fa133462da85c9cf88fdbcbf3c79be3885527e3262d712f6c401f", + "size": 38611 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/trial.log": { + "sha256": "7a3b20244a33f3f6de07afdb2ba50f72c707a3542ea9b680ef4f4944ca0d5563", + "size": 15130 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/attempts/1/ctrf.json": { + "sha256": "e788cdb0222e50ed8f5928c824fc5dbbccd5e6f6dae8b9b5ccd336c706b83fb3", + "size": 1536 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/attempts/1/test-stdout.txt": { + "sha256": "1e2fd14dfdf81ec03547cabc064de5e29d0cd2cd8da991e1a4ff5283cefa5a59", + "size": 3074 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/ctrf.json": { + "sha256": "e788cdb0222e50ed8f5928c824fc5dbbccd5e6f6dae8b9b5ccd336c706b83fb3", + "size": 1536 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/retry-summary.json": { + "sha256": "def4d425355167b7ed6f681fbd9073123667afb8f085866c837d315240e75076", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/test-stdout.txt": { + "sha256": "1e2fd14dfdf81ec03547cabc064de5e29d0cd2cd8da991e1a4ff5283cefa5a59", + "size": 3074 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/config.json": { + "sha256": "b559520e2bb2c45ef1ecc262789d0e22942e7d8ad8efdf8877d85517e38bdbca", + "size": 3404 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/harbor.log": { + "sha256": "91d7d359d1bb321196d0b3b338d63101253efcb87903452997ea4be23314fdcd", + "size": 1375 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/manifest.json": { + "sha256": "8f7f32cac0cc83854951ac709404254254eb34b3d088406f3e437dc43c1e7927", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "59a76807b2a9d301a804f16699ca2ca3da843d0a92aab179236b0dfd3e8a348e", + "size": 574 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "83f1f6883b2d5d547445d52a710039e840901b7c56a21b67de1f9312f34802a7", + "size": 18752 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "818cd0c1a5b7bf89e8d7db77c0a7e4ed0d677c5f6187f9e20b860edb5d2a65e3", + "size": 11220 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "e1b815bb96c396377d48aa81ca39afb434e63383d9bbc9a2b128cad3d711836c", + "size": 16647 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "f552621bed4336a909ad023fbfc6cc92d519a636c9a9ec9a111ff7e18da443b5", + "size": 13948 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/summary.json": { + "sha256": "8414045dc78cfc184286ac632a4458262cd44bcfb42be899d90ac810427eb2ab", + "size": 26531 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/config.json": { + "sha256": "fce9c58160caecd933d80ceef6358e290234c987844e6d4899d9fa0c50787ddf", + "size": 1868 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/job.log": { + "sha256": "9b2155e667aa08ceaa620011d0ebdb5c8993240b30bf9ac1a64aa327998bc87e", + "size": 14756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/lock.json": { + "sha256": "f49c8b93cb8169f1f33f0f6647ce51e5a6d5cbf98d4de042f4d3c334d4a7d427", + "size": 3573 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/result.json": { + "sha256": "992b978af523605543a18254a7c7ec54c0085bd3136ecdf0fc72da58b4c70c0b", + "size": 1133 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/bootstrap.json": { + "sha256": "de6386848a0641d6b1d2e621e79f123858769d9d0245868f7adc8fb20198840b", + "size": 23136 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/pi.txt": { + "sha256": "848b9e05a46052df967ce6d20e2ae444ac14c2fe3a2f29c8048a6d618aa00a87", + "size": 286485 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/pi/sessions/2026-10-29T16-43-18-513Z_01a1888b-6271-74f5-855f-5bee6f84d1ac.jsonl": { + "sha256": "e6f9fcc1a4bd138a8ae7a6b12b925e459405d6a48fb991432e1697a4779c91ba", + "size": 81716 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/config.json": { + "sha256": "f36bac7468a34bb4d4b55c08d5eb3b394f8528b2968a0a7c21159fde53798beb", + "size": 1840 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/lock.json": { + "sha256": "3d9e1de75cccef2ca22c5fd6493df0ce73397e40eb0978014fb46c1bf78e4fa9", + "size": 2411 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/result.json": { + "sha256": "2f728b9db3eba0974022fe8d830895cb5a55e252a734a1d25f3bddf70dbe8db2", + "size": 38585 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/trial.log": { + "sha256": "9b2155e667aa08ceaa620011d0ebdb5c8993240b30bf9ac1a64aa327998bc87e", + "size": 14756 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/attempts/1/ctrf.json": { + "sha256": "3fefbf805f9c53acb52e4cc90c0d603c615c094f0a574ef20a02496e3fa6cade", + "size": 1770 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/attempts/1/test-stdout.txt": { + "sha256": "24e20ffdf267d986308587eb5183202b20f3e4bbdd9060c2688e5a6449e8eae6", + "size": 15112 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/ctrf.json": { + "sha256": "3fefbf805f9c53acb52e4cc90c0d603c615c094f0a574ef20a02496e3fa6cade", + "size": 1770 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/retry-summary.json": { + "sha256": "5e4f9f49a5a81dea915dd9e07566b1356873f57911d2d5c9bd78a6a293f61cb6", + "size": 229 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/test-stdout.txt": { + "sha256": "24e20ffdf267d986308587eb5183202b20f3e4bbdd9060c2688e5a6449e8eae6", + "size": 15112 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/config.json": { + "sha256": "562152781607ee0826aabc277362fca6c0edb3cddf6602d2f0b4e0d541f1ea5f", + "size": 3380 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/harbor.log": { + "sha256": "21af15ee4d70fa6d55dd398bb53e178763176658399304e71b93ca1e71848e4b", + "size": 1358 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/manifest.json": { + "sha256": "99d2cf512cdef16b24b8014a02642fde07c9dc295254b88cb478beaa899cfcda", + "size": 36747 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/pi-models.json": { + "sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "size": 1050 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__init__.py": { + "sha256": "47db4dc0f110bb7c9b5af115a3188166abbde743c3c6f71a1e844aefc903058e", + "size": 248 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__main__.py": { + "sha256": "1458974abcf315361d7a8d2dba4cec00949e35889452c3f1905c96061a7ee043", + "size": 11409 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__pycache__/__init__.cpython-313.pyc": { + "sha256": "bf47984664cd39a07a0d4b8e26f8e4a9a2815a9a8af2100336eb419c9a364803", + "size": 566 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__pycache__/adapter.cpython-313.pyc": { + "sha256": "554ffe7a40155204bb3074042554c04f2744faccb1566ac25542ca54192d01e3", + "size": 18744 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__pycache__/bootstrap.cpython-313.pyc": { + "sha256": "81cb811dd4e0c0c057d178320079f97c947fa46d2cf92900c478ffbc07870e03", + "size": 11212 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__pycache__/pi.cpython-313.pyc": { + "sha256": "7f267297cc06918baea1e0edf719ac2032d8570c5fcfe74fe8a197919a454ad8", + "size": 16639 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/__pycache__/verifier.cpython-313.pyc": { + "sha256": "876bad558136172e4cff17e20e399105a3d46b26a695abe560443e161c424d0a", + "size": 13940 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/_agent_runner.py": { + "sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "size": 5489 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/_verifier_runner.py": { + "sha256": "d718f7c5528c156df549d68ab2deff247a289d08a453c6d083b72d0e9c7013dc", + "size": 6712 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/adapter.py": { + "sha256": "d1aa154ec6972d706f0725cfa69c56635506d4499dff2a129e2c957ea7034c02", + "size": 13070 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/bootstrap.py": { + "sha256": "f499bfddfc3d29338cae15a7f8f6e9c66d19eb69ad9577e607ced4b1aa057be2", + "size": 6988 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/pi.py": { + "sha256": "76317028cb1c41112e5bc6dab806bf6588959e38442a59ab0e9d3ea041517d92", + "size": 11123 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/source/harbor_adapter/verifier.py": { + "sha256": "35592bf5f342b43df57c2524df40eb0f0d8284b9ce9acbe145a104a3955d48da", + "size": 9270 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/summary.json": { + "sha256": "76881f5635af18d58872888b9e0033e8c94f546e8ca298db0b1a3c50e32bc689", + "size": 22156 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/working-tree.patch": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/config.json": { + "sha256": "d065f2628703d184734d5738e68a763f21ea1c0421ed71ed96f52d091df75f3c", + "size": 1844 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/job.log": { + "sha256": "bd4896d54e645a24c581f9a8fbad2a8aafa2209e0ba1d9f44481dbba283af317", + "size": 13634 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/lock.json": { + "sha256": "4a5e7d60c5fdac1137e6af83e98f89444cbf43b591a142e7ac150fc3646a3bac", + "size": 3557 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/result.json": { + "sha256": "1a68f32993d3bb99e9517996a93a74ac072b7c77f82718241d9ef063182e9c21", + "size": 1131 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/bootstrap.json": { + "sha256": "7916b597efe47de0c85af554178d5a6c602b8ebeb9f99cd3b6a0a78ae17ae367", + "size": 19235 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/pi-stderr.txt": { + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855", + "size": 0 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/pi.txt": { + "sha256": "964e72fb3b4068a0b45ea83df2ff47883e4ade1292724e3bda1d4b9f7f175a40", + "size": 763926 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/pi/sessions/2026-10-29T16-59-42-205Z_01a1889a-64fd-75de-a2c0-014393c77247.jsonl": { + "sha256": "9a6b10c7753d4f6963785e39267ef43af028c36ee988bbd966faca59d6ca8dc5", + "size": 207703 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/process-supervision-required": { + "sha256": "94e95f0bb38f8e0fb52a6dab21f7708dde104b7edeb2eb5ce6b7e68f1328c295", + "size": 3 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/process-supervision.json": { + "sha256": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af", + "size": 42 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/artifacts/manifest.json": { + "sha256": "c1db653f487fb773c4f9247162ab3314e7f0584a486914b9874c4b3d6949e42d", + "size": 159 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/config.json": { + "sha256": "9c58d3185cc12401b89eddfb8c80df29020d21ed66f9a3aaab884386da46e771", + "size": 1808 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/lock.json": { + "sha256": "e600183465696b775ec60db76ba47a3b4fae1c47285d6ef70775b56613f669a5", + "size": 2395 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/result.json": { + "sha256": "e82f944631ef759a2dc3f206908effb487aff1c18532933304185235fdbe2523", + "size": 32990 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/trial.log": { + "sha256": "bd4896d54e645a24c581f9a8fbad2a8aafa2209e0ba1d9f44481dbba283af317", + "size": 13634 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/attempts/1/ctrf.json": { + "sha256": "1fc344036b8b92c0bc112b6410a40bb6674c6bbd7f6e9b8cad74edf589dbb3c7", + "size": 1466 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/attempts/1/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/attempts/1/test-stdout.txt": { + "sha256": "8a706364dc5ab77f58519e1f1c4fa5b65d6a31977b063e876d7e9a04a7113b72", + "size": 1690 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/ctrf.json": { + "sha256": "1fc344036b8b92c0bc112b6410a40bb6674c6bbd7f6e9b8cad74edf589dbb3c7", + "size": 1466 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/retry-summary.json": { + "sha256": "93b49e6f798e95f4e2b82696b0f6cd594b0328be3c2b536d3d2788b5fa3902ee", + "size": 233 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/reward.txt": { + "sha256": "4355a46b19d348dc2f57c046f8ef63d4538ebb936000f3c9ee954a27460dd865", + "size": 2 + }, + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/test-stdout.txt": { + "sha256": "8a706364dc5ab77f58519e1f1c4fa5b65d6a31977b063e876d7e9a04a7113b72", + "size": 1690 + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/path-tracing.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/path-tracing.json new file mode 100644 index 0000000..a0910e8 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/path-tracing.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json new file mode 100644 index 0000000..bc28496 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-distribution.json @@ -0,0 +1,852 @@ +{ + "version": "1.4.0", + "source": "Existing local upstream binary distribution; no rebuild or source modification. Its registered file hashes pin the local bytes; an upstream archive checksum has not been independently verified.", + "original_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "files": { + "CHANGELOG.md": { + "sha256": "b0eac55f99b20254f81cf925a52f054ec87c1320454a3fd7ee624683e585add6", + "mode": 420 + }, + "README.md": { + "sha256": "fbc7abec7d0829ac8bc4b217e950794e1ba4d8d1b3e0850f970da318ad72e863", + "mode": 420 + }, + "assets/clankolas.png": { + "sha256": "169acd0dfe6fbb8d8742ed24a3fc654fd0b2e2d4223c733249c5493723f1b72d", + "mode": 420 + }, + "docs/cli-integration.md": { + "sha256": "edf8e723a70b4c3c40275db0391219e55d30597936212a9cfdef33963dd37f93", + "mode": 420 + }, + "docs/cli.md": { + "sha256": "c45e59f31574de71770a71c9a7c9b6ac03bf9cad4928aa31bb6614d661733679", + "mode": 420 + }, + "docs/codemode.md": { + "sha256": "3a96ee10da36395bbd22beb60ac029f8ca8a859f2fd3c3104cfc38f9d5a1f95a", + "mode": 420 + }, + "docs/compaction.md": { + "sha256": "b5b23d10bb86d8ab32b67f80a7e57febd953222c6da6fd422ae91d232ef28faf", + "mode": 420 + }, + "docs/configuration.md": { + "sha256": "fc48d1ce7d885c5253222415fb7a4b46d75f80e6431eb924174657ff775099eb", + "mode": 420 + }, + "docs/containerization.md": { + "sha256": "e309fcb9a0a7592c0dba000ef8ee90f2d7ef1e01e6a1ac2f0cd9760dd63dee0c", + "mode": 420 + }, + "docs/custom-provider.md": { + "sha256": "fe926ed6194756c85ab6b56b7bfd12e81abe385acf39e0bb2a470eb6c70b8c59", + "mode": 420 + }, + "docs/docs.json": { + "sha256": "fc0bccd6bf66716ad1067d4fbf7258f674d89c89a12cb0b2587a027b9d7fdc49", + "mode": 420 + }, + "docs/environment-variables.md": { + "sha256": "097f2101e285f51f509be0b176d458825b83b929a635ae29d2a43703168170dd", + "mode": 420 + }, + "docs/extensions.md": { + "sha256": "46fc0baeb00db8ca8db399e33e0f506e86e425693b291258357e8063e2f3353c", + "mode": 420 + }, + "docs/how-pi-works.md": { + "sha256": "7d43ab8e2b886cd1047dcee503588a1c828b92333dd4f177526b89e6db8aaa8b", + "mode": 420 + }, + "docs/images/doom-extension.png": { + "sha256": "c3b89b5e1fba711a5bcfa188e13d8695a84f7cdda12d9e127bb45b36d13cbb27", + "mode": 420 + }, + "docs/images/exy.png": { + "sha256": "a8f253347440aed01ff659f41f573d340bcfd6010d829726ae92fe18abf601de", + "mode": 420 + }, + "docs/images/interactive-mode.png": { + "sha256": "4078001e39978151bd66b1d0a20123e7ca2ec2d1b572dfd70f552be5e88c0692", + "mode": 420 + }, + "docs/images/tree-view.png": { + "sha256": "ca61e7bbf300dfb3840cc191dc66dc4e9f5166b2f1f4089e6fb13fd04a2c3c66", + "mode": 420 + }, + "docs/index.md": { + "sha256": "c107426e88eac570e9ecbea325cb45c2a6b778a324fd6e424bd032380a65614d", + "mode": 420 + }, + "docs/json.md": { + "sha256": "f103191816fdcf3627dfc98b89acad8a28eee737697130ddaeb2581131213543", + "mode": 420 + }, + "docs/keybindings.md": { + "sha256": "413aa1c5e95a934d216aa5aba20052fa2cf35940ba5f02d1f4b22ff53c6ec678", + "mode": 420 + }, + "docs/llama-cpp.md": { + "sha256": "dccba80951463657c57a831aa2da3a576b92c05455f9b73610b208c942dfc8f4", + "mode": 420 + }, + "docs/mcp.md": { + "sha256": "23b9cca35d1340e8adce6bed8b495da0ebb58931179a937db87d610a1b40b9bf", + "mode": 420 + }, + "docs/message-types.md": { + "sha256": "b832893fb762f87fdbbfd694241d484592942823791f3e81a6c4d915421ff5db", + "mode": 420 + }, + "docs/models-configuration.md": { + "sha256": "1eb60b0f1faad6f9fffd62ea257b545dbc983dbe4c5decc4803b1a241d604155", + "mode": 420 + }, + "docs/models.md": { + "sha256": "c9914f5411db5ddd158b5391ec1da76c1f65788cfaea3a90e1e1555451b3d12e", + "mode": 420 + }, + "docs/nvidia-rtx-llama-cpp.md": { + "sha256": "bcd12ebfd283d8f528cf206ee4ec74f981ab36d3604419acc434c595decbebdb", + "mode": 420 + }, + "docs/packages.md": { + "sha256": "5fe4a3009d7260fe1e2f54b64086fc3c8b4d678232bfa2dbc7e91fe588fc9ea5", + "mode": 420 + }, + "docs/prompt-templates.md": { + "sha256": "ad5113cce0d05ef0ac77b3c71f9ebe41ea68c072275f9273b9616e4924feda42", + "mode": 420 + }, + "docs/providers.md": { + "sha256": "bc8798e4cc3209e455e3dc50c6652ae73c057d40122a4a741d8b5b65a041c2f1", + "mode": 420 + }, + "docs/quickstart.md": { + "sha256": "10c69b394304f6e7d6409b6bde02b0165f5723ff4acfd49cfa8f7414f977266a", + "mode": 420 + }, + "docs/rpc-commands.md": { + "sha256": "4deeee367852f4633ef594bc079f9e9fb14640a4e3ba8a69b7c89b6f54740838", + "mode": 420 + }, + "docs/rpc-extension-ui.md": { + "sha256": "3c6f2c6edfc8b1ace6642343727aef97a7e9234c85509850921b52d6ff09a48e", + "mode": 420 + }, + "docs/rpc.md": { + "sha256": "37a03d180115460a0a275022f4aeabef1e5345458df6a9b6e8a6de18474de720", + "mode": 420 + }, + "docs/sdk.md": { + "sha256": "d9034ccbf2219a58712d8dd4fcdf47e687d77131e71dd5d119d8a71e9bd2a2e8", + "mode": 420 + }, + "docs/security.md": { + "sha256": "3b1e40a8d28b17c9eb7e496a7ba88fbe04665e3e6ae02873a58b943e325ad504", + "mode": 420 + }, + "docs/session-format.md": { + "sha256": "4f8fc42179b332bcf71fe4c85cc95d914853ff7486d68742954a45316b512a4a", + "mode": 420 + }, + "docs/sessions.md": { + "sha256": "152d6489f61a6de9349c8ceaade95d5498fbc74f21771f41ba8e87ce0e5d8936", + "mode": 420 + }, + "docs/settings.md": { + "sha256": "2915d26f91f4c46ab1b92e63cbc46b8d48888fa0e491168642a5a90f973bd814", + "mode": 420 + }, + "docs/shell-aliases.md": { + "sha256": "ebb55456870455b4da91cfd054ae2ac9b3a417e6ff48a009587c33e68ecd7807", + "mode": 420 + }, + "docs/skills.md": { + "sha256": "5b9805525b194474ef1fe5455a0917ef15ed7f710aa9bec2eee11879556469e5", + "mode": 420 + }, + "docs/slash-commands.md": { + "sha256": "d7e1db36e9186ff146a6c27de31fe169671f52cb6f7b93e1bf8d7fe76209a79c", + "mode": 420 + }, + "docs/terminal-setup.md": { + "sha256": "583381fb9dc466135a865ca406a3be299a7a8da2c9d810c5b73f401912a0a357", + "mode": 420 + }, + "docs/termux.md": { + "sha256": "1b0687157f2384385d07800ec23e6c225ef28fbf6648f55453cc7ce62f4c64ea", + "mode": 420 + }, + "docs/themes.md": { + "sha256": "6f11eb43ca6dad4c71bcfa2fc72ac13c013e3cb9383688b88923e0139b2d097c", + "mode": 420 + }, + "docs/tmux.md": { + "sha256": "9ec0abdbf74c57d51d4a93f308fb4aec4d8a3574a24cbfd9ba481d2e39f1273c", + "mode": 420 + }, + "docs/tui.md": { + "sha256": "20d3695ebd11e179a194f535f73870247544f655ebf22e42726c3be559c6d786", + "mode": 420 + }, + "docs/usage.md": { + "sha256": "6dfe7446c81d601d5ebef5769523115229f802da364fe90fcdf96e322ae9fbcd", + "mode": 420 + }, + "docs/virtual-models.md": { + "sha256": "5ab04dee3b3712b317522bb0784e28a3fef2e6b7fe1aeb1c1ca27778685a38f7", + "mode": 420 + }, + "docs/windows.md": { + "sha256": "e5a51792381a9eec859f034153c67f9ff52c36d3cff4c7a1faa0651f77f627c3", + "mode": 420 + }, + "examples/README.md": { + "sha256": "294836cfa47cd2cff1cab628533aefc5fe2d888621e43fe9f4101c09c2917a98", + "mode": 420 + }, + "examples/extensions/README.md": { + "sha256": "b51e1095d338ed8bc6cb0a40dca12da66ffd12d6f4ec7bfd57815fac84d87662", + "mode": 420 + }, + "examples/extensions/auto-commit-on-exit.ts": { + "sha256": "b89b5cb4c6f1f07b0b4291ab85a3c5f025d7d31ebb285821716502452b46af62", + "mode": 420 + }, + "examples/extensions/bash-spawn-hook.ts": { + "sha256": "c79c87653e20ac6bd9f4813aa4bb01a483b3637700fa2edda220703958532662", + "mode": 420 + }, + "examples/extensions/bookmark.ts": { + "sha256": "a71d9449d415ad75c9f3d0d13afde129b9bea2d695b889566db17ecfca256d79", + "mode": 420 + }, + "examples/extensions/border-status-editor.ts": { + "sha256": "ccbbfddd4ff402432293b7adb21f58e869f37a5d12d4da81cea10cc5fc2311cd", + "mode": 420 + }, + "examples/extensions/built-in-tool-renderer.ts": { + "sha256": "4bfd0953e3dbf96ee0cdb26989a60ec91d4850323852ef09eafe5c2efdf23c00", + "mode": 420 + }, + "examples/extensions/claude-rules.ts": { + "sha256": "8a935f14f62c1c1a3e937f5c912c2215e25aa5fa591716e23c86224ce1dc34cd", + "mode": 420 + }, + "examples/extensions/commands.ts": { + "sha256": "36716b53da169936c7e1360a4fde1e2fc0c3356a6505f177235f09c5538c6e4f", + "mode": 420 + }, + "examples/extensions/confirm-destructive.ts": { + "sha256": "55502f82109df07e922ef3ec41af1b1a5ed33d4d6d56076654ab2bc88e2e0888", + "mode": 420 + }, + "examples/extensions/custom-compaction.ts": { + "sha256": "ea797b3508fd8600526f381466e8b08bb84f3d234932a9d6ce512698527d0aa9", + "mode": 420 + }, + "examples/extensions/custom-footer.ts": { + "sha256": "1e5f17bec95a48d407fbcc092faf7e1f6021c30cee4a66ec09bc120412292cb2", + "mode": 420 + }, + "examples/extensions/custom-header.ts": { + "sha256": "5de09cf861d3965201991d6e837d6b68e8a2e80cb89279ee9177cc05d31f723d", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/index.ts": { + "sha256": "52bb733e1a25d9e175a644e6a6e17bbf61071beecf0f2903869baa9579a05ab3", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/package-lock.json": { + "sha256": "4c1ccae817de13a0d0eee610fb12ee699b3c1f97cf28cbe83ec50cab0f338a77", + "mode": 420 + }, + "examples/extensions/custom-provider-anthropic/package.json": { + "sha256": "b2ceede9b12cef79f37d02c8e850132afdab1b3316c16d622c0c8343244e85c3", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/index.ts": { + "sha256": "0c4adac82eff25fef23223af77e11cd53535834fece9731009b125446e213158", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/package.json": { + "sha256": "4af767a958f402efc19fc6157c75c209edc45c8ab6103b60f13973369ba41990", + "mode": 420 + }, + "examples/extensions/custom-provider-gitlab-duo/test.ts": { + "sha256": "58dfa6fdb63521ccad551b7463ff56d4aecf735074934a8e9fc34b71e7083d00", + "mode": 420 + }, + "examples/extensions/debug-provider.ts": { + "sha256": "a915f86a26a0257537939701797b1f613072275e76bdd8a27dd941bbf519643a", + "mode": 420 + }, + "examples/extensions/dirty-repo-guard.ts": { + "sha256": "0037a69abdf4cf98181db799bef5bdee8981861315eb82ddab78126c2b0f13c4", + "mode": 420 + }, + "examples/extensions/doom-overlay/.gitignore": { + "sha256": "b26aa26693c610d46d9c8b387b28d20dc5196a73b979f5522e24dd689aab7bc8", + "mode": 420 + }, + "examples/extensions/doom-overlay/README.md": { + "sha256": "e17bc73b76c7977ffeef99120c7896ff992cf5938a60c4d7427a7b19fc180d97", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom/build/doom.js": { + "sha256": "c3147cf4881c88b4210c017c26828f52aff83f0c4db27d7dad3fe6cda9ade1fb", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom/build/doom.wasm": { + "sha256": "571d161956593508cf4ade732ae93753f00484bb526667a8676571cca14dec7d", + "mode": 493 + }, + "examples/extensions/doom-overlay/doom/build.sh": { + "sha256": "ab09c9b6dc98ea7c1fa8e74c13981ce69b698c01aec77309c5c7d40ad114d38c", + "mode": 493 + }, + "examples/extensions/doom-overlay/doom/doomgeneric_pi.c": { + "sha256": "98210909e1e7dee71cbbdba64f1db4bebd88b504e861ee36ea42093c59b251a1", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom-component.ts": { + "sha256": "d95dc47c92db0fdc5b0e0b56a277c67f277012d3f7296df745d8a93f64d5a42e", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom-engine.ts": { + "sha256": "706e43f70bd374b73928e5a44d196d1dd5c03d93979d08bdfc01d049b5083855", + "mode": 420 + }, + "examples/extensions/doom-overlay/doom-keys.ts": { + "sha256": "e8bd99a4ce90dd44ad120c58f3b2ba7bc0b0c662bcdc4ed4994d059130be9fdd", + "mode": 420 + }, + "examples/extensions/doom-overlay/index.ts": { + "sha256": "f89818dd75c7d8970458b1dbb9c5e85fea9b86534fd740c898c1f6f9c9f9ceba", + "mode": 420 + }, + "examples/extensions/doom-overlay/wad-finder.ts": { + "sha256": "20254888262081bf96932253bfd346971deefa48d8df011fd3baa7aa2d653b7d", + "mode": 420 + }, + "examples/extensions/dynamic-resources/SKILL.md": { + "sha256": "f8f34b0d81aa791a2d56501756fe0ace21fb78ca2754025d093f41d3b0ccaf69", + "mode": 420 + }, + "examples/extensions/dynamic-resources/dynamic.json": { + "sha256": "2bf0164779b4ea3c2572f33b1e979c427a4839193998fda19bfc5cba2a4692c5", + "mode": 420 + }, + "examples/extensions/dynamic-resources/dynamic.md": { + "sha256": "e61e2c971d02d628a58f7c36b9b37bd3e636e4ad60618a7f65e2b41410c38f35", + "mode": 420 + }, + "examples/extensions/dynamic-resources/index.ts": { + "sha256": "dfbc49418fe46ebc9696fa335709df2ed0d4c4703da8c1ad0eb317d11fce8433", + "mode": 420 + }, + "examples/extensions/dynamic-tools.ts": { + "sha256": "4394098c11f98ffc26e2487acda2e09b9467ef9ff5867def7273cb0c82a2b6c0", + "mode": 420 + }, + "examples/extensions/entry-renderer.ts": { + "sha256": "e4c01fcf382972d570ebf37d044e32065eae062fe70987b1b885d75d46aa0b11", + "mode": 420 + }, + "examples/extensions/event-bus.ts": { + "sha256": "1185942b7afd83692d9e8da029e8594d702157398c693ec8b3ddb02b0780828a", + "mode": 420 + }, + "examples/extensions/file-trigger.ts": { + "sha256": "6f5fc3dc8fa387d8ca6973786c521feb94da58838b0e171a881fb6b3a5243ac0", + "mode": 420 + }, + "examples/extensions/git-checkpoint.ts": { + "sha256": "b60c85f797091a1f21ccb2a998027c6229fe995bdd62dbb1cdcc22946768f40b", + "mode": 420 + }, + "examples/extensions/git-merge-and-resolve.ts": { + "sha256": "a3c077c87544122abe83b275f3d527f60c80c87f460ea2499d66bde9b4880133", + "mode": 420 + }, + "examples/extensions/github-issue-autocomplete.ts": { + "sha256": "4646ef360e57aa98c787ba21ca2f2e8acb3db30c4db0503e64cdbea4a4c2ac65", + "mode": 420 + }, + "examples/extensions/gondolin/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/gondolin/index.ts": { + "sha256": "9a9897639de8127a2abe6bb0d00f0005cd9e9cf9d51fd15893a827e67326bf62", + "mode": 420 + }, + "examples/extensions/gondolin/package-lock.json": { + "sha256": "256d3e45b7c6d293bdfd88a3c4884d8bb38ecdbc4064c290255d2b4d193ea535", + "mode": 420 + }, + "examples/extensions/gondolin/package.json": { + "sha256": "010f1a949005f706239d788a2481c355028eb4c366b4ea16c87577e63caab905", + "mode": 420 + }, + "examples/extensions/handoff.ts": { + "sha256": "ea07b6f680357b677d0f43f0016961ee8cb1331d37ae113e9466a3acffb2e381", + "mode": 420 + }, + "examples/extensions/hello.ts": { + "sha256": "0aa4e9800c2526914d4c1edb00b2cfa9bd9dd6da5218289994cccd5f5bfa4934", + "mode": 420 + }, + "examples/extensions/hidden-thinking-label.ts": { + "sha256": "179dc3704b5c563b49a7f4577222bf9bbaa5becf98b85c75e56ab26c001c5b22", + "mode": 420 + }, + "examples/extensions/inline-bash.ts": { + "sha256": "69b9dfc34e7b1ada9afe10cda3384b0b1a6f3c3a47da5fb7d16f914399b023d4", + "mode": 420 + }, + "examples/extensions/input-transform-streaming.ts": { + "sha256": "5c251ebda6163c8279e3c6d5f0aee56421ed150255dbb954f08ca7d2280b3ad2", + "mode": 420 + }, + "examples/extensions/input-transform.ts": { + "sha256": "cf0f65d610631ca75d18aae1c8f1408139702f674cb2835f8c009b943776474b", + "mode": 420 + }, + "examples/extensions/interactive-shell.ts": { + "sha256": "36a7efb5b132e666c643129ce09c3f83f56aed9924563e6b7269e0e126611ecc", + "mode": 420 + }, + "examples/extensions/jev-router.ts": { + "sha256": "cf2c9ef4baf9233faa133e99255786b00518cb4977d068c811a11b3c96f0a2d6", + "mode": 420 + }, + "examples/extensions/mac-system-theme.ts": { + "sha256": "d7349bae999a798c71b283165e06d0142d41b0264bfdad11ddcb1ec8b62f8710", + "mode": 420 + }, + "examples/extensions/message-renderer.ts": { + "sha256": "bedc198df5215c0a758e0743224599003630741a4ff72d4327837ce309e5ec8c", + "mode": 420 + }, + "examples/extensions/minimal-mode.ts": { + "sha256": "040cbc7619b2e903888c8626e2fa55ca5e3f820535dfd1aadcb2060e156a8770", + "mode": 420 + }, + "examples/extensions/modal-editor.ts": { + "sha256": "4742283930949fad1920359540aa9d42c833d2d24eaa49564bfc08f90e328b0d", + "mode": 420 + }, + "examples/extensions/model-status.ts": { + "sha256": "b6c073c0bdf26ac17e2fed01d02b6447fd4e2763b7b6ca2f2b493574ced25247", + "mode": 420 + }, + "examples/extensions/notify.ts": { + "sha256": "70e4333e09ce00d546c116fd2e918abf7616c70b5da6e88ba8ee21a326afd483", + "mode": 420 + }, + "examples/extensions/overlay-qa-tests.ts": { + "sha256": "4b1778e9e247c9eb24fe48e13fff0272f6f78b8102b4ed085d03598c3b98773f", + "mode": 420 + }, + "examples/extensions/overlay-test.ts": { + "sha256": "801b7c2eeec57c3569187b4b4d851f2159223ec6e9772e406ec65ed728ca6db9", + "mode": 420 + }, + "examples/extensions/permission-gate.ts": { + "sha256": "e32086ba9dd9b4ba0fd308b65241d9affa0b5ecd3f90ef7ea1b9f2b30cab8cc1", + "mode": 420 + }, + "examples/extensions/pirate.ts": { + "sha256": "dd6ce684bbe7630e4a749fe133ef8b8b875adbc7e31284b17fe0b8157d51e013", + "mode": 420 + }, + "examples/extensions/plan-mode/README.md": { + "sha256": "1ff8d611b8fbafef4ca1e3dbc72136fbcc166461b69f859b5e961b15147bff39", + "mode": 420 + }, + "examples/extensions/plan-mode/index.ts": { + "sha256": "bef2bf0712aea7b7d0f73d174e24ac5309679e6cc143f7bcbf1a9e0121222af8", + "mode": 420 + }, + "examples/extensions/plan-mode/utils.ts": { + "sha256": "12ea28d57cc55e68e8d52a2f1d16df8b2e7f2749cd694d41453028002a2bdad9", + "mode": 420 + }, + "examples/extensions/preset.ts": { + "sha256": "7942b7f933f97bcc29024a87f8526969413ba6f87b0bec537648a2926a0a4e29", + "mode": 420 + }, + "examples/extensions/project-trust.ts": { + "sha256": "78781c1f2a1613cf31c02a29ff6a6abd86b60bdf34343465e46cd85c01491578", + "mode": 420 + }, + "examples/extensions/prompt-customizer.ts": { + "sha256": "429546f46795bccda4078deace5729b8230f6103807d424eaf71817c6e07da0a", + "mode": 420 + }, + "examples/extensions/protected-paths.ts": { + "sha256": "9ca66b1f3b1b9a61cc5ddc660f92cca2ea66c4264a1bdb6b00e0da5844399fe2", + "mode": 420 + }, + "examples/extensions/provider-payload.ts": { + "sha256": "e0016077798098f528b1129954c30a28809a1513924d9c2e0efda9328d503a49", + "mode": 420 + }, + "examples/extensions/qna.ts": { + "sha256": "a639a65a486fd05a42afcf14aa59dc32e505e41ada43869167ddfd789a3c7da6", + "mode": 420 + }, + "examples/extensions/question.ts": { + "sha256": "a1625d2c72a65d859154a7e66bef222d48ca9e3654a8bb910829ad350378a4c4", + "mode": 420 + }, + "examples/extensions/questionnaire.ts": { + "sha256": "3f1f719d81152ff013f5d36a97cc3976388d43f2d89bff0f19827d3443d903e6", + "mode": 420 + }, + "examples/extensions/rainbow-editor.ts": { + "sha256": "f95f17ba8a0376180bd23510c20151e83aadc74142e989f14227bf5c4196883f", + "mode": 420 + }, + "examples/extensions/reload-runtime.ts": { + "sha256": "911f26b872860a551572fa1d94d251ddd476d4ea9483b6052e9658d2784b38fc", + "mode": 420 + }, + "examples/extensions/rpc-demo.ts": { + "sha256": "3d54a396eee12eb04ea12d0ea03716a6edd3e8743502b6a529e3b8d429ae640d", + "mode": 420 + }, + "examples/extensions/sandbox/.gitignore": { + "sha256": "16d30e4462189fb14dd611bdb708c510630c576a1f35b9383e89a4352da36c97", + "mode": 420 + }, + "examples/extensions/sandbox/index.ts": { + "sha256": "b5e527aac27254002f27b8bb9de7bc62e242e39eeb7c963683096039e0aa173e", + "mode": 420 + }, + "examples/extensions/sandbox/package-lock.json": { + "sha256": "cd4e34dbd8a9d4b0efdb58becb6c4201eb2ad9e44228b9a272b6853bfb54539d", + "mode": 420 + }, + "examples/extensions/sandbox/package.json": { + "sha256": "4b4bfe0ebeecf2af747a3e023a5edc143cba9c36137d10051e1124d1c1ad9732", + "mode": 420 + }, + "examples/extensions/send-user-message.ts": { + "sha256": "9261e433c782f354530105e1c9723a19798868f72fd1f5d68af3a4624ad7e446", + "mode": 420 + }, + "examples/extensions/session-name.ts": { + "sha256": "a0b9fd94e2cc827931150cc409238b473da635ab1a1cddce633cae5937a3ae24", + "mode": 420 + }, + "examples/extensions/shutdown-command.ts": { + "sha256": "68dc7f9561aa9b1c0f39cae19c0f41d56171a99875972e5998de3caa75f96e09", + "mode": 420 + }, + "examples/extensions/snake.ts": { + "sha256": "3ef506ab52927e866fad096f83b7cbfd0eb50e1504dcabf1b1eb28a06745c937", + "mode": 420 + }, + "examples/extensions/space-invaders.ts": { + "sha256": "43281ae25bea645d44db01234f24df80e2f866771d02de0b416ab32fb3883a1f", + "mode": 420 + }, + "examples/extensions/ssh.ts": { + "sha256": "c03bdef919457013e3798d10250f34255742fd3dc79885ac3d321b34f6a1149d", + "mode": 420 + }, + "examples/extensions/status-line.ts": { + "sha256": "26e27b1158e980551f63f46e55dad055ac613779c2b8c39df4cddc11be26103d", + "mode": 420 + }, + "examples/extensions/structured-output.ts": { + "sha256": "72a3a665ce3de1419c837db6eb8a0e7a890310c6e02e4d59c454e5c9514c15ec", + "mode": 420 + }, + "examples/extensions/subagent/README.md": { + "sha256": "6a888330bf974657751e09f5d375a9039872d59d6606fc217c5105c7eca51b1a", + "mode": 420 + }, + "examples/extensions/subagent/agents/planner.md": { + "sha256": "452f21f65957753cec3b0772004209c108071be168e627fffd7191234c0ce64e", + "mode": 420 + }, + "examples/extensions/subagent/agents/reviewer.md": { + "sha256": "b5c3f13b67207e8e32de631f8cd52a373d9b956028e2f8c1fd8b737b5f239265", + "mode": 420 + }, + "examples/extensions/subagent/agents/scout.md": { + "sha256": "3b84e8e01c6e05c8dfbff1c845c0c6371f4f16350735d4d1dbd02bfa98b1d4e6", + "mode": 420 + }, + "examples/extensions/subagent/agents/worker.md": { + "sha256": "ca2b2298fcbd8bfafcfe61977dc8075001a64b9c4d8c289c3b5eaa896efe6238", + "mode": 420 + }, + "examples/extensions/subagent/agents.ts": { + "sha256": "cc85ecd33b89aa575bea3d2a9a9da7654a6305da4de02e5041aa5342a1de29b4", + "mode": 420 + }, + "examples/extensions/subagent/index.ts": { + "sha256": "1d11858d163e2a3a258af51c1923fc55ad825813acd6224462fd25ad61f5401b", + "mode": 420 + }, + "examples/extensions/subagent/prompts/implement-and-review.md": { + "sha256": "019ba4bf42d596ea03fc4564be5b2cc55a06227c95db19a4d5a5bb96697a75f9", + "mode": 420 + }, + "examples/extensions/subagent/prompts/implement.md": { + "sha256": "65462de33c97ac1372cfed270dad9af164605b05028c069dd4c013ff9792675b", + "mode": 420 + }, + "examples/extensions/subagent/prompts/scout-and-plan.md": { + "sha256": "9506399391e0d2c71826c45f26cc332175dd964465f99a8bb880e109f8ed89f4", + "mode": 420 + }, + "examples/extensions/summarize.ts": { + "sha256": "9c75e838071773d5bacce27ac8c5305160c56958018884f960c3db96b3d09368", + "mode": 420 + }, + "examples/extensions/system-prompt-header.ts": { + "sha256": "aa97f1add4fe347b3fc37cd3709950a105b0db5d6cdaf19325b58a9fbfd05223", + "mode": 420 + }, + "examples/extensions/tic-tac-toe.ts": { + "sha256": "479597f4e76ef1eae3dae71ec9d5c26978c5b9221f950e3e134078c5b532314b", + "mode": 420 + }, + "examples/extensions/timed-confirm.ts": { + "sha256": "c35304599e6f07238ef5472efe9c3f2fef2a022f801750ed792ef9061c7de10f", + "mode": 420 + }, + "examples/extensions/titlebar-spinner.ts": { + "sha256": "ec56e7edf954966b5991bcad63141be1a09d17753cdd0c59efca1b9e649dadf2", + "mode": 420 + }, + "examples/extensions/todo.ts": { + "sha256": "e46824d00217e25242c186d41837cc84ca81b23f978500323448502a9a424ee2", + "mode": 420 + }, + "examples/extensions/tool-override.ts": { + "sha256": "5d3cc796311a16132db810fcd9e437c16abd46978b723b9120734e57cc545726", + "mode": 420 + }, + "examples/extensions/tools.ts": { + "sha256": "5eaebc40cffe885fee25c95e3d8aec79cfbd595b6eac8ac5a6de6c4afec3d5d8", + "mode": 420 + }, + "examples/extensions/trigger-compact.ts": { + "sha256": "92c3b831f9f7bf01a3e0a2df11f4e404f5ac5843000b94e8340a5d78f3cb3add", + "mode": 420 + }, + "examples/extensions/truncated-tool.ts": { + "sha256": "4ed5fbeb6da53eab8d1e04721d77961b8d1b3bf5e11d1512a17fc5a3f58f9955", + "mode": 420 + }, + "examples/extensions/widget-placement.ts": { + "sha256": "ffabd33cf58efb908de2025aac4cb41daf13dbeee24b5ad8dcd0361b43eff6d7", + "mode": 420 + }, + "examples/extensions/with-deps/.gitignore": { + "sha256": "4d56952b0fb13bf8f9b6c13a6d4c34a075bac3af447636a1df4335d7576e2f97", + "mode": 420 + }, + "examples/extensions/with-deps/index.ts": { + "sha256": "8d4dbc7028c891eac7f0386d25eecea3e9ac0de7fd841b5bb08137e2ec5ab443", + "mode": 420 + }, + "examples/extensions/with-deps/package-lock.json": { + "sha256": "a4863bc4e63ff31dd43a6b469835dd0057f8792d940af0a349ec35187ad03663", + "mode": 420 + }, + "examples/extensions/with-deps/package.json": { + "sha256": "1cedd74eb4891d548ff4ae0b90adb3ce73086fa21b11fe489e0d8b1ead1c5267", + "mode": 420 + }, + "examples/extensions/working-indicator.ts": { + "sha256": "e88295d765b78fd6aac0e93db92b183ee9b7103dc21577e3c5a42665dfad032c", + "mode": 420 + }, + "examples/extensions/working-message-test.ts": { + "sha256": "91ee09378f13cce142a4314bec1bfe9784e10484c33463c4fe3139c391c117a0", + "mode": 420 + }, + "examples/rpc-client.ts": { + "sha256": "b46b19c291fbcb59f8f9c539c2dbe12a43eae25e4d7528084844bbb79d27b6f7", + "mode": 420 + }, + "examples/rpc-extension-ui.ts": { + "sha256": "09bc33c28fdc9032e3752a487d659ac21c6776c6f589c5b1ae0967ef4ee5f127", + "mode": 420 + }, + "examples/sdk/01-minimal.ts": { + "sha256": "ffe5215e36f750f5ca7146cc7dfa5be36461292036d55b8db652b4d26331f351", + "mode": 420 + }, + "examples/sdk/02-custom-model.ts": { + "sha256": "dccc8dbdb1742ea00beeed87d302b8a23ac7f7e36a0b6a4c43b283c21be0ee55", + "mode": 420 + }, + "examples/sdk/03-custom-prompt.ts": { + "sha256": "d6db1f36a8e04466d9cbcc24ffb6fe0697858aa6c0e5013d5d90f12798314ef5", + "mode": 420 + }, + "examples/sdk/04-skills.ts": { + "sha256": "4cbf189bf88d14155281349d4d39cc236066d98095aae2d184d4ba7267ab5e42", + "mode": 420 + }, + "examples/sdk/05-tools.ts": { + "sha256": "5df5f9577b343d22d072d839341916536299c8191dc66c6c801a8bbddda23544", + "mode": 420 + }, + "examples/sdk/06-extensions.ts": { + "sha256": "8e01ee409200b6516e9a6a0d9f3b3f61d2ad78aaaa18bd2ef9a8f77820d149b2", + "mode": 420 + }, + "examples/sdk/07-context-files.ts": { + "sha256": "37bd18a284e961903d646e0cd1abdeda72cd968dbe4f40892e8d345a66916b36", + "mode": 420 + }, + "examples/sdk/08-prompt-templates.ts": { + "sha256": "a1b314988f1f5b2e61c80c19c79b68630a2f3c443008a200376286710a2744ed", + "mode": 420 + }, + "examples/sdk/09-api-keys-and-oauth.ts": { + "sha256": "d1d2b1bd4e57a3b2ab6f910068e8382759d12a94fbecf1332e96a9ebe4683eb4", + "mode": 420 + }, + "examples/sdk/10-settings.ts": { + "sha256": "558f82b9d0655935e6a28ec09f6c35f0f9593f8b034f25d45400c02bc77a778a", + "mode": 420 + }, + "examples/sdk/11-sessions.ts": { + "sha256": "538c7028bf6823c84be930eb6745731f2c87ba05a45df2d86d93e66f01674576", + "mode": 420 + }, + "examples/sdk/12-full-control.ts": { + "sha256": "3cf794d470ddbde4504ae965d24a23c42378d1ac86033522e46907ac05518343", + "mode": 420 + }, + "examples/sdk/13-session-runtime.ts": { + "sha256": "0e9f408720a561a62b2630eccb18ece49f9dba567dc65494757da0a91608827a", + "mode": 420 + }, + "examples/sdk/14-codemode-mcp.ts": { + "sha256": "02ae7714e22753a76b0f44033366d43529185b325b92d47bb5d1d0eb5ae34f5e", + "mode": 420 + }, + "examples/sdk/README.md": { + "sha256": "c1f9d8f58c300195d03a420222d2d5370d2884a18a4190a1d8c8ea9371c016ff", + "mode": 420 + }, + "export-html/ansi-to-html.d.ts": { + "sha256": "57d23f45aef05ba2b1a52a8e6b0e1f1ba51b55ab0b54e8d42938a438b088977a", + "mode": 420 + }, + "export-html/ansi-to-html.d.ts.map": { + "sha256": "2847dc4f48c9ec5c8cd7aeacf1a6083ab4fc44f4b7e6667547b1549215c68576", + "mode": 420 + }, + "export-html/ansi-to-html.js": { + "sha256": "44e5fd3585104e5e8cc0c7bea91a10efa7cb7e3b5bc5e2fb9090d5be4419df14", + "mode": 420 + }, + "export-html/ansi-to-html.js.map": { + "sha256": "d24d23d27ad31a21a4fb133c6e7ba2dc2c876f28656ff1254b1af3da234dc51c", + "mode": 420 + }, + "export-html/index.d.ts": { + "sha256": "cd83b5d85e893b8ca18742b28038883dd7cf06c98a203e00d74040f78a6a656d", + "mode": 420 + }, + "export-html/index.d.ts.map": { + "sha256": "1c31d2cbab7deb811274f473316e1fed48bc2a93707f2f964db86ba2f200d522", + "mode": 420 + }, + "export-html/index.js": { + "sha256": "9b7a893c46341d2557ba79dd88307797054577c8789665281d1dfb4134d347a3", + "mode": 420 + }, + "export-html/index.js.map": { + "sha256": "f2bf1920c337f74e83454665e5a1af9621cd9e67fbcca9aade7231eb624bf9a7", + "mode": 420 + }, + "export-html/template.css": { + "sha256": "8ee19851f8e583277ed396cbb76496687aa556707fdfc1f78d1118bff87c740a", + "mode": 420 + }, + "export-html/template.html": { + "sha256": "47d208235b066c856dae69c60281a3b99d164b99be9b81153d880375c9327105", + "mode": 420 + }, + "export-html/template.js": { + "sha256": "946bd9e3e8255c1ebb40f806def18007aa9397816291a7bf2ac0e4e99cec07a4", + "mode": 420 + }, + "export-html/tool-renderer.d.ts": { + "sha256": "ea982244e87c3568ad7bbb28427b83a19506136b5a498c7b496454338483ffd4", + "mode": 420 + }, + "export-html/tool-renderer.d.ts.map": { + "sha256": "2fab18f196e020f0b896b27db7cf790c233f0d0af55de04194e60b5bff137fce", + "mode": 420 + }, + "export-html/tool-renderer.js": { + "sha256": "8d3bd1f1b068996fe8f7d003bdb6623284043e1393a2db92ff45aeb9d6c33b9a", + "mode": 420 + }, + "export-html/tool-renderer.js.map": { + "sha256": "7bead30f87d1b345c70c66953a517fb51d1e1092089158ad5e92280cc97f6ea8", + "mode": 420 + }, + "export-html/vendor/highlight.min.js": { + "sha256": "837a6fa5b0c736b52bbde2b2b6190f305da3fc9ed41681db5321507057b5c846", + "mode": 420 + }, + "export-html/vendor/marked-cjk-friendly.min.js": { + "sha256": "93d2c938754cb24eef5b9ffecc268a98ef6beeb75dad614b0e874176aeeba565", + "mode": 420 + }, + "export-html/vendor/marked.min.js": { + "sha256": "438eedfcf932a414d0d0bfeea32dc365c063563b8ca713b4687fc8f8b501e5e4", + "mode": 420 + }, + "native/linux/prebuilds/linux-x64/linux-platform-x11.node": { + "sha256": "522d2550f7fa5d0d5bd204f45983f786ce2be9cf80f27f00179640d31d0689d6", + "mode": 493 + }, + "package.json": { + "sha256": "df6883685382d6a914e5797232744c26579fe7db5576ea0e9c859d018ca2d54e", + "mode": 420 + }, + "photon_rs_bg.wasm": { + "sha256": "10468181565c56004c867f3a4af96f89a0ef5a63a72f2b5fb12c1f1992a3615c", + "mode": 420 + }, + "pi": { + "sha256": "0d7611d0b86d38ec8f65dda8e8bee9c5e907c7bae381e9cfff9161b21e8087d0", + "mode": 493 + }, + "theme/dark.json": { + "sha256": "e1f96982f8cb7d2a48eca7db03c9cb282cb1e14a243680cd855638ecd6a251b6", + "mode": 420 + }, + "theme/light.json": { + "sha256": "2dee6a4be7f765e7fd7728166b7ebf8c29e4ce313f8ec6b38ae50699e3abc2f3", + "mode": 420 + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json new file mode 100644 index 0000000..32fd1b7 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json @@ -0,0 +1,42 @@ +{ + "providers": { + "deepseek": { + "api": "openai-completions", + "baseUrl": "https://api.deepseek.com", + "models": [ + { + "id": "deepseek-flash", + "name": "DeepSeek V4.1 Flash", + "reasoning": true, + "input": [ + "text", + "image" + ], + "thinkingLevelMap": { + "minimal": null, + "low": "low", + "medium": null, + "high": "high", + "max": "max" + }, + "contextWindow": 1000000, + "maxTokens": 384000, + "cost": { + "input": 0.3, + "output": 1.2, + "cacheRead": 0.006, + "cacheWrite": 0 + }, + "compat": { + "supportsStore": false, + "supportsDeveloperRole": false, + "maxTokensField": "max_tokens", + "requiresReasoningContentOnAssistantMessages": true, + "thinkingFormat": "deepseek" + } + } + ], + "apiKey": "${DEEPSEEK_API_KEY}" + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-complete-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-complete-evidence.json new file mode 100644 index 0000000..d9274af --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-complete-evidence.json @@ -0,0 +1,316 @@ +{ + "verified_at": "2026-10-29T13:56:06.008725+00:00", + "plan": "experiment-v6.json", + "installation_checks_passed": 34, + "registered_verifier_profiles_passed": 1, + "verifier_profiles_not_configured": 33, + "policy": "All 34 native installation-only results pass strict source, distribution, task, bootstrap, and input-config provenance checks. No model call or official task assertion occurred in these installation trials. QEMU is the only reviewed verifier profile.", + "accepted_trials": [ + { + "arm": "pi", + "task": "terminal-bench/adaptive-rejection-sampler", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json", + "result_sha256": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "bootstrap_sha256": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "manifest_sha256": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "input_config_sha256": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c" + }, + { + "arm": "pi", + "task": "terminal-bench/build-cython-ext", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json", + "result_sha256": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "bootstrap_sha256": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "manifest_sha256": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "input_config_sha256": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c" + }, + { + "arm": "pi", + "task": "terminal-bench/cancel-async-tasks", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json", + "result_sha256": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "bootstrap_sha256": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "manifest_sha256": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "input_config_sha256": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0" + }, + { + "arm": "pi", + "task": "terminal-bench/cobol-modernization", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json", + "result_sha256": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "bootstrap_sha256": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "manifest_sha256": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "input_config_sha256": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0" + }, + { + "arm": "pi", + "task": "terminal-bench/db-wal-recovery", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json", + "result_sha256": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "bootstrap_sha256": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "manifest_sha256": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "input_config_sha256": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2" + }, + { + "arm": "pi", + "task": "terminal-bench/fix-code-vulnerability", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json", + "result_sha256": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "bootstrap_sha256": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "manifest_sha256": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "input_config_sha256": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2" + }, + { + "arm": "pi", + "task": "terminal-bench/git-multibranch", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json", + "result_sha256": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "bootstrap_sha256": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "manifest_sha256": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "input_config_sha256": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa" + }, + { + "arm": "pi", + "task": "terminal-bench/kv-store-grpc", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json", + "result_sha256": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "input_config_sha256": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa" + }, + { + "arm": "pi", + "task": "terminal-bench/log-summary-date-ranges", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json", + "result_sha256": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "input_config_sha256": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496" + }, + { + "arm": "pi", + "task": "terminal-bench/merge-diff-arc-agi-task", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json", + "result_sha256": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "input_config_sha256": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496" + }, + { + "arm": "pi", + "task": "terminal-bench/model-extraction-relu-logits", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json", + "result_sha256": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "input_config_sha256": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd" + }, + { + "arm": "pi", + "task": "terminal-bench/multi-source-data-merger", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json", + "result_sha256": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "bootstrap_sha256": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "manifest_sha256": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "input_config_sha256": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd" + }, + { + "arm": "pi", + "task": "terminal-bench/nginx-request-logging", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json", + "result_sha256": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "bootstrap_sha256": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "manifest_sha256": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "input_config_sha256": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252" + }, + { + "arm": "pi", + "task": "terminal-bench/openssl-selfsigned-cert", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json", + "result_sha256": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "input_config_sha256": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252" + }, + { + "arm": "pi", + "task": "terminal-bench/pypi-server", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json", + "result_sha256": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "bootstrap_sha256": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "manifest_sha256": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "input_config_sha256": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563" + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-cli", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json", + "result_sha256": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "input_config_sha256": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563" + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-recovery", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/result.json", + "result_sha256": "237980344951ee2285e28499288d745bc857c1c96806129340725a5429acb0b4", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "input_config_sha256": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305" + }, + { + "arm": "pi", + "task": "terminal-bench/qemu-alpine-ssh", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/result.json", + "result_sha256": "35329f2737d85aa325756e9d62650e63ade91222fda61d370d8e5422d63074c6", + "bootstrap_sha256": "fc48c7dee6de95fe0edcd1631e1b3dbd900f2652d1ad5cbaa1d4a91f777feb53", + "manifest_sha256": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "input_config_sha256": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305" + }, + { + "arm": "pi", + "task": "terminal-bench/query-optimize", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/result.json", + "result_sha256": "783879855d9c2b4b7f592d981231b02d3f8d6ce92f62f2fcdb511556919c0722", + "bootstrap_sha256": "37f681eae6736199a9acbe0f9d1b1774eb82c7a097799622073de4b74d61c620", + "manifest_sha256": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "input_config_sha256": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39" + }, + { + "arm": "pi", + "task": "terminal-bench/regex-log", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/result.json", + "result_sha256": "00bf033a6dc65fd8557f7a2bd9cd6132508565d33815224057086789d8fe98a5", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "input_config_sha256": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39" + }, + { + "arm": "pi", + "task": "terminal-bench/torch-pipeline-parallelism", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/result.json", + "result_sha256": "198681d196f12f8a480dcfb2516e75b08f4b77516673f2562d5a82f770a2259a", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "input_config_sha256": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb" + }, + { + "arm": "pi", + "task": "terminal-bench/torch-tensor-parallelism", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/result.json", + "result_sha256": "6f06e8262444c01a2bfa76dfdbaf630eb2e9c416b4df66c273f10801aac77950", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "input_config_sha256": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb" + }, + { + "arm": "pi", + "task": "terminal-bench/constraints-scheduling", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/result.json", + "result_sha256": "d60e5bc61d23265f69ebe35ae2077a57c319220a653d09712fc7942394d2b6ff", + "bootstrap_sha256": "cf3d634a13469c9792d5f85e9e97f6a6fd2770edd21edd2323b1a4a0ff9b88bf", + "manifest_sha256": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "input_config_sha256": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49" + }, + { + "arm": "pi", + "task": "terminal-bench/write-compressor", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/result.json", + "result_sha256": "2f117df7e8d0f27aeed1e3a86f14255bbc4faeb3e06ef4464c6d0bb5f6556887", + "bootstrap_sha256": "a777f4d674a14f3a92030c0d28a0aeea8efc22dac6e5c348302d0ddccc73925e", + "manifest_sha256": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "input_config_sha256": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49" + }, + { + "arm": "pi", + "task": "terminal-bench/custom-memory-heap-crash", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/result.json", + "result_sha256": "9869ea4ef71a7e1e8005983d5439985d64d3c52ae4c1db2f0898d5be46f2561d", + "bootstrap_sha256": "f7e10349b73b17d26271980d4778162d3b278f8e1d4a71d42a4cb0d23b866852", + "manifest_sha256": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "input_config_sha256": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023" + }, + { + "arm": "pi", + "task": "terminal-bench/dna-assembly", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/result.json", + "result_sha256": "160dab77106b3651c220484d1b884dd8263aecbb5f78be413cec93e8a352d7e3", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "input_config_sha256": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023" + }, + { + "arm": "pi", + "task": "terminal-bench/llm-inference-batching-scheduler", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/result.json", + "result_sha256": "4cf30715e661bc55ded77f425af33e87b309aad80ba102f28e9e2ccf2cef3148", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "input_config_sha256": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362" + }, + { + "arm": "pi", + "task": "terminal-bench/path-tracing", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/result.json", + "result_sha256": "b6c8d393ebf552037d56d260f1b3035d5903f8b40a32f083682f518a2f37c2bd", + "bootstrap_sha256": "d2f6dec48072dbd7f15f33bdc2361c6fbfd5d345ddd4748d292bac71e13e8e82", + "manifest_sha256": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "input_config_sha256": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362" + }, + { + "arm": "pi", + "task": "terminal-bench/caffe-cifar-10", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/result.json", + "result_sha256": "64fd984e34c1d1d95be0f12e4b4e6637f67a3b75374a3bc69390f2f0a6df9854", + "bootstrap_sha256": "bfe8b063bf0a185c19fb3d6b3eba32ada212d94dfd2b1f082b9a49e957760a0a", + "manifest_sha256": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "input_config_sha256": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16" + }, + { + "arm": "pi", + "task": "terminal-bench/schemelike-metacircular-eval", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/result.json", + "result_sha256": "8ef5cdf5b93a0dc5f491d2651abc187516954f3866ff4df1faa18801f0ba9104", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "input_config_sha256": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16" + }, + { + "arm": "pi", + "task": "terminal-bench/circuit-fibsqrt", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-16-20261027/circuit-fibsqrt__AfGkgwD/result.json", + "result_sha256": "4202448e25c322c88057ef409cd3492e152ddcaa1c6f7132fd3e329759d59475", + "bootstrap_sha256": "bcfa3a0d9c4be3798cb32bcf016009e48b02df7e31360f67504ac46696df2865", + "manifest_sha256": "7a627b27b1887f38ef0352e0dc5fd84ac5043be6a5a76f5372f21afa6ee5ec09", + "input_config_sha256": "23cdd0d3a0a4635627074eec6e5a6bb3bb0bbf70b953a89c91a8ae141010d635" + }, + { + "arm": "pi", + "task": "terminal-bench/mteb-leaderboard", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-16-20261027/mteb-leaderboard__ouJofdv/result.json", + "result_sha256": "771427d6ac48cfdc28918962faa4ccafeef634328d9520e3628768c8093153c3", + "bootstrap_sha256": "66d0aa4ea40df556d0d0e8ac988025d2014de5428848589e768e3aaba0130c7f", + "manifest_sha256": "7a627b27b1887f38ef0352e0dc5fd84ac5043be6a5a76f5372f21afa6ee5ec09", + "input_config_sha256": "23cdd0d3a0a4635627074eec6e5a6bb3bb0bbf70b953a89c91a8ae141010d635" + }, + { + "arm": "pi", + "task": "terminal-bench/build-pov-ray", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-17-20261027/build-pov-ray__TDQmsPY/result.json", + "result_sha256": "beb680a5fcc6e08fb771a6585206c445f5c24236d2cca7056e437344958f2ea0", + "bootstrap_sha256": "bfe8b063bf0a185c19fb3d6b3eba32ada212d94dfd2b1f082b9a49e957760a0a", + "manifest_sha256": "3ba830a82a70dd2f2d364e2455d0830c2ec566708af927eb4aa850057b7568ef", + "input_config_sha256": "763aa30b7b78f981bcb7c81b09cef53c9545e719891160f57efc5c7cc39ebf3d" + }, + { + "arm": "pi", + "task": "terminal-bench/regex-chess", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-17-20261027/regex-chess__tRkUrhv/result.json", + "result_sha256": "c54a30aa68acaf53c88957db600d4aa548144318ce8f7875e9b185de9fcb9447", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "3ba830a82a70dd2f2d364e2455d0830c2ec566708af927eb4aa850057b7568ef", + "input_config_sha256": "763aa30b7b78f981bcb7c81b09cef53c9545e719891160f57efc5c7cc39ebf3d" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-30-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-30-evidence.json new file mode 100644 index 0000000..e50655f --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-30-evidence.json @@ -0,0 +1,294 @@ +{ + "verified_at": "2026-10-29T13:15:19.194052+00:00", + "source_plan": "experiment-v4.json", + "accepted_trials": [ + { + "arm": "pi", + "task": "terminal-bench/adaptive-rejection-sampler", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json", + "result_sha256": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "bootstrap_sha256": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "manifest_sha256": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "input_config_sha256": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c" + }, + { + "arm": "pi", + "task": "terminal-bench/build-cython-ext", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json", + "result_sha256": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "bootstrap_sha256": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "manifest_sha256": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "input_config_sha256": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c" + }, + { + "arm": "pi", + "task": "terminal-bench/cancel-async-tasks", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json", + "result_sha256": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "bootstrap_sha256": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "manifest_sha256": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "input_config_sha256": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0" + }, + { + "arm": "pi", + "task": "terminal-bench/cobol-modernization", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json", + "result_sha256": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "bootstrap_sha256": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "manifest_sha256": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "input_config_sha256": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0" + }, + { + "arm": "pi", + "task": "terminal-bench/db-wal-recovery", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json", + "result_sha256": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "bootstrap_sha256": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "manifest_sha256": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "input_config_sha256": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2" + }, + { + "arm": "pi", + "task": "terminal-bench/fix-code-vulnerability", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json", + "result_sha256": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "bootstrap_sha256": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "manifest_sha256": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "input_config_sha256": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2" + }, + { + "arm": "pi", + "task": "terminal-bench/git-multibranch", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json", + "result_sha256": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "bootstrap_sha256": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "manifest_sha256": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "input_config_sha256": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa" + }, + { + "arm": "pi", + "task": "terminal-bench/kv-store-grpc", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json", + "result_sha256": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "input_config_sha256": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa" + }, + { + "arm": "pi", + "task": "terminal-bench/log-summary-date-ranges", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json", + "result_sha256": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "input_config_sha256": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496" + }, + { + "arm": "pi", + "task": "terminal-bench/merge-diff-arc-agi-task", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json", + "result_sha256": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "input_config_sha256": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496" + }, + { + "arm": "pi", + "task": "terminal-bench/model-extraction-relu-logits", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json", + "result_sha256": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "input_config_sha256": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd" + }, + { + "arm": "pi", + "task": "terminal-bench/multi-source-data-merger", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json", + "result_sha256": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "bootstrap_sha256": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "manifest_sha256": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "input_config_sha256": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd" + }, + { + "arm": "pi", + "task": "terminal-bench/nginx-request-logging", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json", + "result_sha256": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "bootstrap_sha256": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "manifest_sha256": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "input_config_sha256": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252" + }, + { + "arm": "pi", + "task": "terminal-bench/openssl-selfsigned-cert", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json", + "result_sha256": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "input_config_sha256": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252" + }, + { + "arm": "pi", + "task": "terminal-bench/pypi-server", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json", + "result_sha256": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "bootstrap_sha256": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "manifest_sha256": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "input_config_sha256": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563" + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-cli", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json", + "result_sha256": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "input_config_sha256": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563" + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-recovery", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/pytorch-model-recovery__AnHZLLY/result.json", + "result_sha256": "237980344951ee2285e28499288d745bc857c1c96806129340725a5429acb0b4", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "input_config_sha256": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305" + }, + { + "arm": "pi", + "task": "terminal-bench/qemu-alpine-ssh", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-09-20261027/qemu-alpine-ssh__uiSwmJr/result.json", + "result_sha256": "35329f2737d85aa325756e9d62650e63ade91222fda61d370d8e5422d63074c6", + "bootstrap_sha256": "fc48c7dee6de95fe0edcd1631e1b3dbd900f2652d1ad5cbaa1d4a91f777feb53", + "manifest_sha256": "205227ed593d2e7089c95ddeb8c312b63c55fde199f9eee1e53e00abadcddddd", + "input_config_sha256": "fac554c1bd86ab46e0b68de769f244729e323622fb51f4f5319505537262e305" + }, + { + "arm": "pi", + "task": "terminal-bench/query-optimize", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/query-optimize__YoXxwZA/result.json", + "result_sha256": "783879855d9c2b4b7f592d981231b02d3f8d6ce92f62f2fcdb511556919c0722", + "bootstrap_sha256": "37f681eae6736199a9acbe0f9d1b1774eb82c7a097799622073de4b74d61c620", + "manifest_sha256": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "input_config_sha256": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39" + }, + { + "arm": "pi", + "task": "terminal-bench/regex-log", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-10-20261027/regex-log__N2XBsZY/result.json", + "result_sha256": "00bf033a6dc65fd8557f7a2bd9cd6132508565d33815224057086789d8fe98a5", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "73029898ccaa50e2329dab019769a02c4e2f2f8494ee2d7d0604942d93136019", + "input_config_sha256": "9bc5349337785a8ab8cf60ce4cef6d0251c2dc9eff7c5788d2a98705f058be39" + }, + { + "arm": "pi", + "task": "terminal-bench/torch-pipeline-parallelism", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-pipeline-parallelism__5YUs6Bj/result.json", + "result_sha256": "198681d196f12f8a480dcfb2516e75b08f4b77516673f2562d5a82f770a2259a", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "input_config_sha256": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb" + }, + { + "arm": "pi", + "task": "terminal-bench/torch-tensor-parallelism", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-11-20261027/torch-tensor-parallelism__4sM6zDL/result.json", + "result_sha256": "6f06e8262444c01a2bfa76dfdbaf630eb2e9c416b4df66c273f10801aac77950", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "c8f0caaef12ec583bc45b0f07d57f28b6e12411a01a2e83f0f9b5fc3358eded3", + "input_config_sha256": "76369d400f1b5cdfe50da369ef1cbd9237c488c909d4058a0ac3eb9452f76ffb" + }, + { + "arm": "pi", + "task": "terminal-bench/constraints-scheduling", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/constraints-scheduling__AZGqHo7/result.json", + "result_sha256": "d60e5bc61d23265f69ebe35ae2077a57c319220a653d09712fc7942394d2b6ff", + "bootstrap_sha256": "cf3d634a13469c9792d5f85e9e97f6a6fd2770edd21edd2323b1a4a0ff9b88bf", + "manifest_sha256": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "input_config_sha256": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49" + }, + { + "arm": "pi", + "task": "terminal-bench/write-compressor", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-12-20261027/write-compressor__jKdJYs3/result.json", + "result_sha256": "2f117df7e8d0f27aeed1e3a86f14255bbc4faeb3e06ef4464c6d0bb5f6556887", + "bootstrap_sha256": "a777f4d674a14f3a92030c0d28a0aeea8efc22dac6e5c348302d0ddccc73925e", + "manifest_sha256": "a993aa10afbc62f6d3194be71bfe78702cce21b655dc631ca236e8f3c9e74d0d", + "input_config_sha256": "f4bf827824c33c0905f014c04583b1c430227c4cdffad4815671f1df93787a49" + }, + { + "arm": "pi", + "task": "terminal-bench/custom-memory-heap-crash", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/custom-memory-heap-crash__7vzaXoA/result.json", + "result_sha256": "9869ea4ef71a7e1e8005983d5439985d64d3c52ae4c1db2f0898d5be46f2561d", + "bootstrap_sha256": "f7e10349b73b17d26271980d4778162d3b278f8e1d4a71d42a4cb0d23b866852", + "manifest_sha256": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "input_config_sha256": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023" + }, + { + "arm": "pi", + "task": "terminal-bench/dna-assembly", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-13-20261027/dna-assembly__9yECeYj/result.json", + "result_sha256": "160dab77106b3651c220484d1b884dd8263aecbb5f78be413cec93e8a352d7e3", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "49226525f97777a058bd5b6ee687b680643ceea9adb5bdceea3bc8f81b45ae23", + "input_config_sha256": "470fee212c4643ad04fed5cd56fd005324bcfe078dab8a5233586ba76083b023" + }, + { + "arm": "pi", + "task": "terminal-bench/llm-inference-batching-scheduler", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/llm-inference-batching-scheduler__K5Y4oFq/result.json", + "result_sha256": "4cf30715e661bc55ded77f425af33e87b309aad80ba102f28e9e2ccf2cef3148", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "input_config_sha256": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362" + }, + { + "arm": "pi", + "task": "terminal-bench/path-tracing", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-14-20261027/path-tracing__4iWGEgU/result.json", + "result_sha256": "b6c8d393ebf552037d56d260f1b3035d5903f8b40a32f083682f518a2f37c2bd", + "bootstrap_sha256": "d2f6dec48072dbd7f15f33bdc2361c6fbfd5d345ddd4748d292bac71e13e8e82", + "manifest_sha256": "6ee748971da3d53d2ee775bce318bd35a01e5b68487c9a1265b951a6c4725622", + "input_config_sha256": "997577d496b9f850ad312a4651f1bdae61798a2aeca088b6f4a1bda75b9f1362" + }, + { + "arm": "pi", + "task": "terminal-bench/caffe-cifar-10", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/caffe-cifar-10__AY4jS2o/result.json", + "result_sha256": "64fd984e34c1d1d95be0f12e4b4e6637f67a3b75374a3bc69390f2f0a6df9854", + "bootstrap_sha256": "bfe8b063bf0a185c19fb3d6b3eba32ada212d94dfd2b1f082b9a49e957760a0a", + "manifest_sha256": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "input_config_sha256": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16" + }, + { + "arm": "pi", + "task": "terminal-bench/schemelike-metacircular-eval", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-15-20261027/schemelike-metacircular-eval__eFmy8Fs/result.json", + "result_sha256": "8ef5cdf5b93a0dc5f491d2651abc187516954f3866ff4df1faa18801f0ba9104", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "7f82aef91c843c1e9f5a6735a4cda739cca4abde9810d33ca2414dc5d0dc801c", + "input_config_sha256": "d1230bcc921e5e92ec82864d2239e01cd7ca620a96904c48ab9cd2914a2acc16" + } + ], + "finished_jobs": [ + "pi-flash-dev34-install-pair-01-20261027", + "pi-flash-dev34-install-pair-02-20261027", + "pi-flash-dev34-install-pair-03-20261027", + "pi-flash-dev34-install-pair-04-20261027", + "pi-flash-dev34-install-pair-05-20261027", + "pi-flash-dev34-install-pair-06-20261027", + "pi-flash-dev34-install-pair-07-20261027", + "pi-flash-dev34-install-pair-08-20261027", + "pi-flash-dev34-install-pair-09-20261027", + "pi-flash-dev34-install-pair-10-20261027", + "pi-flash-dev34-install-pair-11-20261027", + "pi-flash-dev34-install-pair-12-20261027", + "pi-flash-dev34-install-pair-13-20261027", + "pi-flash-dev34-install-pair-14-20261027", + "pi-flash-dev34-install-pair-15-20261027" + ], + "policy": "Reuse exactly these 30 hash-bound, strictly validated installation-only records without rerunning them. QEMU verifier dependency preflight passed; other verifier dependency profiles remain not_configured." +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json new file mode 100644 index 0000000..51bafab --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/preflight-resume-evidence.json @@ -0,0 +1,161 @@ +{ + "verified_at": "2026-10-28T13:48:16.388757+00:00", + "source_plan": "experiment-v3.json", + "accepted_trials": [ + { + "arm": "pi", + "task": "terminal-bench/adaptive-rejection-sampler", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/adaptive-rejection-sampler__6Te54Hh/result.json", + "result_sha256": "bede3ee33bf8c4c86efeace98fdf7b9c8f84599b4aa28d2c53fa7b02998b153e", + "bootstrap_sha256": "107a57050bf7a93be2c59cc0bcd284fb89f8b9d55747bcf56d6b1492058adb12", + "manifest_sha256": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "input_config_sha256": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c" + }, + { + "arm": "pi", + "task": "terminal-bench/build-cython-ext", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-01-20261027/build-cython-ext__SoCrSf2/result.json", + "result_sha256": "417646b0f96760e99dd3c06d279ce1a5dfa17dae092db405d27d7865ae1cb224", + "bootstrap_sha256": "5e97300cf8ae1a899444962ea6a72d0776d7808f7217f645bf4897bac3bc0a7a", + "manifest_sha256": "96526880f69848f9b5dda5eede68be14bdf1573066a0ac719d281abb38d34a6a", + "input_config_sha256": "b93fb995c12eb836d3041fec030ae61ce43c325964bfbf94e0f39f2231b6464c" + }, + { + "arm": "pi", + "task": "terminal-bench/cancel-async-tasks", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cancel-async-tasks__4MwHm6o/result.json", + "result_sha256": "245f6ffa301f4436f3bf9af3e73c5115eb009890af20b3b8bfcb4ecf381908c5", + "bootstrap_sha256": "e36d9db211f82d053f323119da6c0481f2e54c4574c2ff416a549a21cee9ee11", + "manifest_sha256": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "input_config_sha256": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0" + }, + { + "arm": "pi", + "task": "terminal-bench/cobol-modernization", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-02-20261027/cobol-modernization__4x6hSTu/result.json", + "result_sha256": "6eba0295f874d2f0584ad1f2756bd3b8da93382f596adae8ee2b75310973f6a3", + "bootstrap_sha256": "58a0ffcdf92c941f982406666f7a28dc75c6c2d0ec3ad00ee53a061b0a5ea929", + "manifest_sha256": "957281108108cbb73f2540dd3c1dcd3cf7950892697efd2be95c230625c4e2da", + "input_config_sha256": "30720241f994771105113fff7dadd8e15713bb659edf367227a320c9b2e4d1f0" + }, + { + "arm": "pi", + "task": "terminal-bench/db-wal-recovery", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/db-wal-recovery__dfqafBo/result.json", + "result_sha256": "9b9219d3b76f2adb01bf4ecd00ba4755e6d57e75405b515163eb37adfed301bb", + "bootstrap_sha256": "cda890350d8f5d3436179f88f57b28bb479344e1092eed4c0183b83188de87d5", + "manifest_sha256": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "input_config_sha256": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2" + }, + { + "arm": "pi", + "task": "terminal-bench/fix-code-vulnerability", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-03-20261027/fix-code-vulnerability__PFUMwfw/result.json", + "result_sha256": "4c6a58f9c956638e2ea51654e86096e94209ff079e6a60d192340e3baf995f7a", + "bootstrap_sha256": "8f35283650fc595d50197361773886d6e38dd32a3536630691181dce490ad32e", + "manifest_sha256": "9c80a452fa536e61e9c630b73e138f6034d7124ace63f9cd20576aa09956dfd6", + "input_config_sha256": "a3bd8d582eaa6a2627fb9fad25377b7edd8fa24400454592e1b2714825e656f2" + }, + { + "arm": "pi", + "task": "terminal-bench/git-multibranch", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/git-multibranch__eQv2sNb/result.json", + "result_sha256": "84cd3c8c13dc436aeded24d289172eb5945dd288da7847bf2b53d6bffc9663ad", + "bootstrap_sha256": "1720e3c5fb724d4c6e97a1ee62e8b770141860b4d113115e449d2e0cb04a9d84", + "manifest_sha256": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "input_config_sha256": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa" + }, + { + "arm": "pi", + "task": "terminal-bench/kv-store-grpc", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-04-20261027/kv-store-grpc__tFkuA9x/result.json", + "result_sha256": "d79fa8a665c2a07cb94abfc7a10253e4252a9c110117b0a9b6b12219f1efa5b2", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "501839c542a32f566b7cdc1ef5cad8bc4e95f6582d2fc581e453ae8c492155f3", + "input_config_sha256": "60b13db69aff59276b54e200bcae55c0e1b479e8673caa85753a7f2f23c002aa" + }, + { + "arm": "pi", + "task": "terminal-bench/log-summary-date-ranges", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/log-summary-date-ranges__84sKwfB/result.json", + "result_sha256": "669bc907ae427deec28a9c94aeedb9bb7e9242c8df70c706ff7f23f9abc29601", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "input_config_sha256": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496" + }, + { + "arm": "pi", + "task": "terminal-bench/merge-diff-arc-agi-task", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-05-20261027/merge-diff-arc-agi-task__XezJkBY/result.json", + "result_sha256": "b68c55449459684b78871f764436a4536f8ca1be67fdf398f49605c2ec5e5cd3", + "bootstrap_sha256": "820faecc214746538858149a4d4b04c5b658d94e72a718cd8443ee1ea0642aab", + "manifest_sha256": "4ca9dc31f6cdcf0c3b3b3669dfeb5b5801967ae33b2f45f3e6014bf94cc83c30", + "input_config_sha256": "9845efd14817cc92ee85e66788e9e915a5b2a5aa51f63e745b0d43c1f14c5496" + }, + { + "arm": "pi", + "task": "terminal-bench/model-extraction-relu-logits", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/model-extraction-relu-logits__yZqevWz/result.json", + "result_sha256": "1d2726c8148e02864bc4f1c1f48ecc8b2914a0779bfe797dd718fc7dfa4559e7", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "input_config_sha256": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd" + }, + { + "arm": "pi", + "task": "terminal-bench/multi-source-data-merger", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-06-20261027/multi-source-data-merger__hqtrag6/result.json", + "result_sha256": "3ecbd1501eab148728beae4ff1d429276f2b0beb15c6148c7036c763e807e57f", + "bootstrap_sha256": "8ca622bedf9c3b7e582ecf18f6400c013835c31b4b5247f2332e65410776b84e", + "manifest_sha256": "a715046eff622e710f22c1a8f348c5397993a458eb495fc713dc9acd0bd86cf9", + "input_config_sha256": "e654d257d61c8deb61c0486900776b34fc54aa76303b8cbea664b0b8a57daecd" + }, + { + "arm": "pi", + "task": "terminal-bench/nginx-request-logging", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/nginx-request-logging__qAFnPiS/result.json", + "result_sha256": "0d935feb932f6a69fbc9de8c9ba116993605e083b51b10ce9a92792593c6a4a7", + "bootstrap_sha256": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "manifest_sha256": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "input_config_sha256": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252" + }, + { + "arm": "pi", + "task": "terminal-bench/openssl-selfsigned-cert", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-07-20261027/openssl-selfsigned-cert__KKFCiB5/result.json", + "result_sha256": "4b0d018e03547decacfcd0b1f63f5ded07b4145b2480085530ac8d49d7bd15af", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "0ef2a0610ef5b5282c5ee938aa3513e941cede4e81e93717ea2aa0d5b9f9d85b", + "input_config_sha256": "b89d7dcccc553313e9a0354638005b4236f9394809a24adf5c898029542a0252" + }, + { + "arm": "pi", + "task": "terminal-bench/pypi-server", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pypi-server__EY626hM/result.json", + "result_sha256": "a8d57e6d03023384f4ba0922543e19e31176f2821cde67414abcf2fc79555970", + "bootstrap_sha256": "c4bc7e0dfa218c9eff7094a838fe90d6f5b417c644e12c804ac6f31839944772", + "manifest_sha256": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "input_config_sha256": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563" + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-cli", + "result_path": "jobs/runtime-source/jobs/pi-flash-dev34-install-pair-08-20261027/pytorch-model-cli__ydFu92B/result.json", + "result_sha256": "5d73a586566ecde607f6f4ecf3ab41f394d70a40df501f8c12f56ab64a99f0c9", + "bootstrap_sha256": "77632931bfa9b4c5e29b6c27520eae7e23e30537a6e23fac4a06964c94d9bf17", + "manifest_sha256": "bd0bcb80ff146cd0ffa73c3b7a3009663a5e1ceb4aee96473de6980dcc685592", + "input_config_sha256": "6237cd926ec89929381c2e67501744c196f9471b463b5ceec84449f417458563" + } + ], + "finished_jobs": [ + "pi-flash-dev34-install-pair-01-20261027", + "pi-flash-dev34-install-pair-02-20261027", + "pi-flash-dev34-install-pair-03-20261027", + "pi-flash-dev34-install-pair-04-20261027", + "pi-flash-dev34-install-pair-05-20261027", + "pi-flash-dev34-install-pair-06-20261027", + "pi-flash-dev34-install-pair-07-20261027", + "pi-flash-dev34-install-pair-08-20261027" + ], + "policy": "All 16 native results, bootstrap records, source manifests, input configurations, and Pi model snapshots are hash-bound. Completed installation checks are reused without rerunning them. None contains agent execution or verifier results." +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json new file mode 100644 index 0000000..722c18c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/prior-image-digests.json @@ -0,0 +1,311 @@ +{ + "observed_at": "2026-10-19T13:14:02.324661+00:00", + "images": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "image": "alexgshaw/adaptive-rejection-sampler:20251031", + "image_id": "sha256:985bed39e8448bc95f138b5f766b59003a95c4be171c96af8e76448338f3dd8f", + "repo_digests": [ + "alexgshaw/adaptive-rejection-sampler@sha256:985bed39e8448bc95f138b5f766b59003a95c4be171c96af8e76448338f3dd8f" + ], + "size_bytes": 29731756 + }, + { + "task": "terminal-bench/build-cython-ext", + "image": "alexgshaw/build-cython-ext:20251031", + "image_id": "sha256:3612a38fadb89a96f74a1a951fb0b0af734198fd160571eeaba6401593234594", + "repo_digests": [ + "alexgshaw/build-cython-ext@sha256:3612a38fadb89a96f74a1a951fb0b0af734198fd160571eeaba6401593234594" + ], + "size_bytes": 303222492 + }, + { + "task": "terminal-bench/cancel-async-tasks", + "image": "alexgshaw/cancel-async-tasks:20251031", + "image_id": "sha256:84c7fae6b256dcc56a350790e2a9715eefc7dad662a9d8e8a472363aa71ef18d", + "repo_digests": [ + "alexgshaw/cancel-async-tasks@sha256:84c7fae6b256dcc56a350790e2a9715eefc7dad662a9d8e8a472363aa71ef18d" + ], + "size_bytes": 44171676 + }, + { + "task": "terminal-bench/cobol-modernization", + "image": "alexgshaw/cobol-modernization:20251031", + "image_id": "sha256:593ab9df3d83f771e927888b2d9436b380cdf7ec4d0af2a9f0802d541edebba0", + "repo_digests": [ + "alexgshaw/cobol-modernization@sha256:593ab9df3d83f771e927888b2d9436b380cdf7ec4d0af2a9f0802d541edebba0" + ], + "size_bytes": 162020401 + }, + { + "task": "terminal-bench/db-wal-recovery", + "image": "alexgshaw/db-wal-recovery:20251031", + "image_id": "sha256:0e33ea5ec823975d1bd6c3778395c9f94251dd88f571146057bff6adb7e4594e", + "repo_digests": [ + "alexgshaw/db-wal-recovery@sha256:0e33ea5ec823975d1bd6c3778395c9f94251dd88f571146057bff6adb7e4594e" + ], + "size_bytes": 175276145 + }, + { + "task": "terminal-bench/fix-code-vulnerability", + "image": "alexgshaw/fix-code-vulnerability:20251031", + "image_id": "sha256:cac325252991f823713b2d0441502972901dd782bd67f66c03d9b1e410dac5c0", + "repo_digests": [ + "alexgshaw/fix-code-vulnerability@sha256:cac325252991f823713b2d0441502972901dd782bd67f66c03d9b1e410dac5c0" + ], + "size_bytes": 190696453 + }, + { + "task": "terminal-bench/git-multibranch", + "image": "alexgshaw/git-multibranch:20251031", + "image_id": "sha256:5bc449921c138295f6ddd8dca2977319fee2ce523811dbe56cec4ca663385fb5", + "repo_digests": [ + "alexgshaw/git-multibranch@sha256:5bc449921c138295f6ddd8dca2977319fee2ce523811dbe56cec4ca663385fb5" + ], + "size_bytes": 222134454 + }, + { + "task": "terminal-bench/kv-store-grpc", + "image": "alexgshaw/kv-store-grpc:20251031", + "image_id": "sha256:3399400800dcb207634daa42bc1b052e831e285cc9d221eea66c47bc0fc79791", + "repo_digests": [ + "alexgshaw/kv-store-grpc@sha256:3399400800dcb207634daa42bc1b052e831e285cc9d221eea66c47bc0fc79791" + ], + "size_bytes": 44171671 + }, + { + "task": "terminal-bench/log-summary-date-ranges", + "image": "alexgshaw/log-summary-date-ranges:20251031", + "image_id": "sha256:cbeb6ba905c2fec294f16cd5e16e3ea7f2e04d38ac2484d51a11de262aa7dc51", + "repo_digests": [ + "alexgshaw/log-summary-date-ranges@sha256:cbeb6ba905c2fec294f16cd5e16e3ea7f2e04d38ac2484d51a11de262aa7dc51" + ], + "size_bytes": 44866158 + }, + { + "task": "terminal-bench/merge-diff-arc-agi-task", + "image": "alexgshaw/merge-diff-arc-agi-task:20251031", + "image_id": "sha256:bfc2a235f2ceea64ffb1ffc5529c2acebf570bff61f3668f0e5669ef570c17d4", + "repo_digests": [ + "alexgshaw/merge-diff-arc-agi-task@sha256:bfc2a235f2ceea64ffb1ffc5529c2acebf570bff61f3668f0e5669ef570c17d4" + ], + "size_bytes": 29728285 + }, + { + "task": "terminal-bench/model-extraction-relu-logits", + "image": "alexgshaw/model-extraction-relu-logits:20251031", + "image_id": "sha256:52fe1f089f38650f0dc22d7531ee6e01ebd526de1b349aaa2ecb331dca9fabce", + "repo_digests": [ + "alexgshaw/model-extraction-relu-logits@sha256:52fe1f089f38650f0dc22d7531ee6e01ebd526de1b349aaa2ecb331dca9fabce" + ], + "size_bytes": 83698971 + }, + { + "task": "terminal-bench/multi-source-data-merger", + "image": "alexgshaw/multi-source-data-merger:20251031", + "image_id": "sha256:8b32782078ff7383a1b4e5d3cecca8ce287e30f50bb4c0e5db009a18064e666e", + "repo_digests": [ + "alexgshaw/multi-source-data-merger@sha256:8b32782078ff7383a1b4e5d3cecca8ce287e30f50bb4c0e5db009a18064e666e" + ], + "size_bytes": 220880128 + }, + { + "task": "terminal-bench/nginx-request-logging", + "image": "alexgshaw/nginx-request-logging:20251031", + "image_id": "sha256:693e64431a0dbf45952e8b91a298d54fd60247e501c6623fb8fa3ebc07fc3d3d", + "repo_digests": [ + "alexgshaw/nginx-request-logging@sha256:693e64431a0dbf45952e8b91a298d54fd60247e501c6623fb8fa3ebc07fc3d3d" + ], + "size_bytes": 67285778 + }, + { + "task": "terminal-bench/openssl-selfsigned-cert", + "image": "alexgshaw/openssl-selfsigned-cert:20251031", + "image_id": "sha256:4c948a4e630af2435ae0a19108fc0814a946ac2fa29a512469e0fc77b38c8c12", + "repo_digests": [ + "alexgshaw/openssl-selfsigned-cert@sha256:4c948a4e630af2435ae0a19108fc0814a946ac2fa29a512469e0fc77b38c8c12" + ], + "size_bytes": 44173158 + }, + { + "task": "terminal-bench/pypi-server", + "image": "alexgshaw/pypi-server:20251031", + "image_id": "sha256:d18bb30f47c7dcaa3acdbc31ac7413c98eadca2ea16540bbd380f2f063b95276", + "repo_digests": [ + "alexgshaw/pypi-server@sha256:d18bb30f47c7dcaa3acdbc31ac7413c98eadca2ea16540bbd380f2f063b95276" + ], + "size_bytes": 83726049 + }, + { + "task": "terminal-bench/pytorch-model-cli", + "image": "alexgshaw/pytorch-model-cli:20251031", + "image_id": "sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "repo_digests": [ + "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67" + ], + "size_bytes": 44349611 + }, + { + "task": "terminal-bench/pytorch-model-recovery", + "image": "alexgshaw/pytorch-model-recovery:20260430", + "image_id": "sha256:7c2bb52851cef25bdc3939e473496d25fbf11fd0cb7ab58bcf3cefcb751cdc0e", + "repo_digests": [ + "alexgshaw/pytorch-model-recovery@sha256:7c2bb52851cef25bdc3939e473496d25fbf11fd0cb7ab58bcf3cefcb751cdc0e" + ], + "size_bytes": 269661811 + }, + { + "task": "terminal-bench/qemu-alpine-ssh", + "image": "alexgshaw/qemu-alpine-ssh:20251031", + "image_id": "sha256:541e8214a8c4faa7b556531b899d45da21048fa2ca6dd328d8cb6c4cffa8e964", + "repo_digests": [ + "alexgshaw/qemu-alpine-ssh@sha256:541e8214a8c4faa7b556531b899d45da21048fa2ca6dd328d8cb6c4cffa8e964" + ], + "size_bytes": 1308553944 + }, + { + "task": "terminal-bench/query-optimize", + "image": "alexgshaw/query-optimize:20251031", + "image_id": "sha256:28de7da87004eab5b29bd5eaf1823579bc5b73ccea4b80349a5623185770da14", + "repo_digests": [ + "alexgshaw/query-optimize@sha256:28de7da87004eab5b29bd5eaf1823579bc5b73ccea4b80349a5623185770da14" + ], + "size_bytes": 120494963 + }, + { + "task": "terminal-bench/regex-log", + "image": "alexgshaw/regex-log:20251031", + "image_id": "sha256:90101b2e815323a8da20528a1439bebc407eb9761c9c68a3d557730856c878e9", + "repo_digests": [ + "alexgshaw/regex-log@sha256:90101b2e815323a8da20528a1439bebc407eb9761c9c68a3d557730856c878e9" + ], + "size_bytes": 29725514 + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "image": "alexgshaw/torch-pipeline-parallelism:20251031", + "image_id": "sha256:3cb7b39d86f1704b20e668851da0b08b34b3270c01d583f136d65190984a8d1b", + "repo_digests": [ + "alexgshaw/torch-pipeline-parallelism@sha256:3cb7b39d86f1704b20e668851da0b08b34b3270c01d583f136d65190984a8d1b" + ], + "size_bytes": 29725531 + }, + { + "task": "terminal-bench/torch-tensor-parallelism", + "image": "alexgshaw/torch-tensor-parallelism:20251031", + "image_id": "sha256:c50ac169e11465f41b49749fe87e4487f24d31a0c39c0906cd3242cc5499c690", + "repo_digests": [ + "alexgshaw/torch-tensor-parallelism@sha256:c50ac169e11465f41b49749fe87e4487f24d31a0c39c0906cd3242cc5499c690" + ], + "size_bytes": 29725529 + }, + { + "task": "terminal-bench/write-compressor", + "image": "alexgshaw/write-compressor:20251031", + "image_id": "sha256:3618e1f8a997b437c09cd3dbaac736705809c05f6f12f584b65722175970ebe1", + "repo_digests": [ + "alexgshaw/write-compressor@sha256:3618e1f8a997b437c09cd3dbaac736705809c05f6f12f584b65722175970ebe1" + ], + "size_bytes": 291927726 + }, + { + "task": "terminal-bench/constraints-scheduling", + "image": "alexgshaw/constraints-scheduling:20251031", + "image_id": "sha256:567ce5a189f8d11ac461790876e934cc7af38391baf89a78f95f4dafc1fec3b0", + "repo_digests": [ + "alexgshaw/constraints-scheduling@sha256:567ce5a189f8d11ac461790876e934cc7af38391baf89a78f95f4dafc1fec3b0" + ], + "size_bytes": 49852952 + }, + { + "task": "terminal-bench/custom-memory-heap-crash", + "image": "alexgshaw/custom-memory-heap-crash:20251031", + "image_id": "sha256:1895d7d6240d5fb2ec774d6bfdd1057d170f8d199c15c582e4585d3c63802557", + "repo_digests": [ + "alexgshaw/custom-memory-heap-crash@sha256:1895d7d6240d5fb2ec774d6bfdd1057d170f8d199c15c582e4585d3c63802557" + ], + "size_bytes": 1267999268 + }, + { + "task": "terminal-bench/dna-assembly", + "image": "alexgshaw/dna-assembly:20251031", + "image_id": "sha256:d1adf6835f1dd91205ba70e452c699d0aea601010038e5617f370716efb50569", + "repo_digests": [ + "alexgshaw/dna-assembly@sha256:d1adf6835f1dd91205ba70e452c699d0aea601010038e5617f370716efb50569" + ], + "size_bytes": 29727501 + }, + { + "task": "terminal-bench/llm-inference-batching-scheduler", + "image": "alexgshaw/llm-inference-batching-scheduler:20251031", + "image_id": "sha256:19e79aa49be4e55dccca1dde966515ebe6852314b6010bcb9230afb48b996774", + "repo_digests": [ + "alexgshaw/llm-inference-batching-scheduler@sha256:19e79aa49be4e55dccca1dde966515ebe6852314b6010bcb9230afb48b996774" + ], + "size_bytes": 44188631 + }, + { + "task": "terminal-bench/path-tracing", + "image": "alexgshaw/path-tracing:20251031", + "image_id": "sha256:4b44725102cd627dfa895191a5a30d16dae3c5623bb43da6459f79135db6b482", + "repo_digests": [ + "alexgshaw/path-tracing@sha256:4b44725102cd627dfa895191a5a30d16dae3c5623bb43da6459f79135db6b482" + ], + "size_bytes": 399390468 + }, + { + "task": "terminal-bench/schemelike-metacircular-eval", + "image": "alexgshaw/schemelike-metacircular-eval:20251031", + "image_id": "sha256:b485be38bf80d56c1a4cd9da95bbf7ec129dc03f6786b86c0525518a96f6d1db", + "repo_digests": [ + "alexgshaw/schemelike-metacircular-eval@sha256:b485be38bf80d56c1a4cd9da95bbf7ec129dc03f6786b86c0525518a96f6d1db" + ], + "size_bytes": 44188336 + }, + { + "task": "terminal-bench/caffe-cifar-10", + "image": "alexgshaw/caffe-cifar-10:20260403", + "image_id": "sha256:929a6d631b592c82e0d0f5d4a65cf03c946dc554cfc9bb09916e82076e7d3215", + "repo_digests": [ + "alexgshaw/caffe-cifar-10@sha256:929a6d631b592c82e0d0f5d4a65cf03c946dc554cfc9bb09916e82076e7d3215" + ], + "size_bytes": 213171982 + }, + { + "task": "terminal-bench/circuit-fibsqrt", + "image": "alexgshaw/circuit-fibsqrt:20251031", + "image_id": "sha256:29783439f529eaed2145592f15af7a2281161528860392a382c825489030ae3a", + "repo_digests": [ + "alexgshaw/circuit-fibsqrt@sha256:29783439f529eaed2145592f15af7a2281161528860392a382c825489030ae3a" + ], + "size_bytes": 149136738 + }, + { + "task": "terminal-bench/mteb-leaderboard", + "image": "alexgshaw/mteb-leaderboard:20260430", + "image_id": "sha256:a8538f4882ff132e0a20a75be3fffc07164318a7425ad19a6b4bf5268107feb4", + "repo_digests": [ + "alexgshaw/mteb-leaderboard@sha256:a8538f4882ff132e0a20a75be3fffc07164318a7425ad19a6b4bf5268107feb4" + ], + "size_bytes": 663825427 + }, + { + "task": "terminal-bench/regex-chess", + "image": "alexgshaw/regex-chess:20251031", + "image_id": "sha256:e3b6d61d2d3930fbbdc5f7bf091a7f8b7f6125e1951ef8a7dbe165b1ef62942c", + "repo_digests": [ + "alexgshaw/regex-chess@sha256:e3b6d61d2d3930fbbdc5f7bf091a7f8b7f6125e1951ef8a7dbe165b1ef62942c" + ], + "size_bytes": 55625584 + }, + { + "task": "terminal-bench/build-pov-ray", + "image": "alexgshaw/build-pov-ray:20251031", + "image_id": "sha256:874baf49341e2853a15a9184e308c7eb632e97f434359fcff2891c6e5ca1b39b", + "repo_digests": [ + "alexgshaw/build-pov-ray@sha256:874baf49341e2853a15a9184e308c7eb632e97f434359fcff2891c6e5ca1b39b" + ], + "size_bytes": 163066497 + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/process-supervision-probe-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/process-supervision-probe-evidence.json new file mode 100644 index 0000000..0653f45 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/process-supervision-probe-evidence.json @@ -0,0 +1,13 @@ +{ + "observed_at": "2026-10-29T14:58:28.080840+00:00", + "kind": "isolated network-disabled Docker probe; a tool forks a new session and writes repeatedly", + "runner_sha256": "171c880501bdfc7ea4e85157daa6f8d3b48b54d66e10a7e97f292b6bc2e34cd4", + "base_image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "no_model_calls": true, + "state": { + "status": "cancelled", + "processes_stopped": true, + "stopped_processes": 2 + }, + "output_frozen": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pypi-server.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pypi-server.json new file mode 100644 index 0000000..4c292ab --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pypi-server.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-cli.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-cli.json new file mode 100644 index 0000000..1490c16 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-cli.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-recovery.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-recovery.json new file mode 100644 index 0000000..28cdb6b --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pytorch-model-recovery.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/qemu-alpine-ssh.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/qemu-alpine-ssh.json new file mode 100644 index 0000000..65b8e75 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/qemu-alpine-ssh.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/query-optimize.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/query-optimize.json new file mode 100644 index 0000000..1e7a3f7 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/query-optimize.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-chess.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-chess.json new file mode 100644 index 0000000..5bd156e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-chess.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-log.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-log.json new file mode 100644 index 0000000..3281d02 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/regex-log.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/review-fixes-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/review-fixes-evidence.json new file mode 100644 index 0000000..55bc048 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/review-fixes-evidence.json @@ -0,0 +1,68 @@ +{ + "schema_version": 1, + "checked_at": "2026-10-30T14:55:28.216309+00:00", + "review_target": "PR #54 at 0168e32fc9c976aa18cc49d7fd680399b0e5da6d", + "findings": [ + { + "priority": 2, + "finding": "Pi can warn and fall back to built-in models while exiting zero.", + "fix": "Reject model-loading diagnostics and require the exact requested provider/model in the offline listing." + }, + { + "priority": 2, + "finding": "Multiple Pi agents can overwrite a shared job-local model snapshot.", + "fix": "Preserve distinct immutable configuration files and record each model_config_file plus its SHA256." + }, + { + "priority": 2, + "finding": "Failed streams can retain initialized zero usage and appear complete after native retry success.", + "fix": "Diagnose zero-usage failed assistant messages as unknown, keep known subtotals, and mark usage/cost partial." + }, + { + "priority": 2, + "finding": "Concurrent first publication of a shared distribution can fail on rename or staging-name collision.", + "fix": "Use unique staging directories, atomically publish, verify and reuse a concurrent winner, reject corrupt bytes, and clean temporary directories." + } + ], + "validation": { + "full_harbor_tests_passed": 238, + "pi_tests_passed": 26, + "old_source_regression_failures": 9, + "old_source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "old_source_regression_command": "PYTHONDONTWRITEBYTECODE=1 PYTHONPATH=jobs/supervised-source/benchmarks/harbor/src jobs/harbor-supervised-runtime-venv/bin/python -B -m pytest benchmarks/harbor/tests/test_pi.py -q --tb=no -p no:cacheprovider -k \"model_fallback_or_missing_registration or preserves_distinct_model_settings or failed_stream_zero_usage or concurrent_distribution_publication\"", + "full_suite_command": "PYTHONPATH=benchmarks/harbor/src jobs/harbor-supervised-runtime-venv/bin/python -m pytest benchmarks/harbor/tests -q", + "pinned_pi_1_4_0_offline_probe": [ + { + "configuration": "registered", + "pi_exit_code": 0, + "accepted": true + }, + { + "configuration": "invalid maxTokens string", + "pi_exit_code": 0, + "accepted": false + } + ], + "model_calls": 0 + }, + "archived_baseline_checks": { + "model_jobs": 34, + "agents_per_model_job": 1, + "public_model_config_sha256": "8751fac554abb9f363b188c102045f38e36693d52956f984e0e427b31998b299", + "all_archived_public_model_config_hashes_match": true, + "model_loading_warning_tasks": [], + "failed_zero_usage_event_tasks": [], + "frozen_files_rehashed": 1449, + "freeze_sha256": "76b775d898bbb84b21f80eee1061806d7d949465f9eef60dcab6dedeb326734b", + "results_sha256": "b85d3bf2b0ea9661940f0e6b08e3bea71b2c41ae34da5c89f9ecd5197c6c8ff5", + "results_unchanged": true, + "native_outputs_or_frozen_sources_modified": false, + "model_trials_replayed": 0 + }, + "changed_source_sha256": { + "benchmarks/harbor/src/harbor_adapter/pi.py": "e703174d76a1b670466ba06904db6b08a26769d42ac974274f61423ff0778fd6", + "benchmarks/harbor/src/harbor_adapter/__main__.py": "06f230f53a74bbafdb4958d6649cb5e277d6a7e48ae7ee003c6967717c723000", + "benchmarks/harbor/tests/test_pi.py": "31a454c063b3f6f3c74de7816aa854a3f7a625f5e2c18da08857086d47df512b" + }, + "interpretation": "These are post-run workflow fixes. Original source pins, raw outcomes, usage, exceptions, and analytical baseline results remain unchanged. No reviewed defect was observed to change the archived model-job results. Existing timeout integrity exceptions and answer/credential exposure caveats still apply." +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/schemelike-metacircular-eval.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/schemelike-metacircular-eval.json new file mode 100644 index 0000000..c024b76 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/schemelike-metacircular-eval.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json new file mode 100644 index 0000000..e4bf64e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/selection.json @@ -0,0 +1,150 @@ +{ + "schema_version": 1, + "selected_at": "2026-10-12T13:52:35.779627+00:00", + "source_ref": "2eb99b608dfb39464d17d109a4a8f11afda607a9", + "dataset_ref": "sha256:7d7bdc1cbedad549fc1140404bd4dc45e5fd0ea7c4186773687d177ad3a0699a", + "catalog_sha256": "7fc7d95a2077ede76c4f47890a09de93c2fe558252a122ef830288ad293ceaaa", + "selection_method": "Entire existing 34-task development pool, explicitly requested by the user. No selection by latest score and no outcome-based replacements.", + "prior_selection_file": "../dev12-baseline-20261011/selection.json", + "prior_selection_sha256": "cd51acea6592c01ce9ec8cde34cd46a4965a4164950194f2de7e8ca54649bd29", + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12" + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + }, + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1" + }, + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88" + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + }, + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + }, + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71" + }, + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + }, + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + }, + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + }, + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + }, + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + }, + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + }, + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + }, + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + }, + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + }, + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + }, + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + }, + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + } + ], + "prior_exposure": "All tasks are development tasks with previous outcomes and some reviewed traces/tests. This is not a held-out capability evaluation. No fresh trace or hidden-test content is reviewed until every new agent output is frozen.", + "remaining_unopened_tasks": 55 +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json new file mode 100644 index 0000000..8637077 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/adaptive-rejection-sampler.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json new file mode 100644 index 0000000..dcb2a04 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-cython-ext.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json new file mode 100644 index 0000000..acdb83d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/build-pov-ray.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json new file mode 100644 index 0000000..dabaa81 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/caffe-cifar-10.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json new file mode 100644 index 0000000..bab191a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cancel-async-tasks.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json new file mode 100644 index 0000000..727dcca --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/circuit-fibsqrt.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json new file mode 100644 index 0000000..f809971 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/cobol-modernization.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json new file mode 100644 index 0000000..ae6ccbc --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/constraints-scheduling.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json new file mode 100644 index 0000000..6272beb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/custom-memory-heap-crash.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json new file mode 100644 index 0000000..1d5efa6 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/db-wal-recovery.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json new file mode 100644 index 0000000..4c3c004 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json @@ -0,0 +1,100 @@ +{ + "services": { + "main": { + "volumes": [ + { + "type": "bind", + "source": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/apt-cache.conf", + "target": "/etc/apt/apt.conf.d/99nanopy-index-cache", + "read_only": true + } + ], + "environment": { + "UV_INSTALLER_GITHUB_BASE_URL": "http://127.0.0.1:18743", + "UV_PYTHON_INSTALL_MIRROR": "http://127.0.0.1:18743/astral-sh/python-build-standalone/releases/download" + } + }, + "apt-index-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/apt_index_proxy.py", + "--bind", + "127.0.0.1", + "--port", + "18742", + "--cache", + "/cache", + "--workers", + "16" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "mem_limit": "256m", + "cpus": 0.5, + "volumes": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/apt_index_proxy.py:/proxy/apt_index_proxy.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/home/minix/projects/nanoPyCodeAgent/.cache/nanopy-harbor/apt-indexes:/cache" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18742), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + }, + "uv-release-cache": { + "image": "alexgshaw/pytorch-model-cli@sha256:c996884b5fb5fb2c0c07d67da2c4f974f271f07234ed28e1a868240375c57a67", + "network_mode": "service:main", + "entrypoint": [ + "python3", + "/proxy/uv_release_cache.py", + "serve", + "--cache", + "/cache", + "--manifest", + "/manifest.json" + ], + "user": "1000:1000", + "read_only": true, + "cap_drop": [ + "ALL" + ], + "security_opt": [ + "no-new-privileges:true" + ], + "cpus": 0.5, + "mem_limit": "256m", + "volumes": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/uv_release_cache.py:/proxy/uv_release_cache.py:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/scripts/cache_image.py:/proxy/cache_image.py:ro", + "/home/minix/projects/nanoPyCodeAgent/.cache/nanopy-harbor/uv-releases:/cache:ro", + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-release-manifest.json:/manifest.json:ro" + ], + "healthcheck": { + "test": [ + "CMD", + "python3", + "-c", + "import socket; socket.create_connection((\"127.0.0.1\",18743), timeout=2).close()" + ], + "interval": "1s", + "timeout": "3s", + "retries": 30 + } + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json new file mode 100644 index 0000000..ffd0aa7 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dna-assembly.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json new file mode 100644 index 0000000..869353d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/fix-code-vulnerability.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json new file mode 100644 index 0000000..70eb0c0 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/git-multibranch.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json new file mode 100644 index 0000000..abd0da3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-01.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/adaptive-rejection-sampler", + "ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de" + }, + { + "name": "terminal-bench/build-cython-ext", + "ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json new file mode 100644 index 0000000..ff43bed --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-02.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/cancel-async-tasks", + "ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1" + }, + { + "name": "terminal-bench/cobol-modernization", + "ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json new file mode 100644 index 0000000..c38f6a3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-03.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/db-wal-recovery", + "ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4" + }, + { + "name": "terminal-bench/fix-code-vulnerability", + "ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json new file mode 100644 index 0000000..b969d06 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-04.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json new file mode 100644 index 0000000..e6c1156 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-05.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json new file mode 100644 index 0000000..9423a5d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-06.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json new file mode 100644 index 0000000..55d0c06 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-07.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json new file mode 100644 index 0000000..cc26925 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-08.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json new file mode 100644 index 0000000..0b0dfd2 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-09.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json new file mode 100644 index 0000000..3109e07 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-10.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json new file mode 100644 index 0000000..f6a87f7 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-11.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json new file mode 100644 index 0000000..4f91e2f --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-12.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json new file mode 100644 index 0000000..13f78b0 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-13.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json new file mode 100644 index 0000000..283dcee --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-14.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json new file mode 100644 index 0000000..065ff3e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-15.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json new file mode 100644 index 0000000..9ed02f2 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-16.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json new file mode 100644 index 0000000..0a7a316 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/install-pair-17.json @@ -0,0 +1,37 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + } + ], + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json new file mode 100644 index 0000000..2271c7d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/kv-store-grpc.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json new file mode 100644 index 0000000..02c1b9a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/llm-inference-batching-scheduler.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json new file mode 100644 index 0000000..161a71c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/log-summary-date-ranges.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json new file mode 100644 index 0000000..910e196 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/merge-diff-arc-agi-task.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json new file mode 100644 index 0000000..3144da3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/model-extraction-relu-logits.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json new file mode 100644 index 0000000..f55d725 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/mteb-leaderboard.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json new file mode 100644 index 0000000..8b212a2 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/multi-source-data-merger.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json new file mode 100644 index 0000000..3eb84c3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/nginx-request-logging.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json new file mode 100644 index 0000000..e349873 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/openssl-selfsigned-cert.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json new file mode 100644 index 0000000..a0da46b --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/path-tracing.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json new file mode 100644 index 0000000..194082e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pypi-server.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json new file mode 100644 index 0000000..a051177 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-cli.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json new file mode 100644 index 0000000..a425853 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/pytorch-model-recovery.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json new file mode 100644 index 0000000..3cce150 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/qemu-alpine-ssh.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json new file mode 100644 index 0000000..1921f0d --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/query-optimize.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json new file mode 100644 index 0000000..fc26f4f --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-chess.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json new file mode 100644 index 0000000..cda11ee --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/regex-log.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json new file mode 100644 index 0000000..78acb45 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/schemelike-metacircular-eval.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json new file mode 100644 index 0000000..d0001bf --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-pipeline-parallelism.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json new file mode 100644 index 0000000..c4e154a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/torch-tensor-parallelism.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json new file mode 100644 index 0000000..976d279 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/write-compressor.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/build-pov-ray.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/build-pov-ray.json new file mode 100644 index 0000000..08ec086 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/build-pov-ray.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/caffe-cifar-10.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/caffe-cifar-10.json new file mode 100644 index 0000000..ba60fcb --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/caffe-cifar-10.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/circuit-fibsqrt.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/circuit-fibsqrt.json new file mode 100644 index 0000000..1def551 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/circuit-fibsqrt.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/constraints-scheduling.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/constraints-scheduling.json new file mode 100644 index 0000000..548c1f4 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/constraints-scheduling.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/custom-memory-heap-crash.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/custom-memory-heap-crash.json new file mode 100644 index 0000000..4b6f6ad --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/custom-memory-heap-crash.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/dna-assembly.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/dna-assembly.json new file mode 100644 index 0000000..e131e31 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/dna-assembly.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/git-multibranch.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/git-multibranch.json new file mode 100644 index 0000000..8381dc3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/git-multibranch.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-01.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-01.json new file mode 100644 index 0000000..4501ef0 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-01.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/git-multibranch", + "ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa" + }, + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-02.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-02.json new file mode 100644 index 0000000..dfdf52e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-02.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + }, + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-03.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-03.json new file mode 100644 index 0000000..c9376f3 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-03.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + }, + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-04.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-04.json new file mode 100644 index 0000000..eec9fea --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-04.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + }, + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-05.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-05.json new file mode 100644 index 0000000..2a5407a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-05.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + }, + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-06.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-06.json new file mode 100644 index 0000000..2cc0304 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-06.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + }, + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-07.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-07.json new file mode 100644 index 0000000..7e0878a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-07.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + }, + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-08.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-08.json new file mode 100644 index 0000000..1ea49a4 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-08.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + }, + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-09.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-09.json new file mode 100644 index 0000000..7bd1273 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-09.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + }, + { + "name": "terminal-bench/constraints-scheduling", + "ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-10.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-10.json new file mode 100644 index 0000000..4d9072a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-10.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/custom-memory-heap-crash", + "ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e" + }, + { + "name": "terminal-bench/dna-assembly", + "ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-11.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-11.json new file mode 100644 index 0000000..4e88bc2 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-11.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + }, + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-12.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-12.json new file mode 100644 index 0000000..b57b02c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-12.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + }, + { + "name": "terminal-bench/caffe-cifar-10", + "ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-13.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-13.json new file mode 100644 index 0000000..246b110 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-13.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/circuit-fibsqrt", + "ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3" + }, + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-14.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-14.json new file mode 100644 index 0000000..a130477 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/install-pair-14.json @@ -0,0 +1,44 @@ +{ + "n_concurrent_trials": 2, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + }, + { + "name": "terminal-bench/build-pov-ray", + "ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + }, + "install_only": true +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/kv-store-grpc.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/kv-store-grpc.json new file mode 100644 index 0000000..02fc1a4 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/kv-store-grpc.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/kv-store-grpc", + "ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/llm-inference-batching-scheduler.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/llm-inference-batching-scheduler.json new file mode 100644 index 0000000..e54f30b --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/llm-inference-batching-scheduler.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/llm-inference-batching-scheduler", + "ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/log-summary-date-ranges.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/log-summary-date-ranges.json new file mode 100644 index 0000000..3a2180a --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/log-summary-date-ranges.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/log-summary-date-ranges", + "ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/merge-diff-arc-agi-task.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/merge-diff-arc-agi-task.json new file mode 100644 index 0000000..7b72608 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/merge-diff-arc-agi-task.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/merge-diff-arc-agi-task", + "ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/model-extraction-relu-logits.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/model-extraction-relu-logits.json new file mode 100644 index 0000000..de66f6e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/model-extraction-relu-logits.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/model-extraction-relu-logits", + "ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/mteb-leaderboard.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/mteb-leaderboard.json new file mode 100644 index 0000000..7350550 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/mteb-leaderboard.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/mteb-leaderboard", + "ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/multi-source-data-merger.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/multi-source-data-merger.json new file mode 100644 index 0000000..808c063 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/multi-source-data-merger.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/multi-source-data-merger", + "ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/nginx-request-logging.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/nginx-request-logging.json new file mode 100644 index 0000000..25f4789 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/nginx-request-logging.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/nginx-request-logging", + "ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/openssl-selfsigned-cert.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/openssl-selfsigned-cert.json new file mode 100644 index 0000000..64db233 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/openssl-selfsigned-cert.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/openssl-selfsigned-cert", + "ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/path-tracing.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/path-tracing.json new file mode 100644 index 0000000..1b93bea --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/path-tracing.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/path-tracing", + "ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pypi-server.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pypi-server.json new file mode 100644 index 0000000..710cba1 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pypi-server.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pypi-server", + "ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-cli.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-cli.json new file mode 100644 index 0000000..72cf04c --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-cli.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-cli", + "ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-recovery.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-recovery.json new file mode 100644 index 0000000..833b4d5 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/pytorch-model-recovery.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/pytorch-model-recovery", + "ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/qemu-alpine-ssh.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/qemu-alpine-ssh.json new file mode 100644 index 0000000..48afb01 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/qemu-alpine-ssh.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/qemu-alpine-ssh", + "ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/query-optimize.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/query-optimize.json new file mode 100644 index 0000000..2a29b82 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/query-optimize.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/query-optimize", + "ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-chess.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-chess.json new file mode 100644 index 0000000..5d963be --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-chess.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-chess", + "ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-log.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-log.json new file mode 100644 index 0000000..e1b548e --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/regex-log.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/regex-log", + "ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/schemelike-metacircular-eval.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/schemelike-metacircular-eval.json new file mode 100644 index 0000000..0afc971 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/schemelike-metacircular-eval.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/schemelike-metacircular-eval", + "ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-pipeline-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-pipeline-parallelism.json new file mode 100644 index 0000000..f5f3f57 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-pipeline-parallelism.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-tensor-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-tensor-parallelism.json new file mode 100644 index 0000000..586e163 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/torch-tensor-parallelism.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/write-compressor.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/write-compressor.json new file mode 100644 index 0000000..df52a53 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/supervision-v7/write-compressor.json @@ -0,0 +1,39 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/mnt/ubuntu/home/omarchy_data/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/storage-v2/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + } + ], + "verifier": { + "import_path": "harbor_adapter:RetryingVerifier", + "kwargs": { + "max_attempts": 3, + "require_agent_quiescence": true + } + } +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/timeout-boundary-evidence.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/timeout-boundary-evidence.json new file mode 100644 index 0000000..07e9a3f --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/timeout-boundary-evidence.json @@ -0,0 +1,245 @@ +{ + "observed_at": "2026-10-29T14:57:26.714324+00:00", + "policy": "Preserve every original output and native reward. These timeout trials do not establish a verified frozen boundary. Metadata only was inspected; no solution or hidden-test content was reviewed. No completed model trial is retried.", + "cases": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "exception_type": "AgentTimeoutError", + "agent_execution": { + "started_at": "2026-10-29T13:54:34.019472Z", + "finished_at": "2026-10-29T14:09:35.305831Z" + }, + "verifier": { + "started_at": "2026-10-29T14:09:35.589907Z", + "finished_at": "2026-10-29T14:16:29.067351Z" + }, + "native_rewards": { + "reward": 0.0 + }, + "result_sha256": "54f5f469df311a50cc463503f529e0ad7188823891ab31840965ef4c78bfef97", + "pi_events_sha256": "70022bc1980da1fef1d93a9105d5b7da64753d01d526d93cd32184b677e5ec8b", + "post_agent_deadline_metadata": [ + { + "timestamp": "2026-10-29T14:09:52.609000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:09:52.615000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:09:57.440000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:09:57.446000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:17.261000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:17.270000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:21.173000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:21.194000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:25.093000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:54.575000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:57.583000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:11:57.590000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:01.777000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:01.798000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:56.451000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:56.480000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:59.037000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:59.041000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:13:09.546000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:13:55.368000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:13:57.962000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:14:57.538000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:15:01.266000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:15:01.271000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:15:08.602000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:16:09.197000+00:00", + "role": "toolResult", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:16:12.110000+00:00", + "role": "assistant", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:09:52.618000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:09:57.455000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:17.274000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:21.196000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:10:54.577000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:11:57.592000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:01.799000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:56.483000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:12:59.043000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:13:55.371000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:14:57.540000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:15:01.273000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + }, + { + "timestamp": "2026-10-29T14:16:09.201000+00:00", + "role": "assistant_creation", + "after_verifier_started": true + } + ], + "analytical_status": "invalid_frozen_boundary" + }, + { + "task": "terminal-bench/db-wal-recovery", + "exception_type": "AgentTimeoutError", + "agent_execution": { + "started_at": "2026-10-29T14:26:49.171477Z", + "finished_at": "2026-10-29T14:41:47.862235Z" + }, + "verifier": { + "started_at": "2026-10-29T14:41:48.136670Z", + "finished_at": "2026-10-29T14:42:23.786394Z" + }, + "native_rewards": { + "reward": 0.0 + }, + "result_sha256": "1318b9f55b8ca84b5c8067ba5a36ee2d7b953acb550f3e01062d2612090a0755", + "pi_events_sha256": "fb60c26f364cb61eff8742cf53feb87feb9743f73c4afdb06e3314b343aed3cf", + "post_agent_deadline_metadata": [], + "analytical_status": "timeout_without_verified_quiescence" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-pipeline-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-pipeline-parallelism.json new file mode 100644 index 0000000..17a2cfc --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-pipeline-parallelism.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-pipeline-parallelism", + "ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-tensor-parallelism.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-tensor-parallelism.json new file mode 100644 index 0000000..6a9b720 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/torch-tensor-parallelism.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/torch-tensor-parallelism", + "ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684" + } + ] +} diff --git a/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/write-compressor.json b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/write-compressor.json new file mode 100644 index 0000000..a442618 --- /dev/null +++ b/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/write-compressor.json @@ -0,0 +1,32 @@ +{ + "n_concurrent_trials": 1, + "n_attempts": 1, + "retry": { + "max_retries": 0 + }, + "agents": [ + { + "name": "harbor_adapter:PiCodeAgent", + "model_name": "deepseek/deepseek-flash", + "override_setup_timeout_sec": 1800, + "kwargs": { + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "model_config_path": "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/pi-models.json", + "thinking": "high" + } + } + ], + "environment": { + "type": "docker", + "extra_docker_compose": [ + "/home/minix/projects/nanoPyCodeAgent/jobs/pi-deepseek-flash-dev34-20261024/benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/dependency-cache.compose.json" + ] + }, + "tasks": [ + { + "name": "terminal-bench/write-compressor", + "ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e" + } + ] +} diff --git a/benchmarks/harbor/reports/en/pi-deepseek-flash-dev34-20261024.md b/benchmarks/harbor/reports/en/pi-deepseek-flash-dev34-20261024.md new file mode 100644 index 0000000..95b8508 --- /dev/null +++ b/benchmarks/harbor/reports/en/pi-deepseek-flash-dev34-20261024.md @@ -0,0 +1,252 @@ +# Pi + DeepSeek Flash: 34-task development baseline + +All **34 tasks received one fresh model attempt** and finished on October 29, +2026. The audited outcomes are **26 official passes, six scored failures, and +two unscored legacy timeout attempts**. One of the 26 passes followed confirmed +agent cancellation at its time limit; another retrieved public reference tests +and a solution. This run does **not** establish a complete, uncontaminated +34-task pass rate. + +The unchanged standard summaries contain 25 ordinary passes, six ordinary +failures, and three `AgentTimeoutError` execution errors. Native raw rewards are +26 ones and eight zeros. The audit accepts the supervised Caffe timeout's raw +one and leaves the two legacy timeout zeros analytically null. It never +rewrites exceptions, changes native rewards, or replaces model outputs. + +The [audited results](../../results/pi-deepseek-flash-dev34-20261024.json) contain +all trial IDs, source/task/configuration provenance, raw and analytical outcomes, +official test counts, usage, timing, audit flags, and evidence hashes. The +[v7 registration](../../configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json) preserves the six original +attempts and registers the remaining 28 under corrected process supervision. +All outputs were hashed at **2026-10-30 13:54:30.827662 UTC**, before new solution +trajectory or hidden-test-content review; the +[freeze manifest](../../configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json) covers **1,449 files**. + +## Agent, model, and protocol + +| Property | Value | +| --- | --- | +| Pool | The exact October 12 development selection: 34 tasks with identical task revisions; the remaining 55 packages stayed unopened | +| Agent | Unmodified upstream Pi 1.4.0 local binary distribution; all files and permissions pinned; upstream archive checksum not independently verified | +| Model | `deepseek/deepseek-flash`, official `https://api.deepseek.com` OpenAI-compatible endpoint | +| Alias | Official documentation checked October 24 and 27 mapped it to DeepSeek-V4.1-Flash-1017; mutable alias, not an immutable checkpoint | +| Reasoning and sampling | `high`; temperature unset | +| Agent behavior | Stock prompt, tools, compaction, and request retry behavior; personal settings, extensions, skills, history, and other providers excluded | +| Model metadata | Context 1,000,000; output ceiling 384,000; native Pi generation behavior | +| Limits | Original task-native agent timeout and resources; separate 1,800-second setup limit | +| Queue | Ascending native timeout then name; consecutive pair barriers; concurrency two; one output per task | +| Verifier | Original official scripts and per-attempt limits; timeout-only retries permitted up to three attempts on the same output; zero used | +| Installation | Original 34 checks passed; 28 additional checks/probes passed after the supervision amendment | + +QEMU's registered `qemu-alpine-ssh-v1` dependency profile passed under both +sources. The other 33 task profiles explicitly remain `not_configured`; +installation success does not certify unsupported verifier dependencies. +Verified caches preserve original package names, sizes, SHA256 digests, sources, +and task images. Original image identities and repository digests were verified +before each batch. Only newly introduced image references were evicted without +force; protected images and nine older stopped recovery containers were retained. + +```text +plan SHA256: d90107ed15e47981fb3a09e74eb5d3b0e4f72acc96b30308c6d66ec28151128c +freeze SHA256: 76b775d898bbb84b21f80eee1061806d7d949465f9eef60dcab6dedeb326734b +results SHA256: b85d3bf2b0ea9661940f0e6b08e3bea71b2c41ae34da5c89f9ecd5197c6c8ff5 +selection SHA256: 4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85 +Pi distribution SHA256: 60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9 +first six adapter source: d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67 +remaining 28 adapter source: 29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d +``` + +The historical Nano baseline used 100 replies, a 65,536-token output request, +native timeout minus 180 seconds for work, an Anthropic-compatible endpoint, and +unset reasoning effort. Pi used its native loop, the full task-native timeout, +an OpenAI-compatible endpoint, and `high` reasoning. Different dates, a mutable +model alias, and these protocol differences prevent a causal agent-quality or +checkpoint comparison. There was no agent/prompt optimization between tasks. + +## Process-supervision amendment and timeout accounting + +The first six attempts used `d6ed70a`. Three passed and one failed normally. +`adaptive-rejection-sampler` and `db-wal-recovery` timed out. Operational +timestamps exposed Pi activity during sampler verification, so the controller +stopped at the completed-pair boundary before task seven. The 28 unstarted +tasks moved to `29db8d7` after **225 Harbor tests**, a network-disabled Docker +cancellation probe, and 28 fresh installation/supervisor checks. Pi bytes, +model settings, prompt/tools, task order/revisions, and budgets were unchanged. +See the [configuration history](../../configs/pi-deepseek-flash-dev34-20261024/README.md), +[original-six evidence](../../configs/pi-deepseek-flash-dev34-20261024/model-prefix-evidence.json), +[timeout evidence](../../configs/pi-deepseek-flash-dev34-20261024/timeout-boundary-evidence.json), and +[Docker probe](../../configs/pi-deepseek-flash-dev34-20261024/process-supervision-probe-evidence.json). + +- **Sampler:** 40 message metadata records occurred after the reported agent + end, including during verification. The verifier also encountered an APT lock + and a `scipy==1.16.2` download timeout; official pytest never started. Its raw + zero has both an invalid frozen boundary and a verifier dependency error. +- **WAL recovery:** no post-end message was observed, but the original adapter + never acknowledged process stoppage. Official tests reported missing + `/app/recovered.json` (0 passed, 7 failed). The raw zero remains unscored + because the timeout boundary is unverified. +- **Caffe:** the corrected adapter reached the native 3,600-second agent limit, + acknowledged cancellation with **13 stopped processes**, and then ran the + official verifier. All **six tests passed**. The native `AgentTimeoutError` + and standard `execution_error` summary remain; the analytical score is one. + No post-end or post-verifier-start message was recorded. + +All 28 corrected-source runs had completed or confirmed-cancelled supervision +records before official grading. The required guard rejects missing or +incomplete records. Successful solution servers may remain running by design; +cancellation stops detached agent tools as well. This amendment does not +retroactively validate the first two timeout attempts. There were **zero model +setup errors, zero whole-trial retries, and zero verifier retries**. The sampler +dependency error overlaps its invalid-boundary exclusion; it is not a third +unscored task. + +## Per-task outcomes + +| Task | Previous Nano raw | Pi raw | Audited outcome | Official tests passed / failed | Agent seconds | +| --- | ---: | ---: | --- | ---: | ---: | +| adaptive-rejection-sampler | 0 | 0 | Unscored: invalid boundary + verifier error | Not started | 901.3 | +| build-cython-ext | 1 | 1 | Passed | 11 / 0 | 834.2 | +| cancel-async-tasks | 1 | 0 | Failed | 5 / 1 | 104.8 | +| cobol-modernization | 1 | 1 | Passed | 3 / 0 | 430.2 | +| db-wal-recovery | 0 | 0 | Unscored: unverified timeout boundary | 0 / 7 | 898.7 | +| fix-code-vulnerability | 1 | 1 | Passed | 6 / 0 | 28.2 | +| git-multibranch | 1 | 1 | Passed | 1 / 0 | 80.5 | +| kv-store-grpc | 0 | 1 | Passed | 7 / 0 | 36.1 | +| log-summary-date-ranges | 1 | 0 | Failed | 1 / 1 | 13.9 | +| merge-diff-arc-agi-task | 1 | 1 | Passed | 5 / 0 | 77.1 | +| model-extraction-relu-logits | 1 | 0 | Failed | 0 / 1 | 177.0 | +| multi-source-data-merger | 1 | 1 | Passed | 3 / 0 | 34.3 | +| nginx-request-logging | 1 | 1 | Passed | 8 / 0 | 52.1 | +| openssl-selfsigned-cert | 1 | 1 | Passed | 6 / 0 | 16.5 | +| pypi-server | 1 | 1 | Passed | 1 / 0 | 70.3 | +| pytorch-model-cli | 1 | 0 | Failed | 5 / 1 | 660.7 | +| pytorch-model-recovery | 1 | 1 | Passed | 5 / 0 | 178.3 | +| qemu-alpine-ssh | 1 | 1 | Passed | 1 / 0 | 654.9 | +| query-optimize | 0 | 0 | Failed | 5 / 1 | 318.0 | +| regex-log | 1 | 1 | Passed | 1 / 0 | 157.0 | +| torch-pipeline-parallelism † | 0 | 1 | Passed | 4 / 0 | 562.2 | +| torch-tensor-parallelism | 1 | 1 | Passed | 13 / 0 | 90.8 | +| write-compressor | 1 | 1 | Passed | 3 / 0 | 304.1 | +| constraints-scheduling | 1 | 1 | Passed | 3 / 0 | 23.0 | +| custom-memory-heap-crash | 1 | 1 | Passed | 6 / 0 | 96.2 | +| dna-assembly | 1 | 0 | Failed | 0 / 1 | 763.3 | +| llm-inference-batching-scheduler | 1 | 1 | Passed | 6 / 0 | 337.7 | +| path-tracing | 1 | 1 | Passed | 5 / 0 | 133.1 | +| schemelike-metacircular-eval | 1 | 1 | Passed | 1 / 0 | 800.1 | +| caffe-cifar-10 | 1 | 1 | Timeout; confirmed stop, passed | 6 / 0 | 3600.4 | +| circuit-fibsqrt | 1 | 1 | Passed | 3 / 0 | 239.0 | +| mteb-leaderboard | 1 | 1 | Passed | 2 / 0 | 667.7 | +| regex-chess | 0 | 1 | Passed | 4 / 0 | 1618.2 | +| build-pov-ray | 1 | 1 | Passed | 3 / 0 | 267.4 | + +† `torch-pipeline-parallelism` accessed public Terminal-Bench tests and a +reference solution. Its official pass is retained and flagged below. +Test counts use the final official pytest summary, preserving parameterized +cases: pipeline has four cases and tensor parallelism 13; native CTRF summaries +deduplicate these to three each. The 32 accepted evaluations executed **139 +official cases: 133 passed, six failed, zero errors/skips**. The unscored WAL +trial's seven failed cases are separate. Embedded upstream/self-test counts +are not added to the final official wrapper totals. + +The six valid failures were: + +| Task | Official failure evidence | +| --- | --- | +| cancel-async-tasks | Excess tasks missed cleanup messages: 0 instead of 2 | +| log-summary-date-ranges | Month-to-date ERROR count 14160 instead of 4682 | +| model-extraction-relu-logits | Recovered matrix failed to match row 24 | +| pytorch-model-cli | CLI predictions differed from expected values | +| query-optimize | Runtime 0.8657636019925121 seconds exceeded 1.05 × 0.6539265539904591 seconds | +| dna-assembly | Primer melting temperatures differed by 5.4112089999999995 degrees C, above the 5-degree limit | + +These are official assertion findings, not proofs of complete root causes. +Against the [October 12 Nano baseline](dev34-main-baseline-20261012.md), three +previous failures now have official passes: `kv-store-grpc`, +`torch-pipeline-parallelism` (exposure flagged), and `regex-chess`. Five previous +passes now fail: `cancel-async-tasks`, `log-summary-date-ranges`, +`model-extraction-relu-logits`, `pytorch-model-cli`, and `dna-assembly`. +Sampler and WAL remain unresolved. Historical Nano's 28/34 is not directly +comparable to a complete new Pi rate, which this run does not supply. + +## Exposure and private artifacts + +After freezing outputs, a targeted scan reviewed **925 recorded tool starts** +and matching tool outputs for benchmark repositories, hidden paths, and +environment access. `torch-pipeline-parallelism` downloaded upstream +`tests/test_outputs.py` (178 lines), `solution/solve.sh` (112 lines), and +`instruction.md` (33 lines) from public repository `harbor-framework/terminal-bench-2`, +read the reference solution, and ran the downloaded tests. The remote `main` +was not pinned to the registered package revision. Its pass does not establish +unaided task completion. WAL and Git listed `/logs/verifier` while it was empty; +those listing outputs contained no official tests. MTEB application sources, +public results, and the Scandinavian leaderboard were consulted for that task, +separately from Terminal-Bench reference material. + +WAL and `llm-inference-batching-scheduler` printed environment variables that +included the configured DeepSeek API key. An exact-match scan found the key in +**four local raw files**, the event and native-session files for those two tasks. +The key value and raw traces are not published. Rotate that key before another +experiment. Published artifacts contain derived findings and hashes only. +This targeted audit is not proof that every other exposure or exfiltration +channel was absent. The pool was already exposed development material. + +## Timing, usage, and reproduction + +Model jobs ran from **2026-10-29 13:54:04.318461 UTC** to +**2026-10-29 19:11:52.019151 UTC**: **317.8 minutes**, including pair barriers, +verification, and the supervision-amendment gap. Summed agent execution was +**15,227.44641 seconds**. All controllers and task containers exited; older +stopped recovery containers were preserved. + +Recorded usage totals **27,164,763 input tokens** (including **26,331,264 cached +tokens**) and **718,414 output tokens**. The known peak-rate cost subtotal is +approximately **$1.270134084**. All three interrupted trials have partial usage +and cost coverage; the old adapter's false completeness flags remain raw and +are corrected only in derived audit fields. In-flight or later legacy activity +can be missing. These are known subtotals, not a complete token count or a +reconciled bill; registered pricing also excludes off-peak discounts. + +The registered controller was invoked from the experiment worktree with the +key supplied only through the environment. It delegates each job to the +standard Harbor entry point from its corresponding frozen checkout: + +```bash +export UV_PROJECT_ENVIRONMENT="$PWD/jobs/harbor-supervised-runtime-venv" +uv run --frozen --project jobs/supervised-source/benchmarks/harbor \ + python benchmarks/harbor/scripts/run_comparison.py \ + --plan benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json \ + --phase all +``` + +```bash +uv run --project benchmarks/harbor python -m harbor_adapter run --config +``` + +Completed records are skipped; started-but-unfinished records refuse replay. +The final v7 plan validated. All 34 source snapshots, task revisions, public +model configurations, and distribution pins matched; pair barriers and +preflight gates passed. All 1,449 frozen file hashes were rechecked during +audit. Native artifacts remain in ignored `jobs/runtime-source/jobs/` and +`jobs/supervised-source/jobs/`; reports and audited JSON are the shareable +baseline record. A complete English/Simplified Chinese report pair is maintained +in the same change. + +## Post-run workflow review + +Four reproducible workflow defects were corrected after this run: zero-exit +model-loading warnings could hide fallback settings; multiple Pi configurations +could overwrite one job-local snapshot; failed streams with initialized zero +usage could be labeled complete after a successful retry; and concurrent first +publication of a shared distribution could abort one preparation. Regression +tests reproduced nine failures against the evaluated corrected source, while +the updated full Harbor suite passed **238 tests**. The pinned Pi 1.4.0 offline +probe accepted the registered configuration and rejected an invalid `maxTokens` +configuration despite its zero exit code, without a model call. + +The [review evidence](../../configs/pi-deepseek-flash-dev34-20261024/review-fixes-evidence.json) +records the fixes and archive checks. Each archived model job had one Pi agent +and the registered valid public model configuration; no model-loading warning +or failed zero-usage assistant event was found in the checked job logs/events. +All 1,449 frozen file hashes still match. The original evaluated sources, +exceptions, usage, outcomes, and result JSON remain unchanged; these workflow +fixes apply to future runs and do not imply that this baseline was rerun. diff --git a/benchmarks/harbor/reports/zh-CN/pi-deepseek-flash-dev34-20261024.md b/benchmarks/harbor/reports/zh-CN/pi-deepseek-flash-dev34-20261024.md new file mode 100644 index 0000000..54763c9 --- /dev/null +++ b/benchmarks/harbor/reports/zh-CN/pi-deepseek-flash-dev34-20261024.md @@ -0,0 +1,220 @@ +# Pi + DeepSeek Flash:34 题开发集 baseline + +**34 题均获得一次全新的模型尝试**,并于 2026 年 10 月 29 日运行结束。 +审计结果为 **26 题官方验证通过、6 题评分失败、2 次旧适配器超时尝试不评分**。 +26 次通过中,一次发生在 agent 达到时限且确认停止之后,另一次读取过公开参考测试 +和解法。因此,本轮**不能给出完整且无答案接触的 34 题通过率**。 + +未修改的标准摘要记录了 25 次正常通过、6 次正常失败和 3 次 +`AgentTimeoutError` 执行错误。原生 reward 为 26 个一和 8 个零。 +审计采纳有进程托管确认的 Caffe 超时尝试的原始一分,将两次旧超时的原始零分 +在分析中保留为空。审计不改写异常、不修改原始 reward,也不替换模型输出。 + +[审计结果 JSON](../../results/pi-deepseek-flash-dev34-20261024.json) 包含全部 +trial ID、源码/任务/配置来源、原始及分析结论、官方测试数量、用量、时间、 +审计标记和证据哈希。[v7 登记](../../configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json) 保留前六次尝试, +并为剩余 28 题登记修正后的进程托管。全部输出于 +**2026-10-30 13:54:30.827662 UTC** 完成哈希冻结,早于新的解题轨迹或隐藏测试 +内容审阅;[冻结清单](../../configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json) 覆盖 **1,449 个文件**。 + +## Agent、模型与协议 + +| 属性 | 值 | +| --- | --- | +| 题池 | 与 10 月 12 日完全相同的开发集选择:34 题,任务 revision 一致;其余 55 个任务包未打开 | +| Agent | 未修改的上游 Pi 1.4.0 本地二进制分发包;固定所有文件和权限;未独立核验上游发布压缩包校验和 | +| 模型 | `deepseek/deepseek-flash`,官方 `https://api.deepseek.com` OpenAI 兼容端点 | +| 别名 | 10 月 24 日和 27 日核对的官方文档映射为 DeepSeek-V4.1-Flash-1017;别名可变,不是不可变 checkpoint | +| 推理和采样 | `high`;未设置 temperature | +| Agent 行为 | 原生 prompt、工具、上下文压缩和请求重试;不复制个人设置、扩展、技能、历史及其他 provider | +| 模型元数据 | 上下文 1,000,000;输出上限 384,000;保留 Pi 原生生成行为 | +| 限制 | 保留任务原生 agent 时限及资源;独立的 1,800 秒安装时限 | +| 队列 | 按原生时限再按名称升序;相邻两题设屏障;并发二;每题一个输出 | +| Verifier | 原始官方脚本和单次时限;只有 verifier 超时可对同一输出重试,最多三次;实际重试零次 | +| 安装 | 原始 34 次检查通过;进程托管修正后额外 28 次检查/探针通过 | + +QEMU 已登记的 `qemu-alpine-ssh-v1` 依赖 profile 在两个源码版本下均通过。 +其余 33 题的 profile 明确保留为 `not_configured`;安装成功不等于未支持的 +verifier 依赖通过预检。经核验的缓存保留原始包名、大小、SHA256、来源及任务 +镜像。每批启动前核验原始镜像身份和 repository digest。仅在不强制删除的前提下 +清理本轮新增镜像引用;受保护镜像和九个更早的停止态恢复容器全部保留。 + +```text +plan SHA256: d90107ed15e47981fb3a09e74eb5d3b0e4f72acc96b30308c6d66ec28151128c +freeze SHA256: 76b775d898bbb84b21f80eee1061806d7d949465f9eef60dcab6dedeb326734b +results SHA256: b85d3bf2b0ea9661940f0e6b08e3bea71b2c41ae34da5c89f9ecd5197c6c8ff5 +selection SHA256: 4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85 +Pi distribution SHA256: 60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9 +first six adapter source: d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67 +remaining 28 adapter source: 29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d +``` + +此前 Nano baseline 使用 100 次回复、65,536 token 的输出请求、原生时限减 +180 秒的工作预算、Anthropic 兼容端点,且未设置 reasoning effort。 +Pi 使用原生循环、完整任务时限、OpenAI 兼容端点及 `high` 推理。 +日期、可变模型别名和这些协议差异使本轮无法进行 agent 质量或 checkpoint +效果的因果比较。各题之间未优化 agent 或 prompt。 + +## 进程托管修正与超时记账 + +前六次尝试使用 `d6ed70a`,三题正常通过,一题正常失败; +`adaptive-rejection-sampler` 和 `db-wal-recovery` 超时。运行元数据时间戳显示 +sampler 在 verifier 运行期间仍有 Pi 活动,因此控制器在整对任务结束后、 +第七题开始前停止。通过 **225 项 Harbor 测试**、禁网 Docker 取消探针及 +28 次全新安装/托管检查后,尚未启动的 28 题改用 `29db8d7`。 +Pi 字节、模型设置、prompt/工具、任务顺序/revision 和预算均未改变。 +参见[配置历史](../../configs/pi-deepseek-flash-dev34-20261024/README.md)、 +[原始六次尝试证据](../../configs/pi-deepseek-flash-dev34-20261024/model-prefix-evidence.json)、 +[超时证据](../../configs/pi-deepseek-flash-dev34-20261024/timeout-boundary-evidence.json) 和 +[Docker 探针](../../configs/pi-deepseek-flash-dev34-20261024/process-supervision-probe-evidence.json)。 + +- **Sampler:** 在所记录的 agent 结束时间后仍有 40 条消息元数据记录, + 部分与评分重叠。Verifier 还遇到了 APT 锁和 `scipy==1.16.2` 下载超时, + 官方 pytest 未开始。原始零分同时具有冻结边界无效和 verifier 依赖错误。 +- **WAL recovery:** 未观察到结束后的消息,但旧适配器从未确认进程已停止。 + 官方测试报告缺少 `/app/recovered.json`(0 项通过、7 项失败)。由于超时 + 边界未确认,原始零分仍不评分。 +- **Caffe:** 修正后的适配器达到原生 3,600 秒 agent 时限后,确认取消并 + **停止了 13 个进程**,随后运行官方 verifier,**六项测试全部通过**。 + 原生 `AgentTimeoutError` 和标准 `execution_error` 摘要保留;分析分数为一。 + 未记录到 agent 结束后或 verifier 启动后的消息。 + +修正源码下全部 28 次尝试在官方评分前均有正常完成或已确认取消的托管记录。 +强制检查会拒绝缺失或不完整记录。正常完成时允许有意启动的解题服务继续运行; +取消时会同时停止脱离原会话的 agent 工具。修正不能追溯验证前两次超时尝试。 +**模型阶段安装错误零次、整题重试零次、verifier 重试零次**。 +Sampler 的依赖错误与其无效边界排除项重叠,不构成第三个不评分任务。 + +## 逐题结果 + +| 任务 | 此前 Nano 原始分数 | Pi 原始分数 | 审计结论 | 官方测试通过 / 失败 | Agent 秒数 | +| --- | ---: | ---: | --- | ---: | ---: | +| adaptive-rejection-sampler | 0 | 0 | 不评分:边界无效及 verifier 错误 | 未开始 | 901.3 | +| build-cython-ext | 1 | 1 | 通过 | 11 / 0 | 834.2 | +| cancel-async-tasks | 1 | 0 | 失败 | 5 / 1 | 104.8 | +| cobol-modernization | 1 | 1 | 通过 | 3 / 0 | 430.2 | +| db-wal-recovery | 0 | 0 | 不评分:超时边界未确认 | 0 / 7 | 898.7 | +| fix-code-vulnerability | 1 | 1 | 通过 | 6 / 0 | 28.2 | +| git-multibranch | 1 | 1 | 通过 | 1 / 0 | 80.5 | +| kv-store-grpc | 0 | 1 | 通过 | 7 / 0 | 36.1 | +| log-summary-date-ranges | 1 | 0 | 失败 | 1 / 1 | 13.9 | +| merge-diff-arc-agi-task | 1 | 1 | 通过 | 5 / 0 | 77.1 | +| model-extraction-relu-logits | 1 | 0 | 失败 | 0 / 1 | 177.0 | +| multi-source-data-merger | 1 | 1 | 通过 | 3 / 0 | 34.3 | +| nginx-request-logging | 1 | 1 | 通过 | 8 / 0 | 52.1 | +| openssl-selfsigned-cert | 1 | 1 | 通过 | 6 / 0 | 16.5 | +| pypi-server | 1 | 1 | 通过 | 1 / 0 | 70.3 | +| pytorch-model-cli | 1 | 0 | 失败 | 5 / 1 | 660.7 | +| pytorch-model-recovery | 1 | 1 | 通过 | 5 / 0 | 178.3 | +| qemu-alpine-ssh | 1 | 1 | 通过 | 1 / 0 | 654.9 | +| query-optimize | 0 | 0 | 失败 | 5 / 1 | 318.0 | +| regex-log | 1 | 1 | 通过 | 1 / 0 | 157.0 | +| torch-pipeline-parallelism † | 0 | 1 | 通过 | 4 / 0 | 562.2 | +| torch-tensor-parallelism | 1 | 1 | 通过 | 13 / 0 | 90.8 | +| write-compressor | 1 | 1 | 通过 | 3 / 0 | 304.1 | +| constraints-scheduling | 1 | 1 | 通过 | 3 / 0 | 23.0 | +| custom-memory-heap-crash | 1 | 1 | 通过 | 6 / 0 | 96.2 | +| dna-assembly | 1 | 0 | 失败 | 0 / 1 | 763.3 | +| llm-inference-batching-scheduler | 1 | 1 | 通过 | 6 / 0 | 337.7 | +| path-tracing | 1 | 1 | 通过 | 5 / 0 | 133.1 | +| schemelike-metacircular-eval | 1 | 1 | 通过 | 1 / 0 | 800.1 | +| caffe-cifar-10 | 1 | 1 | 超时,确认停止后通过 | 6 / 0 | 3600.4 | +| circuit-fibsqrt | 1 | 1 | 通过 | 3 / 0 | 239.0 | +| mteb-leaderboard | 1 | 1 | 通过 | 2 / 0 | 667.7 | +| regex-chess | 0 | 1 | 通过 | 4 / 0 | 1618.2 | +| build-pov-ray | 1 | 1 | 通过 | 3 / 0 | 267.4 | + +† `torch-pipeline-parallelism` 读取过公开 Terminal-Bench 测试和参考解法; +保留其官方通过结果,并在下文标记。测试数量取最后一条官方 pytest 汇总, +保留参数化实例:pipeline 为四项、tensor parallelism 为 13 项;原生 CTRF +分别去重为三项。32 个采纳的评测执行了 **139 项官方测试:133 项通过、 +6 项失败、错误/跳过均为零**。不评分的 WAL 尝试另有七项失败。 +嵌套的上游测试或自测数量不叠加进最终官方包装测试总数。 + +六个有效失败的证据如下: + +| 任务 | 官方失败证据 | +| --- | --- | +| cancel-async-tasks | 超额任务缺少清理消息:应为 2 次,实际 0 次 | +| log-summary-date-ranges | 月初至今 ERROR 数量应为 4682,实际 14160 | +| model-extraction-relu-logits | 恢复矩阵的第 24 行未匹配 | +| pytorch-model-cli | CLI 预测值与预期不同 | +| query-optimize | 耗时 0.8657636019925121 秒,超过 1.05 × 0.6539265539904591 秒 | +| dna-assembly | 引物熔解温度差 5.4112089999999995 摄氏度,超过 5 度限制 | + +这些是官方断言结果,不是完整根因的证明。相对于 +[10 月 12 日 Nano baseline](dev34-main-baseline-20261012.md),三题从失败变为 +官方通过:`kv-store-grpc`、`torch-pipeline-parallelism`(标记答案接触)和 +`regex-chess`。五题从通过变为失败:`cancel-async-tasks`、 +`log-summary-date-ranges`、`model-extraction-relu-logits`、`pytorch-model-cli` +和 `dna-assembly`。Sampler 和 WAL 仍未解决。此前 Nano 的 28/34 无法与 +完整的新 Pi 通过率直接比较,因为本轮没有提供后者。 + +## 答案接触与私有产物 + +冻结后,针对性扫描审阅了 **925 次记录到的工具启动**及匹配输出,检查 benchmark +仓库、隐藏路径和环境变量访问。`torch-pipeline-parallelism` 从公开仓库 +`harbor-framework/terminal-bench-2` 下载了上游 `tests/test_outputs.py` +(178 行)、`solution/solve.sh`(112 行)和 `instruction.md`(33 行), +读取参考解法并执行下载的测试。远端 `main` 未固定到登记的任务包 revision。 +这次通过不代表独立解题。WAL 和 Git 曾列出当时为空的 `/logs/verifier`; +这些目录列表输出未包含官方测试。MTEB 题访问过其应用上游源码、公开结果和 +Scandinavian leaderboard,这与 Terminal-Bench 参考材料单独记录。 + +WAL 和 `llm-inference-batching-scheduler` 打印的环境变量包含配置中的 +DeepSeek API key。精确匹配扫描在这两题的事件文件及原生会话文件中,共发现 +**四个含 key 的本地原始文件**。不发布 key 值或原始轨迹;下一轮实验前应 +轮换该 key。公开产物只包含分析结论及哈希。这种针对性审计不能证明所有其他 +答案接触或数据外传途径都不存在。该题池本身也是此前已接触的开发材料。 + +## 时间、用量与复现 + +模型任务从 **2026-10-29 13:54:04.318461 UTC** 运行到 +**2026-10-29 19:11:52.019151 UTC**,共 **317.8 分钟**,包含成对屏障、 +评分和进程托管修正期间的间隔。Agent 执行累计 **15,227.44641 秒**。 +全部控制器和任务容器均已退出;更早的停止态恢复容器保留。 + +记录的用量合计 **27,164,763 输入 token**(含 **26,331,264 缓存 token**) +和 **718,414 输出 token**。按登记峰值费率估算的已知费用小计约 +**$1.270134084**。三次中断尝试的用量和费用均不完整;旧适配器错误的完整性 +标记原样保留,仅在派生审计字段中纠正。进行中的请求或旧超时后继续发生的活动 +可能缺失。这些是已知小计,不是完整 token 总数或对账后的账单;登记价格也未 +包含非高峰折扣。 + +登记控制器从实验 worktree 根目录启动,key 仅通过环境变量提供。 +它在各自冻结的源码 checkout 中调用标准 Harbor 入口: + +```bash +export UV_PROJECT_ENVIRONMENT="$PWD/jobs/harbor-supervised-runtime-venv" +uv run --frozen --project jobs/supervised-source/benchmarks/harbor \ + python benchmarks/harbor/scripts/run_comparison.py \ + --plan benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json \ + --phase all +``` + +```bash +uv run --project benchmarks/harbor python -m harbor_adapter run --config +``` + +已完成记录会跳过;已开始但未完成的记录拒绝重放。最终 v7 计划验证通过, +34 份源码快照、任务 revision、公开模型配置和分发包固定值全部匹配, +成对屏障和安装门禁均通过。审计重新核验了全部 1,449 个冻结文件的哈希。 +原生产物保存在忽略的 `jobs/runtime-source/jobs/` 和 +`jobs/supervised-source/jobs/` 下;报告和审计 JSON 构成可分享的 baseline 记录。 +完整英文与简体中文报告在同一变更中维护。 + +## 运行后的工作流评审 + +本轮结束后修正了四个可复现的工作流缺陷:模型加载警告即使返回零退出码也可能 +掩盖配置回退;多个 Pi 配置可能覆盖同一任务的快照;失败流初始化的零用量在 +重试成功后可能被标为完整;共享分发包首次并发发布可能使一次准备中止。 +回归测试在本轮评测使用的修正源码上复现了九个失败,更新后的完整 Harbor +测试集 **238 项通过**。固定的 Pi 1.4.0 离线探针接受了已登记配置,并在不调用 +模型的情况下,拒绝了虽然退出码为零、但 `maxTokens` 无效的配置。 + +[评审证据](../../configs/pi-deepseek-flash-dev34-20261024/review-fixes-evidence.json) +记录了修复和归档检查。每个归档模型任务只有一个 Pi agent,使用已登记的有效 +公开模型配置;在检查的任务日志/事件中未发现模型加载警告或失败的零用量 +assistant 事件。全部 1,449 个冻结文件哈希仍匹配。原始评测源码、异常、用量、 +结论和结果 JSON 均保持不变;这些工作流修复用于后续运行,不表示重新运行过 +这份 baseline。 diff --git a/benchmarks/harbor/results/pi-deepseek-flash-dev34-20261024.json b/benchmarks/harbor/results/pi-deepseek-flash-dev34-20261024.json new file mode 100644 index 0000000..2e0084e --- /dev/null +++ b/benchmarks/harbor/results/pi-deepseek-flash-dev34-20261024.json @@ -0,0 +1,4733 @@ +{ + "schema_version": 1, + "experiment": "pi-deepseek-flash-dev34-20261024", + "audited_at": "2026-10-30T14:00:31.952685+00:00", + "frozen_at": "2026-10-30T13:54:30.827662+00:00", + "freeze_manifest": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/outputs-freeze.json", + "freeze_sha256": "76b775d898bbb84b21f80eee1061806d7d949465f9eef60dcab6dedeb326734b", + "registration": "benchmarks/harbor/configs/pi-deepseek-flash-dev34-20261024/experiment-v7.json", + "registration_sha256": "d90107ed15e47981fb3a09e74eb5d3b0e4f72acc96b30308c6d66ec28151128c", + "selection_sha256": "4572b1cc432a16b9c5d1dd554c8d2189ae80414f386a90bf03829bc4d0c3da85", + "scope": "The same previously exposed 34-task development pool; the remaining 55 tasks were not opened. One fresh output per task; no whole-trial retries or output replacements.", + "model_policy": { + "model": "deepseek/deepseek-flash", + "api": "openai-completions", + "base_url": "https://api.deepseek.com", + "thinking": "high", + "model_config": "pi-models.json", + "model_metadata_max_tokens": 384000, + "pricing": "Peak-rate estimates from the registered Pi model config; not billing reconciliation. Official pricing also has off-peak discounts.", + "alias": "Official documentation rechecked 2026-10-27 still maps deepseek-flash to DeepSeek-V4.1-Flash-1017; the alias is mutable." + }, + "budget_policy": "Original task-native agent timeouts, original resources, and 1800-second setup. Stock Pi loop and generation behavior: no injected 100-turn cap or 180-second finalization reminder. Model metadata maxTokens=384000 is preserved from the local provider configuration. This differs from the historical Nano baseline (100 replies, 65536 requested output tokens, native timeout minus 180 seconds for work), so this is an agent baseline rather than a matched-budget causal comparison. Temperature unset.", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9", + "sources": { + "pi": { + "agent": "pi", + "source_root": "jobs/runtime-source", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + }, + "pi_supervised": { + "agent": "pi", + "source_root": "jobs/supervised-source", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "distribution_path": "/home/minix/.local/share/mise/installs/pi/1.4.0/pi", + "distribution_sha256": "60a810111c1fd9b34b59a815fb6399c6b861017e1a2278ae881bb2a563177dd9" + } + }, + "summary": { + "model_attempts": 34, + "normal_passes": 25, + "normal_scored_failures": 6, + "native_agent_timeouts": 3, + "confirmed_timeout_passes": 1, + "official_passes_with_accepted_boundary": 26, + "accepted_evaluations": 32, + "unscored_boundary_exceptions": 2, + "full_34_task_pass_rate": null, + "raw_reward_ones": 26, + "model_setup_errors": 0, + "verifier_dependency_errors": 1, + "verifier_dependency_errors_overlap_invalid_boundary": true, + "verifier_retries": 0, + "whole_trial_retries": 0, + "pytest_accepted": { + "passed": 133, + "failed": 6, + "errors": 0, + "skipped": 0 + }, + "external_benchmark_material_passes": 1, + "provider_key_exposure_tasks": 2 + }, + "timing": { + "first_model_job_started_at": "2026-10-29T13:54:04.318461+00:00", + "last_model_job_finished_at": "2026-10-29T19:11:52.019151+00:00", + "model_phase_wall_seconds_including_amendment_gap": 19067.70069, + "agent_seconds_sum": 15227.44641 + }, + "usage": { + "n_input_tokens": 27164763, + "n_cache_tokens": 26331264, + "n_output_tokens": 718414, + "known_peak_rate_cost_subtotal_usd": "1.2701340839999999538", + "partial_trials": [ + "terminal-bench/adaptive-rejection-sampler", + "terminal-bench/db-wal-recovery", + "terminal-bench/caffe-cifar-10" + ] + }, + "historical_comparison": { + "baseline": "benchmarks/harbor/results/dev34-main-baseline-20261012.json", + "baseline_sha256": "90873385a2277776f09a37c77db390432d223b28461a892f564d24ab4f9407c3", + "historical_passes": 28, + "historical_failures": 6, + "new_official_passes_from_historical_failures": [ + "terminal-bench/kv-store-grpc", + "terminal-bench/torch-pipeline-parallelism", + "terminal-bench/regex-chess" + ], + "new_failures_from_historical_passes": [ + "terminal-bench/cancel-async-tasks", + "terminal-bench/log-summary-date-ranges", + "terminal-bench/model-extraction-relu-logits", + "terminal-bench/pytorch-model-cli", + "terminal-bench/dna-assembly" + ], + "unresolved": [ + "terminal-bench/adaptive-rejection-sampler", + "terminal-bench/db-wal-recovery" + ], + "interpretation": "Descriptive paired outcomes only. One newly passing task retrieved public reference tests and a solution. Pi uses a different agent loop, prompt, API compatibility endpoint, reasoning setting, and internal budget policy on a later mutable model alias; this does not isolate agent quality or model checkpoint effects." + }, + "image_transport_audit": { + "path": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/image-cache.jsonl", + "sha256": "ad6f3b8fccd2b253df5872b54656391f75130ead448e3ce03b15f07f84839939", + "actions": { + "restored": 48, + "verified": 56, + "evicted": 48 + }, + "remaining_28_original_images_verified": true, + "policy": "Original repository digests and image identities verified before each batch; only newly introduced image references evicted without force. Initial protected images and the nine earlier stopped recovery containers were retained." + }, + "accounting_policy": "Keep native exceptions, raw rewards, and execution_error summaries unchanged. Accept the supervised Caffe timeout reward only because cancellation was acknowledged before official tests. Legacy timeout raw zeros are analytically null. Report no complete 34-task pass rate and no claim of evaluation without benchmark-answer access. The pipeline official pass remains flagged for public tests/reference-solution retrieval. Token/cost totals are known subtotals; interrupted usage may be missing and legacy timeout flags are overridden only in this derived audit. Prices are registered peak-rate estimates, not reconciled billing.", + "validation": { + "frozen_files_rehashed": 1449, + "source_snapshots_match": 34, + "registered_task_refs_match": 34, + "pi_distribution_and_public_model_config_match": 34, + "queue_pair_barriers_passed": true, + "preflight_before_corresponding_model_source": true, + "original_installation_checks_passed": 34, + "supervised_installation_checks_passed": 28, + "supervised_process_probes_passed": 28, + "supervised_finished_or_cancelled_before_verifier": 28, + "official_task_files_match_registered_hashes": true, + "final_registered_plan_validation_passed": true, + "active_trial_containers_after_run": 0 + }, + "exposure_audit": { + "method": "After freezing all outputs, scan all 925 recorded tool starts for benchmark repository, hidden-path, and environment-access candidates; inspect matching paired tool outputs. Exact-match scan native files for the configured provider key without printing or publishing its value. This targeted scan is not proof of exhaustive absence of other exposure or exfiltration.", + "tool_calls_scanned": 925, + "benchmark_finding": "torch-pipeline-parallelism downloaded public upstream tests/test_outputs.py (178 lines), solution/solve.sh (112 lines), and instruction.md (33 lines), read the reference solution, and ran the downloaded tests before submitting. Its official pass is exposure-flagged, not evidence of unaided task completion. The remote main revision was not pinned to the registered package.", + "log_probe_finding": "db-wal-recovery and git-multibranch listed /logs/verifier before it contained official outputs. No official test content was observed through these directory-listing outputs.", + "credential_finding": "db-wal-recovery and llm-inference-batching-scheduler printed environment variables, exposing the configured DeepSeek API key in tool outputs and native sessions. Four raw files contain the exact key; they remain local and must not be published. Rotate the exposed key before another experiment.", + "credential_file_paths": [ + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/pi.txt", + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/pi/sessions/2026-10-29T14-26-49-590Z_01a1880e-6e76-7275-a097-a20814e7b807.jsonl", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/pi.txt", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/pi/sessions/2026-10-29T17-21-07-841Z_01a188ae-0301-7023-953b-0d86de4e7501.jsonl" + ], + "upstream_application_material": "MTEB upstream application sources, results, and the Scandinavian leaderboard were consulted for the mteb-leaderboard task; these are distinct from Terminal-Bench reference tests and solutions.", + "raw_trajectories_published": false + }, + "preflights": [ + { + "arm": "pi", + "task": "terminal-bench/adaptive-rejection-sampler", + "trial": "adaptive-rejection-sampler__6Te54Hh", + "job_name": "pi-flash-dev34-install-pair-01-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-01-20261027.json", + "record_sha256": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/build-cython-ext", + "trial": "build-cython-ext__SoCrSf2", + "job_name": "pi-flash-dev34-install-pair-01-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-01-20261027.json", + "record_sha256": "577ed5673d63363008800d8818d8ebccf22551c23eb0f778701942582c36016f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/cancel-async-tasks", + "trial": "cancel-async-tasks__4MwHm6o", + "job_name": "pi-flash-dev34-install-pair-02-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-02-20261027.json", + "record_sha256": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/cobol-modernization", + "trial": "cobol-modernization__4x6hSTu", + "job_name": "pi-flash-dev34-install-pair-02-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-02-20261027.json", + "record_sha256": "a00701060839eadc381b06d1d2600c5166ac791d698038deb168cbff475d47f0", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/db-wal-recovery", + "trial": "db-wal-recovery__dfqafBo", + "job_name": "pi-flash-dev34-install-pair-03-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-03-20261027.json", + "record_sha256": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/fix-code-vulnerability", + "trial": "fix-code-vulnerability__PFUMwfw", + "job_name": "pi-flash-dev34-install-pair-03-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-03-20261027.json", + "record_sha256": "d3cd35a8bbaa79e6ad548228df1c311f1674997c036cc49e18e401016a48d805", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/git-multibranch", + "trial": "git-multibranch__eQv2sNb", + "job_name": "pi-flash-dev34-install-pair-04-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-04-20261027.json", + "record_sha256": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/kv-store-grpc", + "trial": "kv-store-grpc__tFkuA9x", + "job_name": "pi-flash-dev34-install-pair-04-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-04-20261027.json", + "record_sha256": "36c93c10b34d08e61259dc4d9d156d94b78464a81ae9b0c8c15a665afc586c8e", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/log-summary-date-ranges", + "trial": "log-summary-date-ranges__84sKwfB", + "job_name": "pi-flash-dev34-install-pair-05-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-05-20261027.json", + "record_sha256": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/merge-diff-arc-agi-task", + "trial": "merge-diff-arc-agi-task__XezJkBY", + "job_name": "pi-flash-dev34-install-pair-05-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-05-20261027.json", + "record_sha256": "ac7bcc1db296a771e15057b594392b8d1710ab14bef640ce9328aa4341340534", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/model-extraction-relu-logits", + "trial": "model-extraction-relu-logits__yZqevWz", + "job_name": "pi-flash-dev34-install-pair-06-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-06-20261027.json", + "record_sha256": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/multi-source-data-merger", + "trial": "multi-source-data-merger__hqtrag6", + "job_name": "pi-flash-dev34-install-pair-06-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-06-20261027.json", + "record_sha256": "ea9a4b35a2a4794831b8427992f91df2c77afba5243955b4f0da75a54a08c159", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/nginx-request-logging", + "trial": "nginx-request-logging__qAFnPiS", + "job_name": "pi-flash-dev34-install-pair-07-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-07-20261027.json", + "record_sha256": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/openssl-selfsigned-cert", + "trial": "openssl-selfsigned-cert__KKFCiB5", + "job_name": "pi-flash-dev34-install-pair-07-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-07-20261027.json", + "record_sha256": "b428c9b6f9121da293f63c1212e56a8fb8da99dd5afc0b2a8578c1eec55ec01f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/pypi-server", + "trial": "pypi-server__EY626hM", + "job_name": "pi-flash-dev34-install-pair-08-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-08-20261027.json", + "record_sha256": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-cli", + "trial": "pytorch-model-cli__ydFu92B", + "job_name": "pi-flash-dev34-install-pair-08-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-08-20261027.json", + "record_sha256": "fe2f9ce77466e936365401338ea68c3ea2833bf55930b3d3cf2f7e3e591b244f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/pytorch-model-recovery", + "trial": "pytorch-model-recovery__AnHZLLY", + "job_name": "pi-flash-dev34-install-pair-09-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-09-20261027.json", + "record_sha256": "8b7cda88cdd32e5415096cc5fea995f0cc8f9a1d16f2dfea935b623a53e35f58", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/qemu-alpine-ssh", + "trial": "qemu-alpine-ssh__uiSwmJr", + "job_name": "pi-flash-dev34-install-pair-09-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-09-20261027.json", + "record_sha256": "8b7cda88cdd32e5415096cc5fea995f0cc8f9a1d16f2dfea935b623a53e35f58", + "status": "setup_passed", + "verifier_profile": { + "profile": "qemu-alpine-ssh-v1", + "status": "passed" + } + }, + { + "arm": "pi", + "task": "terminal-bench/query-optimize", + "trial": "query-optimize__YoXxwZA", + "job_name": "pi-flash-dev34-install-pair-10-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-10-20261027.json", + "record_sha256": "614222e8e2c4079b83b7a1c9e7b5ab11055dcbe005e8983295f1f3856392095a", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/regex-log", + "trial": "regex-log__N2XBsZY", + "job_name": "pi-flash-dev34-install-pair-10-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-10-20261027.json", + "record_sha256": "614222e8e2c4079b83b7a1c9e7b5ab11055dcbe005e8983295f1f3856392095a", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/torch-pipeline-parallelism", + "trial": "torch-pipeline-parallelism__5YUs6Bj", + "job_name": "pi-flash-dev34-install-pair-11-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-11-20261027.json", + "record_sha256": "a4707566ddc1a105e1850f07c038f2de8cefadea9e1bff94b85d17cb9c64784f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/torch-tensor-parallelism", + "trial": "torch-tensor-parallelism__4sM6zDL", + "job_name": "pi-flash-dev34-install-pair-11-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-11-20261027.json", + "record_sha256": "a4707566ddc1a105e1850f07c038f2de8cefadea9e1bff94b85d17cb9c64784f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/constraints-scheduling", + "trial": "constraints-scheduling__AZGqHo7", + "job_name": "pi-flash-dev34-install-pair-12-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-12-20261027.json", + "record_sha256": "1991305b8e1c61171cdae0b07dea7dc9be1681300a9d56854e5d37364d525ebf", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/write-compressor", + "trial": "write-compressor__jKdJYs3", + "job_name": "pi-flash-dev34-install-pair-12-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-12-20261027.json", + "record_sha256": "1991305b8e1c61171cdae0b07dea7dc9be1681300a9d56854e5d37364d525ebf", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/custom-memory-heap-crash", + "trial": "custom-memory-heap-crash__7vzaXoA", + "job_name": "pi-flash-dev34-install-pair-13-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-13-20261027.json", + "record_sha256": "55557503afac1f21545f5cdff0f0b5c8f075936ed68edbdab46c4eae04e0b1c6", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/dna-assembly", + "trial": "dna-assembly__9yECeYj", + "job_name": "pi-flash-dev34-install-pair-13-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-13-20261027.json", + "record_sha256": "55557503afac1f21545f5cdff0f0b5c8f075936ed68edbdab46c4eae04e0b1c6", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/llm-inference-batching-scheduler", + "trial": "llm-inference-batching-scheduler__K5Y4oFq", + "job_name": "pi-flash-dev34-install-pair-14-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-14-20261027.json", + "record_sha256": "3bf013957d71092723564930ab2807a2df1ad479d4773855567b743092b873d3", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/path-tracing", + "trial": "path-tracing__4iWGEgU", + "job_name": "pi-flash-dev34-install-pair-14-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-14-20261027.json", + "record_sha256": "3bf013957d71092723564930ab2807a2df1ad479d4773855567b743092b873d3", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/caffe-cifar-10", + "trial": "caffe-cifar-10__AY4jS2o", + "job_name": "pi-flash-dev34-install-pair-15-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-15-20261027.json", + "record_sha256": "f72f78d43cca70f0603c4875767cd982dadc9e6cf574e87bd9a1487cc1c930a3", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/schemelike-metacircular-eval", + "trial": "schemelike-metacircular-eval__eFmy8Fs", + "job_name": "pi-flash-dev34-install-pair-15-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-15-20261027.json", + "record_sha256": "f72f78d43cca70f0603c4875767cd982dadc9e6cf574e87bd9a1487cc1c930a3", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/circuit-fibsqrt", + "trial": "circuit-fibsqrt__AfGkgwD", + "job_name": "pi-flash-dev34-install-pair-16-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-16-20261027.json", + "record_sha256": "048e1d88c8fc8c8ee23555190c45be1ba729fb663a4517ac4e8485dd7b20ed6f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/mteb-leaderboard", + "trial": "mteb-leaderboard__ouJofdv", + "job_name": "pi-flash-dev34-install-pair-16-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-16-20261027.json", + "record_sha256": "048e1d88c8fc8c8ee23555190c45be1ba729fb663a4517ac4e8485dd7b20ed6f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/build-pov-ray", + "trial": "build-pov-ray__TDQmsPY", + "job_name": "pi-flash-dev34-install-pair-17-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-17-20261027.json", + "record_sha256": "2e0930dd3ebcf157cdc1e588183f6da79974f25f84ce574150dd1516673a94c5", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi", + "task": "terminal-bench/regex-chess", + "trial": "regex-chess__tRkUrhv", + "job_name": "pi-flash-dev34-install-pair-17-20261027", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-install-pair-17-20261027.json", + "record_sha256": "2e0930dd3ebcf157cdc1e588183f6da79974f25f84ce574150dd1516673a94c5", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/git-multibranch", + "trial": "git-multibranch__SSwmKg5", + "job_name": "pi-flash-dev34-supervised-install-pair-01-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-01-20261029.json", + "record_sha256": "8060da0de104029b46eba44267afec694726284d89476f7abe754faae4bf7f9e", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/kv-store-grpc", + "trial": "kv-store-grpc__Gs7PnJb", + "job_name": "pi-flash-dev34-supervised-install-pair-01-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-01-20261029.json", + "record_sha256": "8060da0de104029b46eba44267afec694726284d89476f7abe754faae4bf7f9e", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/log-summary-date-ranges", + "trial": "log-summary-date-ranges__jytpmBX", + "job_name": "pi-flash-dev34-supervised-install-pair-02-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-02-20261029.json", + "record_sha256": "2de6ab97ee9092fe976382131234e7afa84cf0ca29d01b13a5f6efc58e2edf06", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/merge-diff-arc-agi-task", + "trial": "merge-diff-arc-agi-task__H2vzztV", + "job_name": "pi-flash-dev34-supervised-install-pair-02-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-02-20261029.json", + "record_sha256": "2de6ab97ee9092fe976382131234e7afa84cf0ca29d01b13a5f6efc58e2edf06", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/model-extraction-relu-logits", + "trial": "model-extraction-relu-logits__RSYF3V4", + "job_name": "pi-flash-dev34-supervised-install-pair-03-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-03-20261029.json", + "record_sha256": "b28ea5056094ce8e4eb031d4d45113d4aca58dbfe99b89d4ed6e94f2460d5ce0", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/multi-source-data-merger", + "trial": "multi-source-data-merger__hsaU7if", + "job_name": "pi-flash-dev34-supervised-install-pair-03-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-03-20261029.json", + "record_sha256": "b28ea5056094ce8e4eb031d4d45113d4aca58dbfe99b89d4ed6e94f2460d5ce0", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/nginx-request-logging", + "trial": "nginx-request-logging__kG7Tkwv", + "job_name": "pi-flash-dev34-supervised-install-pair-04-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-04-20261029.json", + "record_sha256": "60ae309218d6ecd76872fc920bb0cc01d61e04abd0f6c8d51e82532374ed33d8", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/openssl-selfsigned-cert", + "trial": "openssl-selfsigned-cert__K685jUy", + "job_name": "pi-flash-dev34-supervised-install-pair-04-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-04-20261029.json", + "record_sha256": "60ae309218d6ecd76872fc920bb0cc01d61e04abd0f6c8d51e82532374ed33d8", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/pypi-server", + "trial": "pypi-server__RCF66mG", + "job_name": "pi-flash-dev34-supervised-install-pair-05-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-05-20261029.json", + "record_sha256": "995ec6c0cc508fcb7c9ce03be67b02080d5ab23831ea86d2e0331241b8296dc3", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/pytorch-model-cli", + "trial": "pytorch-model-cli__NoDQAty", + "job_name": "pi-flash-dev34-supervised-install-pair-05-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-05-20261029.json", + "record_sha256": "995ec6c0cc508fcb7c9ce03be67b02080d5ab23831ea86d2e0331241b8296dc3", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/pytorch-model-recovery", + "trial": "pytorch-model-recovery__gLVWp4G", + "job_name": "pi-flash-dev34-supervised-install-pair-06-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-06-20261029.json", + "record_sha256": "46fc0f78d291a7720c74e2520ed4e0d308caa927706ff8f0d824a244e4e7d631", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/qemu-alpine-ssh", + "trial": "qemu-alpine-ssh__kN6kznm", + "job_name": "pi-flash-dev34-supervised-install-pair-06-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-06-20261029.json", + "record_sha256": "46fc0f78d291a7720c74e2520ed4e0d308caa927706ff8f0d824a244e4e7d631", + "status": "setup_passed", + "verifier_profile": { + "profile": "qemu-alpine-ssh-v1", + "status": "passed" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/query-optimize", + "trial": "query-optimize__Q95o6zN", + "job_name": "pi-flash-dev34-supervised-install-pair-07-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-07-20261029.json", + "record_sha256": "8dc70c6f7325f4ecf6775ef4b5e6034c7d5bfe1448ee400a44c21d1efe236a6f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/regex-log", + "trial": "regex-log__6U3bqXA", + "job_name": "pi-flash-dev34-supervised-install-pair-07-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-07-20261029.json", + "record_sha256": "8dc70c6f7325f4ecf6775ef4b5e6034c7d5bfe1448ee400a44c21d1efe236a6f", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/torch-pipeline-parallelism", + "trial": "torch-pipeline-parallelism__ZZdvD2u", + "job_name": "pi-flash-dev34-supervised-install-pair-08-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-08-20261029.json", + "record_sha256": "26aaa244f6c3fec349309d86a26076767c21f5e9173e3e7fd03c5e8c61689f6d", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/torch-tensor-parallelism", + "trial": "torch-tensor-parallelism__yGVXAeu", + "job_name": "pi-flash-dev34-supervised-install-pair-08-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-08-20261029.json", + "record_sha256": "26aaa244f6c3fec349309d86a26076767c21f5e9173e3e7fd03c5e8c61689f6d", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/constraints-scheduling", + "trial": "constraints-scheduling__gp7oG5T", + "job_name": "pi-flash-dev34-supervised-install-pair-09-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-09-20261029.json", + "record_sha256": "cb362b17996b94369c5a55bb82f739d95f122b61bd085bfcea7bf81511edb670", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/write-compressor", + "trial": "write-compressor__GNPQekz", + "job_name": "pi-flash-dev34-supervised-install-pair-09-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-09-20261029.json", + "record_sha256": "cb362b17996b94369c5a55bb82f739d95f122b61bd085bfcea7bf81511edb670", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/custom-memory-heap-crash", + "trial": "custom-memory-heap-crash__Bxqwjy7", + "job_name": "pi-flash-dev34-supervised-install-pair-10-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-10-20261029.json", + "record_sha256": "16712c120dd524292deb9594bf1f17caf6267e8b76eae0bc73ea14bea4725979", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/dna-assembly", + "trial": "dna-assembly__z5vgeDJ", + "job_name": "pi-flash-dev34-supervised-install-pair-10-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-10-20261029.json", + "record_sha256": "16712c120dd524292deb9594bf1f17caf6267e8b76eae0bc73ea14bea4725979", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/llm-inference-batching-scheduler", + "trial": "llm-inference-batching-scheduler__M82gmAy", + "job_name": "pi-flash-dev34-supervised-install-pair-11-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-11-20261029.json", + "record_sha256": "3fb349acb927128e9d756099b3eb722be63a6a733e8234ee118ec502c7511f92", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/path-tracing", + "trial": "path-tracing__RuT4LBe", + "job_name": "pi-flash-dev34-supervised-install-pair-11-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-11-20261029.json", + "record_sha256": "3fb349acb927128e9d756099b3eb722be63a6a733e8234ee118ec502c7511f92", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/caffe-cifar-10", + "trial": "caffe-cifar-10__YQNytFP", + "job_name": "pi-flash-dev34-supervised-install-pair-12-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-12-20261029.json", + "record_sha256": "74582c1f70521d13bbf070399f4c021f3e8453d3d146b72c83ae7a9b154cd2e6", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/schemelike-metacircular-eval", + "trial": "schemelike-metacircular-eval__UVqQ2yo", + "job_name": "pi-flash-dev34-supervised-install-pair-12-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-12-20261029.json", + "record_sha256": "74582c1f70521d13bbf070399f4c021f3e8453d3d146b72c83ae7a9b154cd2e6", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/circuit-fibsqrt", + "trial": "circuit-fibsqrt__MLNmvTp", + "job_name": "pi-flash-dev34-supervised-install-pair-13-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-13-20261029.json", + "record_sha256": "a47addd1295f69e5af895bf1c53e7907f718fa4f222e06cabfdcda37659014d2", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/mteb-leaderboard", + "trial": "mteb-leaderboard__sPwXEw5", + "job_name": "pi-flash-dev34-supervised-install-pair-13-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-13-20261029.json", + "record_sha256": "a47addd1295f69e5af895bf1c53e7907f718fa4f222e06cabfdcda37659014d2", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/build-pov-ray", + "trial": "build-pov-ray__BvAZmWJ", + "job_name": "pi-flash-dev34-supervised-install-pair-14-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-14-20261029.json", + "record_sha256": "410c8c2d6c46d4a405450fc4561aeb2515c468188b5e6bfdedede2c3e0af58c4", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + }, + { + "arm": "pi_supervised", + "task": "terminal-bench/regex-chess", + "trial": "regex-chess__ZLw5bP4", + "job_name": "pi-flash-dev34-supervised-install-pair-14-20261029", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-supervised-install-pair-14-20261029.json", + "record_sha256": "410c8c2d6c46d4a405450fc4561aeb2515c468188b5e6bfdedede2c3e0af58c4", + "status": "setup_passed", + "verifier_profile": { + "profile": null, + "status": "not_configured" + } + } + ], + "trials": [ + { + "task": "terminal-bench/adaptive-rejection-sampler", + "task_ref": "sha256:bcaa2399985cd57666018025846289ab25e193ae0dd8fb7f0ffab2410c24d4de", + "job_name": "pi-flash-dev34-adaptive-rejection-sampler-20261027", + "trial": "adaptive-rejection-sampler__p4rzaFM", + "arm": "pi", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-adaptive-rejection-sampler-20261027.json", + "native_reward": 0.0, + "native_exception_type": "AgentTimeoutError", + "standard_summary_status": "execution_error", + "standard_summary_rewards": null, + "analytical_classification": "unscored_invalid_boundary_and_verifier_error", + "analytical_score": null, + "boundary_status": "invalid", + "process_supervision": null, + "pytest": null, + "ctrf_summary": null, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 412.23068232399237 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T13:54:09.538435Z", + "finished_at": "2026-10-29T13:54:11.628845Z" + }, + "agent_setup": { + "started_at": "2026-10-29T13:54:11.628891Z", + "finished_at": "2026-10-29T13:54:34.019339Z" + }, + "agent_execution": { + "started_at": "2026-10-29T13:54:34.019472Z", + "finished_at": "2026-10-29T14:09:35.305831Z" + }, + "verifier": { + "started_at": "2026-10-29T14:09:35.589907Z", + "finished_at": "2026-10-29T14:16:29.067351Z" + } + }, + "agent_seconds": 901.286359, + "usage": { + "n_input_tokens": 20756, + "n_cache_tokens": 17152, + "n_output_tokens": 34690, + "cost_usd": 0.042812112, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 6, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": true, + "audit_cost_is_partial": true + }, + "operational_metadata": { + "event_count": 215, + "tool_calls": 21, + "post_agent_end_messages": 27, + "messages_during_or_after_verifier_start": 27 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "Agent activity continued during verification. The verifier also hit an APT lock and a scipy==1.16.2 download timeout before official pytest ran. The raw zero is unscored.", + "evidence_sha256": { + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/result.json": "54f5f469df311a50cc463503f529e0ad7188823891ab31840965ef4c78bfef97", + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/manifest.json": "1e4792e091786a5affc7b599a1c3a8d996b12e2648c3265e62d5ce858d08af90", + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027-input/config.json": "d80a1afeaf576d6f97179c0a8e28af8643e3f8886ddacba5707054f0b6fe6be4", + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/agent/pi.txt": "70022bc1980da1fef1d93a9105d5b7da64753d01d526d93cd32184b677e5ec8b", + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/test-stdout.txt": "80cdef0fa19603e6d650150a3d54f9cd160088d94b662b2ebfb141fbb99f962c", + "jobs/runtime-source/jobs/pi-flash-dev34-adaptive-rejection-sampler-20261027/adaptive-rejection-sampler__p4rzaFM/verifier/retry-summary.json": "324b277fcaed5439105f721d4bfe5620fbb10bd122bd6bc7967dc93ee10575be" + }, + "historical_nanopy_raw_reward": 0.0 + }, + { + "task": "terminal-bench/build-cython-ext", + "task_ref": "sha256:35dc561790e73f13e59837828ac0863c0893edc6b71d42cf7d58656c843adf12", + "job_name": "pi-flash-dev34-build-cython-ext-20261027", + "trial": "build-cython-ext__PZdLy7W", + "arm": "pi", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-build-cython-ext-20261027.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "legacy_normal_completion", + "process_supervision": null, + "pytest": { + "passed": 11, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 11, + "passed": 11, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793282916.5235858, + "stop": 1793282924.658896 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 20.88644588198804 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T13:54:07.595951Z", + "finished_at": "2026-10-29T13:54:09.738532Z" + }, + "agent_setup": { + "started_at": "2026-10-29T13:54:09.738573Z", + "finished_at": "2026-10-29T13:54:29.366436Z" + }, + "agent_execution": { + "started_at": "2026-10-29T13:54:29.366559Z", + "finished_at": "2026-10-29T14:08:23.560741Z" + }, + "verifier": { + "started_at": "2026-10-29T14:08:23.792021Z", + "finished_at": "2026-10-29T14:08:44.869223Z" + } + }, + "agent_seconds": 834.194182, + "usage": { + "n_input_tokens": 1742263, + "n_cache_tokens": 1676288, + "n_output_tokens": 15587, + "cost_usd": 0.048554627999999996, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 39, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 517, + "tool_calls": 54, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/result.json": "3a49d523dae23770753f74c93bdff2e97494138a2cd436cde8cd8f4705bc93b9", + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/manifest.json": "534f51423f4bc9cbeff1a2f09808aa888c2bd073abdb4659a7298f2322febfec", + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027-input/config.json": "67ef98793375cac7ace3a013eb2c920bbb74c8636950d1d493b59c545486e411", + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/agent/pi.txt": "997b216ad3a1b32d6186a02fca6575e3008ec53ba9832a55ed272af7a285e039", + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/test-stdout.txt": "e924d3527cb36dddb6f692b12a65f27c1dc6746635442c45de16565f8d111d23", + "jobs/runtime-source/jobs/pi-flash-dev34-build-cython-ext-20261027/build-cython-ext__PZdLy7W/verifier/retry-summary.json": "161bd68190a23c89e95ab137b70e79a2e77f6c38e3d8f27339abc61c35311911" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/cancel-async-tasks", + "task_ref": "sha256:a3d048d351136e48070696cda8bb79660dfd74db1fea3b6da88559f0332699c1", + "job_name": "pi-flash-dev34-cancel-async-tasks-20261027", + "trial": "cancel-async-tasks__WAq9YLK", + "arm": "pi", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-cancel-async-tasks-20261027.json", + "native_reward": 0.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 0.0 + }, + "analytical_classification": "failed", + "analytical_score": 0.0, + "boundary_status": "legacy_normal_completion", + "process_supervision": null, + "pytest": { + "passed": 5, + "failed": 1, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 5, + "failed": 1, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793283596.509414, + "stop": 1793283609.2077332 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 37.298333839004044 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T14:17:19.468502Z", + "finished_at": "2026-10-29T14:17:21.564676Z" + }, + "agent_setup": { + "started_at": "2026-10-29T14:17:21.564738Z", + "finished_at": "2026-10-29T14:17:46.789409Z" + }, + "agent_execution": { + "started_at": "2026-10-29T14:17:46.789525Z", + "finished_at": "2026-10-29T14:19:31.565461Z" + }, + "verifier": { + "started_at": "2026-10-29T14:19:31.806874Z", + "finished_at": "2026-10-29T14:20:09.313792Z" + } + }, + "agent_seconds": 104.775936, + "usage": { + "n_input_tokens": 103334, + "n_cache_tokens": 99584, + "n_output_tokens": 10404, + "cost_usd": 0.014207304, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 13, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 139, + "tool_calls": 13, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "The two excess tasks were cancelled before their cleanup message ran; the official cleanup-count assertion failed (0 instead of 2).", + "evidence_sha256": { + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/result.json": "39339742cbac7e4dc77a7efa1287c5717c9c92f22b57aef56251e8d2e40f8293", + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/manifest.json": "40b9662d7f30f8110fb23213eb78e0f2932d87ce8d475f9d5f708f8e20eb45cc", + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027-input/config.json": "b110936d56dd3f287fae2e4de577a17856cefff6eb51d4d0477587f5050fe19f", + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/agent/pi.txt": "dab321388ecd82f38a1b3158bc962da4b029e34c7a7471b3986afd9e3edb4faf", + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/test-stdout.txt": "ad0e96184b6cf24fb5127c1c2691789290ba56a033e98b2b2ea8992b8854878f", + "jobs/runtime-source/jobs/pi-flash-dev34-cancel-async-tasks-20261027/cancel-async-tasks__WAq9YLK/verifier/retry-summary.json": "5a4ed94af6cf95b6d7d1b056a2e685d698444e8f0424db11e4639384445953c3" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/cobol-modernization", + "task_ref": "sha256:e03aa03965eee48c9f772d566ff9bcd741fceeac9693123f82a13fdd46be0d88", + "job_name": "pi-flash-dev34-cobol-modernization-20261027", + "trial": "cobol-modernization__uGQXih9", + "arm": "pi", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-cobol-modernization-20261027.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "legacy_normal_completion", + "process_supervision": null, + "pytest": { + "passed": 3, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793283929.4077954, + "stop": 1793283929.4950545 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 33.88298039798974 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T14:17:17.309946Z", + "finished_at": "2026-10-29T14:17:20.225286Z" + }, + "agent_setup": { + "started_at": "2026-10-29T14:17:20.225346Z", + "finished_at": "2026-10-29T14:17:44.062978Z" + }, + "agent_execution": { + "started_at": "2026-10-29T14:17:44.063167Z", + "finished_at": "2026-10-29T14:24:54.306348Z" + }, + "verifier": { + "started_at": "2026-10-29T14:24:54.533292Z", + "finished_at": "2026-10-29T14:25:29.582273Z" + } + }, + "agent_seconds": 430.243181, + "usage": { + "n_input_tokens": 2173417, + "n_cache_tokens": 2140416, + "n_output_tokens": 47347, + "cost_usd": 0.079559196, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 57, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 617, + "tool_calls": 59, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/result.json": "7f9ac5e0d92e065f66c1a58229952a9c48a2a61d016cb6e559b6238ec6b6a1fd", + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/manifest.json": "29df150affe475379b4d87689a5367d5b4bdba3db50a086ca2eea65741d48abe", + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027-input/config.json": "074e443d06eec327ea4ade6ef74dceb808c14309c37dafd11e6d34120ad16ad5", + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/agent/pi.txt": "e309667ca4a8a464ee43d8e372fe0f2cab1426cad8cae6377ee0043c53960430", + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/test-stdout.txt": "db03a19762d0964bfaad857e3e7a8927a22808260d567dcbbb209218684a9a59", + "jobs/runtime-source/jobs/pi-flash-dev34-cobol-modernization-20261027/cobol-modernization__uGQXih9/verifier/retry-summary.json": "de8a4c3cd24a67a0bc8f059c044c0dc9be08eab11beb943d9d480d01d79ec1a6" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/db-wal-recovery", + "task_ref": "sha256:07e9193513bfce038b9f9d84962ecc85683eee0a642d14ee6fb7104b086ddab4", + "job_name": "pi-flash-dev34-db-wal-recovery-20261027", + "trial": "db-wal-recovery__J9YieXH", + "arm": "pi", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-db-wal-recovery-20261027.json", + "native_reward": 0.0, + "native_exception_type": "AgentTimeoutError", + "standard_summary_status": "execution_error", + "standard_summary_rewards": null, + "analytical_classification": "unscored_unverified_timeout_boundary", + "analytical_score": null, + "boundary_status": "unverified", + "process_supervision": null, + "pytest": { + "passed": 0, + "failed": 7, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 7, + "passed": 0, + "failed": 7, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793284943.6281073, + "stop": 1793284943.689976 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 35.44820864900248 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T14:26:22.619139Z", + "finished_at": "2026-10-29T14:26:24.740420Z" + }, + "agent_setup": { + "started_at": "2026-10-29T14:26:24.740463Z", + "finished_at": "2026-10-29T14:26:49.171281Z" + }, + "agent_execution": { + "started_at": "2026-10-29T14:26:49.171477Z", + "finished_at": "2026-10-29T14:41:47.862235Z" + }, + "verifier": { + "started_at": "2026-10-29T14:41:48.136670Z", + "finished_at": "2026-10-29T14:42:23.786394Z" + } + }, + "agent_seconds": 898.690758, + "usage": { + "n_input_tokens": 2362465, + "n_cache_tokens": 2286848, + "n_output_tokens": 33459, + "cost_usd": 0.07655698799999999, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 44, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": true, + "audit_cost_is_partial": true + }, + "operational_metadata": { + "event_count": 600, + "tool_calls": 63, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": true, + "credential_tool_call_ids": [ + "call_00_b6rpxUNwvLolIP558fWs7138" + ], + "verifier_log_directory_probe": true + }, + "diagnosis": "Agent timeout without a confirmed stop acknowledgement. Official tests reported missing /app/recovered.json (0 passed, 7 failed), but the frozen boundary is unverified and the raw zero is unscored.", + "evidence_sha256": { + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/result.json": "1318b9f55b8ca84b5c8067ba5a36ee2d7b953acb550f3e01062d2612090a0755", + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/manifest.json": "cafa7c5dbbe442c8498e20b5a83ac803ab3eb45615c5e45fc2719f7bbe001757", + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027-input/config.json": "1372342a226701f3df82b7076fb65492eb0c4ae74f357cfd85d4da1287bdef7c", + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/agent/pi.txt": "fb60c26f364cb61eff8742cf53feb87feb9743f73c4afdb06e3314b343aed3cf", + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/test-stdout.txt": "56b743023c01a907fc584d908e57dbc2cfc001e4db337071149ef956176fdd0c", + "jobs/runtime-source/jobs/pi-flash-dev34-db-wal-recovery-20261027/db-wal-recovery__J9YieXH/verifier/retry-summary.json": "8183af77239c947125f446f27893ba20d07a521819592bde99f1bd0ef978e2e1" + }, + "historical_nanopy_raw_reward": 0.0 + }, + { + "task": "terminal-bench/fix-code-vulnerability", + "task_ref": "sha256:d31348aa16b533a15f013420e4d9726dc529e6086adc7d6d48067d22cc18fe71", + "job_name": "pi-flash-dev34-fix-code-vulnerability-20261027", + "trial": "fix-code-vulnerability__4Dqvu3Q", + "arm": "pi", + "source_ref": "d6ed70a5d6a1c5b0f9b6c9d20b4a1c2b49153f67", + "native_root": "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-fix-code-vulnerability-20261027.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "legacy_normal_completion", + "process_supervision": null, + "pytest": { + "passed": 6, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 6, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793284057.893779, + "stop": 1793284057.9142745 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 13.691956608003238 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T14:26:22.836061Z", + "finished_at": "2026-10-29T14:26:24.960650Z" + }, + "agent_setup": { + "started_at": "2026-10-29T14:26:24.960693Z", + "finished_at": "2026-10-29T14:26:55.642868Z" + }, + "agent_execution": { + "started_at": "2026-10-29T14:26:55.643070Z", + "finished_at": "2026-10-29T14:27:23.875621Z" + }, + "verifier": { + "started_at": "2026-10-29T14:27:24.115322Z", + "finished_at": "2026-10-29T14:27:38.001705Z" + } + }, + "agent_seconds": 28.232551, + "usage": { + "n_input_tokens": 67870, + "n_cache_tokens": 52864, + "n_output_tokens": 1613, + "cost_usd": 0.006754584, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 9, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 120, + "tool_calls": 12, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/result.json": "f69a07a3a1588d680942a9aac2f9fb821906f9fe2d08be24d6798d5b13f06a1d", + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/manifest.json": "5df76d2c959c00eccbdb95c8fa6d782d5e7fa1c0dbb77fab7d6059f1da0751a6", + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027-input/config.json": "1072978ce9e8e38d9c8b0d783976d320f89575178ca10db995414798d19b1ebe", + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/agent/pi.txt": "3fc3420e80c5381a3d6e7482cc1287ae66d17cba4b1b736984a8c5fad41c4c1b", + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/test-stdout.txt": "c9cbb5f9e71ab4aa58a3820831cc3c3f83ab08ca508cfa6b5f936482cffd6f56", + "jobs/runtime-source/jobs/pi-flash-dev34-fix-code-vulnerability-20261027/fix-code-vulnerability__4Dqvu3Q/verifier/retry-summary.json": "8b5af411420fc2067398787182a042ccabfcd900cd28f055c398ad04732b7cc0" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/git-multibranch", + "task_ref": "sha256:7759a92a373c967dd31535975bad8659410644025673fe62385444fd2605c5fa", + "job_name": "pi-flash-dev34-git-multibranch-supervised-20261029", + "trial": "git-multibranch__dh6TaNP", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-git-multibranch-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 1, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 1, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288241.8617995, + "stop": 1793288260.1516557 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 45.67610606299422 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:35:15.145908Z", + "finished_at": "2026-10-29T15:35:17.290966Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:35:17.291009Z", + "finished_at": "2026-10-29T15:35:33.641192Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:35:33.641314Z", + "finished_at": "2026-10-29T15:36:54.130211Z" + }, + "verifier": { + "started_at": "2026-10-29T15:36:54.366811Z", + "finished_at": "2026-10-29T15:37:40.235277Z" + } + }, + "agent_seconds": 80.488897, + "usage": { + "n_input_tokens": 153375, + "n_cache_tokens": 125056, + "n_output_tokens": 11489, + "cost_usd": 0.023032836, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 12, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 190, + "tool_calls": 18, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": true + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/result.json": "81755b70671e579c34177c69887e976560af8185db0da1b248bccc7c2a49db1b", + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/manifest.json": "3adde491396731dc6324ea9229eccfbcfd29c206f8fcfe74278d1abf899fb20c", + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029-input/config.json": "d3a7eafa2718d6de07c7e1578c97e2d32564b943c228207b659ac624d3272a18", + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/pi.txt": "918ef5e9fccacf78f64001583e163cd851a1992ff99c8afc590fa25fe6bd5a49", + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/test-stdout.txt": "79b4b15c050bee004a2e5d6ab18d14164da2d11229e144f7b0c75c0e64f2b73e", + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/verifier/retry-summary.json": "05acd31af122723c1f5b403545f619b4c81caaf4ca537f20806e18eac76bf181", + "jobs/supervised-source/jobs/pi-flash-dev34-git-multibranch-supervised-20261029/git-multibranch__dh6TaNP/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/kv-store-grpc", + "task_ref": "sha256:973c5d4c111fb61a344457936f1c36400acd2d9e44389e7b319586fe23a7a307", + "job_name": "pi-flash-dev34-kv-store-grpc-supervised-20261029", + "trial": "kv-store-grpc__Q92ErPw", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-kv-store-grpc-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 7, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 7, + "passed": 7, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288201.606739, + "stop": 1793288201.6681545 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 27.82079537298705 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:35:15.586355Z", + "finished_at": "2026-10-29T15:35:17.652533Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:35:17.652579Z", + "finished_at": "2026-10-29T15:35:37.325062Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:35:37.325200Z", + "finished_at": "2026-10-29T15:36:13.458892Z" + }, + "verifier": { + "started_at": "2026-10-29T15:36:13.697969Z", + "finished_at": "2026-10-29T15:36:41.758858Z" + } + }, + "agent_seconds": 36.133692, + "usage": { + "n_input_tokens": 24375, + "n_cache_tokens": 21888, + "n_output_tokens": 1279, + "cost_usd": 0.0024122279999999998, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 8, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 107, + "tool_calls": 8, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/result.json": "d6ccd8a5b086a93e78f0f2b9469e354c9fb0803057827248e8d23c093a68cd26", + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/manifest.json": "754381e108810b32a76e6a76bfcc235890edf0839c7ed15373f2a889aac25be4", + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029-input/config.json": "658c2fdac5e4cc78c9d1bc84a345a8210e6e7b2c65fc81843db7c7c645774bde", + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/pi.txt": "d609be21182b1ab3065b9c6446b488e241734f852842800e8a3542d059c9c0c2", + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/test-stdout.txt": "23272c762d0632a6e83ec7dcff2e9b33e9c1a7ecb48b8dd907e0e80f18da2e2c", + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/verifier/retry-summary.json": "a10e698b35615d29d9197f68628a013d3b3d05f62c552b05f2b4fcb548ab0d91", + "jobs/supervised-source/jobs/pi-flash-dev34-kv-store-grpc-supervised-20261029/kv-store-grpc__Q92ErPw/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 0.0 + }, + { + "task": "terminal-bench/log-summary-date-ranges", + "task_ref": "sha256:27b074a2f10fff7606e096f3abd8dced418ad8fda0f53d88acbe477f2d9ceaf6", + "job_name": "pi-flash-dev34-log-summary-date-ranges-supervised-20261029", + "trial": "log-summary-date-ranges__CJ2vodJ", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-log-summary-date-ranges-supervised-20261029.json", + "native_reward": 0.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 0.0 + }, + "analytical_classification": "failed", + "analytical_score": 0.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 1, + "failed": 1, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 2, + "passed": 1, + "failed": 1, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288354.1119735, + "stop": 1793288354.1676722 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 18.856935012008762 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:38:18.812146Z", + "finished_at": "2026-10-29T15:38:20.922875Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:38:20.922919Z", + "finished_at": "2026-10-29T15:38:41.088699Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:38:41.088849Z", + "finished_at": "2026-10-29T15:38:54.972097Z" + }, + "verifier": { + "started_at": "2026-10-29T15:38:55.195884Z", + "finished_at": "2026-10-29T15:39:14.242727Z" + } + }, + "agent_seconds": 13.883248, + "usage": { + "n_input_tokens": 37557, + "n_cache_tokens": 29952, + "n_output_tokens": 1391, + "cost_usd": 0.004130412, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 5, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 56, + "tool_calls": 4, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "The month-to-date ERROR count was 14160 instead of 4682; one summary/count assertion failed.", + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/result.json": "174cbf66d5d2534ac24703f63abcb6b534e28c0feb24c267c9559829bd763260", + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/manifest.json": "1d2c4e828148a59f2117cb757884e056fe87185faa86a7b537b0c6c53482853a", + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029-input/config.json": "d52dc9abf1c825f913a594f918110770163d343a90716c49b2611df948b7ebc9", + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/pi.txt": "10e3b070ca0349b9a4275bbc3fdca4a1c55e9071494c2655ffd8e6c1959a1b4e", + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/test-stdout.txt": "220818e1eb90f5519d0670fceebf9c0a291fee10cb3cd1f915e5936d27da1aee", + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/verifier/retry-summary.json": "14655e123944a366ec87ec96825f488b6255ad5a2c91da8746be45ce6a3a2410", + "jobs/supervised-source/jobs/pi-flash-dev34-log-summary-date-ranges-supervised-20261029/log-summary-date-ranges__CJ2vodJ/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/merge-diff-arc-agi-task", + "task_ref": "sha256:6aab6511a5344ce87698293bb1ce4cc51d9a45f1ad9f0c075d2a83197b36727d", + "job_name": "pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029", + "trial": "merge-diff-arc-agi-task__iiAEYRk", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 5, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 5, + "passed": 5, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288453.6773093, + "stop": 1793288453.722798 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 55.00660880099167 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:38:17.911408Z", + "finished_at": "2026-10-29T15:38:20.118734Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:38:20.118778Z", + "finished_at": "2026-10-29T15:38:41.750592Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:38:41.750786Z", + "finished_at": "2026-10-29T15:39:58.834239Z" + }, + "verifier": { + "started_at": "2026-10-29T15:39:59.070453Z", + "finished_at": "2026-10-29T15:40:53.806929Z" + } + }, + "agent_seconds": 77.083453, + "usage": { + "n_input_tokens": 113161, + "n_cache_tokens": 107392, + "n_output_tokens": 10099, + "cost_usd": 0.014493852, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 13, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 165, + "tool_calls": 16, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/result.json": "c5175b21652aacf209836348c5e61f0c3e32c157a591874e2f87c0ee897f7393", + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/manifest.json": "34e505ebc87479b6dfe2afcd608430dde357ff39332f7741ae20b4624c7dd3d6", + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029-input/config.json": "1ecaca7f34984ef20966a66f32194a25cf09fc2e2a40c3cda5a0ddbfeedbdecd", + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/pi.txt": "ffe0cae91c5440baa645a58ee336f87b1d24db586e4dd97f2e3132e18f48a281", + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/test-stdout.txt": "72e16bf9633de796c88a7a1bc3e611f9365e59a5bb14fb9dbe1ef1ec32b85bef", + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/verifier/retry-summary.json": "13ec54081d2e0bc96bd9eaca96bb5d5e16654c69e198bb90a8cb83441da073c4", + "jobs/supervised-source/jobs/pi-flash-dev34-merge-diff-arc-agi-task-supervised-20261029/merge-diff-arc-agi-task__iiAEYRk/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/model-extraction-relu-logits", + "task_ref": "sha256:1ae5045ad68b5d34c3398b612066a07c4a08b6dc330d28868ec4021e17c94b17", + "job_name": "pi-flash-dev34-model-extraction-relu-logits-supervised-20261029", + "trial": "model-extraction-relu-logits__fDX7jio", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029.json", + "native_reward": 0.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 0.0 + }, + "analytical_classification": "failed", + "analytical_score": 0.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 0, + "failed": 1, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 0, + "failed": 1, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288720.8802168, + "stop": 1793288734.6688912 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 45.84750222500588 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:41:28.869312Z", + "finished_at": "2026-10-29T15:41:31.085473Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:41:31.085515Z", + "finished_at": "2026-10-29T15:41:52.107232Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:41:52.107439Z", + "finished_at": "2026-10-29T15:44:49.071742Z" + }, + "verifier": { + "started_at": "2026-10-29T15:44:49.310784Z", + "finished_at": "2026-10-29T15:45:34.742848Z" + } + }, + "agent_seconds": 176.964303, + "usage": { + "n_input_tokens": 278816, + "n_cache_tokens": 268800, + "n_output_tokens": 21905, + "cost_usd": 0.030903599999999996, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 15, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 159, + "tool_calls": 15, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "The recovered matrix did not match reference row 24.", + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/result.json": "58c7bfebd385b768fc80361b9312d4610dcda43bb8fa52a39f356e13af059370", + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/manifest.json": "5f6340022a2bc1ecf0f9fcb410a3a290e9be9d8382fc442cd99b933ff29ec51a", + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029-input/config.json": "7215b0da05d3e8b922773d56ba4acd479feeedafe630013a88790d923fc8d2bd", + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/pi.txt": "a4aa2df98a59efe892a8fff767669937902579d35c759f909f7f1daf753e703f", + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/test-stdout.txt": "3eccc41a5675886aaa7462692b3680678831b286d1d0af59c3e5f20fa9a306fd", + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/verifier/retry-summary.json": "f05f017f4954a9c8c73c4bb0f851ea35a06b71792b5d2b5ba7fb93c5362a108f", + "jobs/supervised-source/jobs/pi-flash-dev34-model-extraction-relu-logits-supervised-20261029/model-extraction-relu-logits__fDX7jio/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/multi-source-data-merger", + "task_ref": "sha256:70367c38732e1beda7b229968a48d60242277c2fa4db91339c3f064c4c230d49", + "job_name": "pi-flash-dev34-multi-source-data-merger-supervised-20261029", + "trial": "multi-source-data-merger__YAw7zpk", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-multi-source-data-merger-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 3, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288593.4572835, + "stop": 1793288594.5737436 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 47.855576438989374 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:41:27.762027Z", + "finished_at": "2026-10-29T15:41:29.967467Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:41:29.967508Z", + "finished_at": "2026-10-29T15:41:52.141635Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:41:52.141763Z", + "finished_at": "2026-10-29T15:42:26.478169Z" + }, + "verifier": { + "started_at": "2026-10-29T15:42:26.705815Z", + "finished_at": "2026-10-29T15:43:14.765950Z" + } + }, + "agent_seconds": 34.336406, + "usage": { + "n_input_tokens": 28333, + "n_cache_tokens": 26112, + "n_output_tokens": 4810, + "cost_usd": 0.006594971999999999, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 6, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 76, + "tool_calls": 7, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/result.json": "f32cb0bb7c0e95039eed52d794d2aa38d0d1ef110c30f48b71a8d4254ead7a7c", + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/manifest.json": "5d6204817bb144202ef40770319b444a95684974dd8bcb6ebba490ff3530efc5", + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029-input/config.json": "d4e4f9cc09913c65a63f6732703a66b9a709d865553f58c1828355b528e4a9b0", + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/pi.txt": "e2f6138c06366bd9c0450f547a29ad0ff67968ec62e1286b1bc2db3d6a1355e9", + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/test-stdout.txt": "9f790b350e552aeb96f0780619d454ea9833de10137949ee439932ebdf6b05e4", + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/verifier/retry-summary.json": "88115986fe6bed6908e1ac51079821b9c4d1a99950f688451e3a193dc70a9d8a", + "jobs/supervised-source/jobs/pi-flash-dev34-multi-source-data-merger-supervised-20261029/multi-source-data-merger__YAw7zpk/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/nginx-request-logging", + "task_ref": "sha256:9d1b8bebd989ea0bc8080c3b159480068caf183c5fd61385868b2574b206e097", + "job_name": "pi-flash-dev34-nginx-request-logging-supervised-20261029", + "trial": "nginx-request-logging__9Nwcxvo", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-nginx-request-logging-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 8, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 8, + "passed": 8, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288869.905689, + "stop": 1793288872.1025999 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 21.2162712749996 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:46:14.772587Z", + "finished_at": "2026-10-29T15:46:16.930282Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:46:16.930349Z", + "finished_at": "2026-10-29T15:46:37.869725Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:46:37.869964Z", + "finished_at": "2026-10-29T15:47:29.955622Z" + }, + "verifier": { + "started_at": "2026-10-29T15:47:30.186559Z", + "finished_at": "2026-10-29T15:47:52.181750Z" + } + }, + "agent_seconds": 52.085658, + "usage": { + "n_input_tokens": 28825, + "n_cache_tokens": 26240, + "n_output_tokens": 2868, + "cost_usd": 0.00437454, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 8, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 104, + "tool_calls": 10, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/result.json": "5a414d38d37c9abe44c0ad6f71beb0b2fd3d7201c3b5c8c9e35e9735b135c401", + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/manifest.json": "59669e92c13d4ebcab091ef371b9727603f67805b15a626db2f92dbe721eaa4b", + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029-input/config.json": "3cb40ad05346d2536b418993ab06a61e3c9dbc5edc05dea08648aba29223d7fb", + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/pi.txt": "47f51ffa0024da64287ee079529a42740a661d11b42a07eb8c8ac80435b01c22", + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/test-stdout.txt": "bc01f8dfdac27e149bf848c8242b2cd96d02f4d434471e5439a2617758b866cd", + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/verifier/retry-summary.json": "9a00e7ad31ef5bcd4a73b63df10137508f65ded14a59225950920c52069296bf", + "jobs/supervised-source/jobs/pi-flash-dev34-nginx-request-logging-supervised-20261029/nginx-request-logging__9Nwcxvo/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/openssl-selfsigned-cert", + "task_ref": "sha256:d4afa2bd2a9ba1420db8d6cfde42ffdb4873ae2d955c35014e8da94444c83302", + "job_name": "pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029", + "trial": "openssl-selfsigned-cert__M9PAeMV", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 6, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 6, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793288832.5340147, + "stop": 1793288832.6853034 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 17.252985710001667 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:46:15.115069Z", + "finished_at": "2026-10-29T15:46:17.192894Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:46:17.192937Z", + "finished_at": "2026-10-29T15:46:38.506269Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:46:38.506411Z", + "finished_at": "2026-10-29T15:46:55.040199Z" + }, + "verifier": { + "started_at": "2026-10-29T15:46:55.274088Z", + "finished_at": "2026-10-29T15:47:12.752118Z" + } + }, + "agent_seconds": 16.533788, + "usage": { + "n_input_tokens": 19814, + "n_cache_tokens": 17664, + "n_output_tokens": 1714, + "cost_usd": 0.002807784, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 7, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 78, + "tool_calls": 7, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/result.json": "21266b563bcbefba84a21e003522a7cccf77cfa1c4620afd7da601a6cf13fa3d", + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/manifest.json": "f9cf5bebc092c639a186533abcbb401043e5ec30357efaaa6e4f74be69377316", + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029-input/config.json": "7ae9dba62e399285cec9a720cfdf7b5d4116d749851654cf96f8702d6312c82b", + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/pi.txt": "79f5b4131e516f91a1d6b61cdf0f2095f4f731a3bb6a14edea8a82b0f51f9f37", + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/test-stdout.txt": "85d46a4db4417a4f11310d4e0013d82592a515ed996ddec67d02478dc6848573", + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/verifier/retry-summary.json": "aa1c3309561fff6ca4baf955ed1cd9495e3d10376e1972dd04ac1bd8a19e6f6f", + "jobs/supervised-source/jobs/pi-flash-dev34-openssl-selfsigned-cert-supervised-20261029/openssl-selfsigned-cert__M9PAeMV/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/pypi-server", + "task_ref": "sha256:1a1e0542f58e2d3362fec17a9bbb98667717d9a4a3e9a4c8413d3150a4fa0ff1", + "job_name": "pi-flash-dev34-pypi-server-supervised-20261029", + "trial": "pypi-server__9RDKNCX", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pypi-server-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 1, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 1, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793289020.3226686, + "stop": 1793289021.3396492 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 22.859247639004025 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:48:24.493359Z", + "finished_at": "2026-10-29T15:48:26.649760Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:48:26.649824Z", + "finished_at": "2026-10-29T15:48:47.824060Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:48:47.824180Z", + "finished_at": "2026-10-29T15:49:58.147916Z" + }, + "verifier": { + "started_at": "2026-10-29T15:49:58.375482Z", + "finished_at": "2026-10-29T15:50:21.425464Z" + } + }, + "agent_seconds": 70.323736, + "usage": { + "n_input_tokens": 58267, + "n_cache_tokens": 53760, + "n_output_tokens": 3160, + "cost_usd": 0.00546666, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 12, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 150, + "tool_calls": 15, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/result.json": "f30b060cdeb4981cfdda2c6cfe949f9ecdbe54c50695765328d915cf2355187c", + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/manifest.json": "28bc2737fa8e588bbba696d401a5a4ccd08a1b121a2498a437ace3a2c5007cf8", + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029-input/config.json": "2cfa8347634c6d4c6b00fbfd664f4fdd56fa169c1536d23335689f55e3cbaa9e", + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/pi.txt": "4479de66aad2c48fb21f66c34b1400e43600b8107c28abf596baab116c9b0dd5", + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/test-stdout.txt": "82a48bfca4a38f372614824d1700815033059e51ec667c7bacebc42055fa16e8", + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/verifier/retry-summary.json": "94dd2600f29cbd6354381d18c87889bcdb141825157f9f68e9e033c08acc2b27", + "jobs/supervised-source/jobs/pi-flash-dev34-pypi-server-supervised-20261029/pypi-server__9RDKNCX/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/pytorch-model-cli", + "task_ref": "sha256:6f11544dcc81a380dc4611902961f5520259c462d9fceb413419d66ab9a96c79", + "job_name": "pi-flash-dev34-pytorch-model-cli-supervised-20261029", + "trial": "pytorch-model-cli__zbEnfLC", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pytorch-model-cli-supervised-20261029.json", + "native_reward": 0.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 0.0 + }, + "analytical_classification": "failed", + "analytical_score": 0.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 5, + "failed": 1, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 5, + "failed": 1, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793290310.3347871, + "stop": 1793290340.0468805 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 753.7782535060105 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T15:48:24.850008Z", + "finished_at": "2026-10-29T15:48:26.929608Z" + }, + "agent_setup": { + "started_at": "2026-10-29T15:48:26.929652Z", + "finished_at": "2026-10-29T15:48:46.183754Z" + }, + "agent_execution": { + "started_at": "2026-10-29T15:48:46.183957Z", + "finished_at": "2026-10-29T15:59:46.903322Z" + }, + "verifier": { + "started_at": "2026-10-29T15:59:47.134211Z", + "finished_at": "2026-10-29T16:12:21.144556Z" + } + }, + "agent_seconds": 660.719365, + "usage": { + "n_input_tokens": 611760, + "n_cache_tokens": 600192, + "n_output_tokens": 28816, + "cost_usd": 0.041650752, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 31, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 378, + "tool_calls": 35, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "The CLI predictions did not match the expected values.", + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/result.json": "cf94167dacab126649a35a465c73ca26a178644c50bdaa178bc080bbb503f923", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/manifest.json": "f6c80d3321729328c1e4c0e676878561129056e0397aebab5d55828343cf6820", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029-input/config.json": "fe47808ada65deac4cf331645fc03f15f78e6f17f75b2ca3890685774660ff56", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/pi.txt": "715d2fd6436c0e9714c91c7ec7ac94621a4867d5dead0f327408576d17c18c56", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/test-stdout.txt": "00c8cc6ef71fecddec8a437d1538cf0c86105cabdb6c2b1ebba51cc344af0b5c", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/verifier/retry-summary.json": "2e7f7867a1f5270be8b276945139c8596770f463ff53dd5a28e92e0404bfaa58", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-cli-supervised-20261029/pytorch-model-cli__zbEnfLC/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/pytorch-model-recovery", + "task_ref": "sha256:2e628841cff93290919172398e573e794f34c95d9382b7425adde4364022decc", + "job_name": "pi-flash-dev34-pytorch-model-recovery-supervised-20261029", + "trial": "pytorch-model-recovery__sv48aHg", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-pytorch-model-recovery-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 5, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 5, + "passed": 5, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793291394.7132595, + "stop": 1793291399.192171 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 804.784374876006 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:13:09.145795Z", + "finished_at": "2026-10-29T16:13:11.305536Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:13:11.305589Z", + "finished_at": "2026-10-29T16:13:36.088226Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:13:36.088367Z", + "finished_at": "2026-10-29T16:16:34.427657Z" + }, + "verifier": { + "started_at": "2026-10-29T16:16:34.700465Z", + "finished_at": "2026-10-29T16:29:59.859722Z" + } + }, + "agent_seconds": 178.33929, + "usage": { + "n_input_tokens": 344156, + "n_cache_tokens": 335488, + "n_output_tokens": 22547, + "cost_usd": 0.031669728, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 18, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 227, + "tool_calls": 20, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/result.json": "34f7528a3ceb49123694b5c7e75299aa9eb5ffd6b14fc5050864da683b859590", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/manifest.json": "50fc2d5e30df0c51581bb2434212d44413f5bc77337a4012e67f04ca6269a57b", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029-input/config.json": "280efb784646e98caa9f841e50f4b0a8d0db2659ca24c0aa019cf79aeca6c4e7", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/pi.txt": "b575c5d73a80df3783ea232e0b6b6b37411d0eb8b25794bc61b3878ce77953d8", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/test-stdout.txt": "24b0e08ac5bea577604b779ea0c8cfb553878e02369fdd8452b3bdf3015f7a30", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/verifier/retry-summary.json": "50321ba1806cffee7e2ecf9efe3c1a096fb48c6b0e35ea337ea98c5b7e331890", + "jobs/supervised-source/jobs/pi-flash-dev34-pytorch-model-recovery-supervised-20261029/pytorch-model-recovery__sv48aHg/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/qemu-alpine-ssh", + "task_ref": "sha256:60b7050b0e0aa51641208cf65766743d340e59575db2c4d2f8628240846c2a28", + "job_name": "pi-flash-dev34-qemu-alpine-ssh-supervised-20261029", + "trial": "qemu-alpine-ssh__5gy8Wvz", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 1, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 1, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793291091.934148, + "stop": 1793291092.261585 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 11.186583331989823 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:13:08.459336Z", + "finished_at": "2026-10-29T16:13:10.600976Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:13:10.601021Z", + "finished_at": "2026-10-29T16:13:45.799815Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:13:45.800012Z", + "finished_at": "2026-10-29T16:24:40.660190Z" + }, + "verifier": { + "started_at": "2026-10-29T16:24:40.910509Z", + "finished_at": "2026-10-29T16:24:52.328008Z" + } + }, + "agent_seconds": 654.860178, + "usage": { + "n_input_tokens": 3105165, + "n_cache_tokens": 3012480, + "n_output_tokens": 27894, + "cost_usd": 0.07935318, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 57, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 966, + "tool_calls": 64, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/result.json": "724a8d49fba4f100da850a5d13b5f2498b02a71f1186ade7c5aadc3a94243e53", + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/manifest.json": "f72aaaba509ee887b0cacd7407d9d2504a45f0de57f1d929b4ae456ac49f30a3", + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029-input/config.json": "a956473611b983bace3ea41d2ce8e932d13378c2646d27ca83ae08fe6836c9f0", + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/pi.txt": "ba9ec194f552f7387b69498884733117b56789c23876a95a107da9157379c2ed", + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/test-stdout.txt": "3668beb7a59e1b205f68dffd189b1ef73505a63045da28e40af3fcbde3baacf3", + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/verifier/retry-summary.json": "c2a8ba6927c83daa1bbc33c260a06ce7c46dbdb719ddfb91c6dd4405631eacbe", + "jobs/supervised-source/jobs/pi-flash-dev34-qemu-alpine-ssh-supervised-20261029/qemu-alpine-ssh__5gy8Wvz/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/query-optimize", + "task_ref": "sha256:169496ea6843cb403b0860132675baf7aa0df0ac8b86221068e27d86395260f4", + "job_name": "pi-flash-dev34-query-optimize-supervised-20261029", + "trial": "query-optimize__kF6mX9U", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-query-optimize-supervised-20261029.json", + "native_reward": 0.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 0.0 + }, + "analytical_classification": "failed", + "analytical_score": 0.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 5, + "failed": 1, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 5, + "failed": 1, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793291797.396989, + "stop": 1793292142.0056736 + }, + "verifier_attempts": { + "timeout_sec": 1800.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 360.3981315790006 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:30:42.750184Z", + "finished_at": "2026-10-29T16:30:44.852623Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:30:44.852666Z", + "finished_at": "2026-10-29T16:31:03.270321Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:31:03.270513Z", + "finished_at": "2026-10-29T16:36:21.245366Z" + }, + "verifier": { + "started_at": "2026-10-29T16:36:21.495179Z", + "finished_at": "2026-10-29T16:42:22.089906Z" + } + }, + "agent_seconds": 317.974853, + "usage": { + "n_input_tokens": 49198, + "n_cache_tokens": 45312, + "n_output_tokens": 3850, + "cost_usd": 0.006057672, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 9, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 108, + "tool_calls": 10, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "The measured solution runtime was 0.8657636019925121 seconds versus a 0.6539265539904591-second reference and exceeded the allowed 1.05 ratio.", + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/result.json": "a27cf55bc68ca117c24e03562facbf12a81d3695cdfaeb59bcbca8059aab29d2", + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/manifest.json": "924695f00b049c287263083780d90cfb1dd3d4166eabe10e08e2247d3589dcf1", + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029-input/config.json": "d29a531bc797c7336b15809edfb2932f002856a2d54aa3dbc2f624bad1ac4d89", + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/pi.txt": "93576a3fecc3b6dd625dc2570ca6da7435cf0076f378f17c8bd974ef79db54a1", + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/test-stdout.txt": "b8caed31f3361102326d883e591ff625ee0f2ce1ae3835eef059c46eed818d06", + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/verifier/retry-summary.json": "edf26eac53ea6fcf5e34e6c67a3b9f173aa4c227d8532eb4de9edeacc6720056", + "jobs/supervised-source/jobs/pi-flash-dev34-query-optimize-supervised-20261029/query-optimize__kF6mX9U/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 0.0 + }, + { + "task": "terminal-bench/regex-log", + "task_ref": "sha256:802c16cfd132e6c457529cb864be5a757c1b23b6cadc57f2d01983cb0110292a", + "job_name": "pi-flash-dev34-regex-log-supervised-20261029", + "trial": "regex-log__oaEBj6S", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-regex-log-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 1, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 1, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793291650.4048774, + "stop": 1793291650.4473019 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 23.864189631989575 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:30:42.079617Z", + "finished_at": "2026-10-29T16:30:44.357411Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:30:44.357456Z", + "finished_at": "2026-10-29T16:31:09.168069Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:31:09.168192Z", + "finished_at": "2026-10-29T16:33:46.214599Z" + }, + "verifier": { + "started_at": "2026-10-29T16:33:46.456880Z", + "finished_at": "2026-10-29T16:34:10.525321Z" + } + }, + "agent_seconds": 157.046407, + "usage": { + "n_input_tokens": 117354, + "n_cache_tokens": 114688, + "n_output_tokens": 25431, + "cost_usd": 0.032005128, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 7, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 71, + "tool_calls": 6, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/result.json": "4adf367274178f551869763e8b0c8e2ccc0893d2d39874eedfc50d1b3120acdb", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/manifest.json": "e479937407626f9a26ae554d2722705d009edf1b24151bbf228526589309b4f4", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029-input/config.json": "78be6b280eeaad2f6a3b4c942696eee1b57075d6181fa8eae3741fdc87fd594c", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/pi.txt": "1b7cc4b4ba2ad4bc78474cae505cdc0020eb6bf53bd11da594ac718a992c1ce6", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/test-stdout.txt": "d9a559b38aa032a33b34351b823962968a1437f318bc93b4c14a5e59c87a1e52", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/verifier/retry-summary.json": "949e120f23fba02920eec7d1dcfb6b8d42fb3f113a6a0bacfcf0d236ac860501", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-log-supervised-20261029/regex-log__oaEBj6S/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/torch-pipeline-parallelism", + "task_ref": "sha256:db605337c749a872cea7b5b413429b3915bb4c3efe0f7875f0c46ce81bd8c4fb", + "job_name": "pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029", + "trial": "torch-pipeline-parallelism__4mz37Tn", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 4, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793293020.802257, + "stop": 1793293117.9499335 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 357.5722898259992 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:42:56.258943Z", + "finished_at": "2026-10-29T16:42:58.326558Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:42:58.326613Z", + "finished_at": "2026-10-29T16:43:18.962138Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:43:18.962263Z", + "finished_at": "2026-10-29T16:52:41.158546Z" + }, + "verifier": { + "started_at": "2026-10-29T16:52:41.398912Z", + "finished_at": "2026-10-29T16:58:39.207620Z" + } + }, + "agent_seconds": 562.196283, + "usage": { + "n_input_tokens": 1673126, + "n_cache_tokens": 1627136, + "n_output_tokens": 30046, + "cost_usd": 0.059615016, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 41, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 515, + "tool_calls": 52, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": true, + "external_benchmark_tool_call_ids": [ + "call_00_CBVxBsUb68Q88C6XTl8I3304", + "call_00_rEGaq3qPvBxpYXIC9kES2675", + "call_01_Hv21U8ck5ZfxxHYvDPK36327", + "call_00_Ycbpw2vtQOSBmiVCwv3N7183", + "call_01_qw6xOTCFgxUjpDLJCNwX7073", + "call_00_3M5lahFKK7wWSq9KicAz2875" + ], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/result.json": "960ff885e63fa133462da85c9cf88fdbcbf3c79be3885527e3262d712f6c401f", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/manifest.json": "37cfd2cf1978f9c3449f3c5f177b8b889994cf071e243a6bbb3e6de1e9fe8b91", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029-input/config.json": "092561620bc3ff8bfe2efcec77ed63a99962242248aff40b77f14cc2c13ac15c", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/pi.txt": "b2bc41b656184603edbc2c894a86215b81fd6fdd5991d0f2f0065fbb7fdf9133", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/test-stdout.txt": "1e2fd14dfdf81ec03547cabc064de5e29d0cd2cd8da991e1a4ff5283cefa5a59", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/verifier/retry-summary.json": "def4d425355167b7ed6f681fbd9073123667afb8f085866c837d315240e75076", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-pipeline-parallelism-supervised-20261029/torch-pipeline-parallelism__4mz37Tn/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 0.0 + }, + { + "task": "terminal-bench/torch-tensor-parallelism", + "task_ref": "sha256:f32ce74a5aeb6638480247ab799fe46127bbee631acdd0921b0f394ec49b3684", + "job_name": "pi-flash-dev34-torch-tensor-parallelism-supervised-20261029", + "trial": "torch-tensor-parallelism__Z79xPBb", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 13, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793292574.0410836, + "stop": 1793292612.9197094 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 324.211396314 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:42:56.025508Z", + "finished_at": "2026-10-29T16:42:58.230206Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:42:58.230276Z", + "finished_at": "2026-10-29T16:43:18.054401Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:43:18.054644Z", + "finished_at": "2026-10-29T16:44:48.838273Z" + }, + "verifier": { + "started_at": "2026-10-29T16:44:49.064563Z", + "finished_at": "2026-10-29T16:50:13.471761Z" + } + }, + "agent_seconds": 90.783629, + "usage": { + "n_input_tokens": 77648, + "n_cache_tokens": 73600, + "n_output_tokens": 12895, + "cost_usd": 0.01713, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 10, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 100, + "tool_calls": 9, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/result.json": "2f728b9db3eba0974022fe8d830895cb5a55e252a734a1d25f3bddf70dbe8db2", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/manifest.json": "8f7f32cac0cc83854951ac709404254254eb34b3d088406f3e437dc43c1e7927", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029-input/config.json": "b559520e2bb2c45ef1ecc262789d0e22942e7d8ad8efdf8877d85517e38bdbca", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/pi.txt": "848b9e05a46052df967ce6d20e2ae444ac14c2fe3a2f29c8048a6d618aa00a87", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/test-stdout.txt": "24e20ffdf267d986308587eb5183202b20f3e4bbdd9060c2688e5a6449e8eae6", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/verifier/retry-summary.json": "5e4f9f49a5a81dea915dd9e07566b1356873f57911d2d5c9bd78a6a293f61cb6", + "jobs/supervised-source/jobs/pi-flash-dev34-torch-tensor-parallelism-supervised-20261029/torch-tensor-parallelism__Z79xPBb/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/write-compressor", + "task_ref": "sha256:d9ddd9a8e925e2c566b37b2492cbf995afecefe58874e4043ef78d7f3c892c7e", + "job_name": "pi-flash-dev34-write-compressor-supervised-20261029", + "trial": "write-compressor__oh2ej7k", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-write-compressor-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 3, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793293503.3404744, + "stop": 1793293503.4706748 + }, + "verifier_attempts": { + "timeout_sec": 900.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 17.21032586399815 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:59:22.442517Z", + "finished_at": "2026-10-29T16:59:24.583154Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:59:24.583196Z", + "finished_at": "2026-10-29T16:59:41.809083Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:59:41.809218Z", + "finished_at": "2026-10-29T17:04:45.920884Z" + }, + "verifier": { + "started_at": "2026-10-29T17:04:46.150536Z", + "finished_at": "2026-10-29T17:05:03.563951Z" + } + }, + "agent_seconds": 304.111666, + "usage": { + "n_input_tokens": 823240, + "n_cache_tokens": 815104, + "n_output_tokens": 47439, + "cost_usd": 0.064258224, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 21, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 228, + "tool_calls": 21, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/result.json": "e82f944631ef759a2dc3f206908effb487aff1c18532933304185235fdbe2523", + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/manifest.json": "99d2cf512cdef16b24b8014a02642fde07c9dc295254b88cb478beaa899cfcda", + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029-input/config.json": "562152781607ee0826aabc277362fca6c0edb3cddf6602d2f0b4e0d541f1ea5f", + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/pi.txt": "964e72fb3b4068a0b45ea83df2ff47883e4ade1292724e3bda1d4b9f7f175a40", + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/test-stdout.txt": "8a706364dc5ab77f58519e1f1c4fa5b65d6a31977b063e876d7e9a04a7113b72", + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/verifier/retry-summary.json": "93b49e6f798e95f4e2b82696b0f6cd594b0328be3c2b536d3d2788b5fa3902ee", + "jobs/supervised-source/jobs/pi-flash-dev34-write-compressor-supervised-20261029/write-compressor__oh2ej7k/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/constraints-scheduling", + "task_ref": "sha256:91dd6f87e1ee508328d5aafbd99a6bd1ccc47bc22671b4aaa4060bda492deb53", + "job_name": "pi-flash-dev34-constraints-scheduling-supervised-20261029", + "trial": "constraints-scheduling__XqZSbiP", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-constraints-scheduling-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 3, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793293221.1078684, + "stop": 1793293221.5010803 + }, + "verifier_attempts": { + "timeout_sec": 1200.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 15.150301875008154 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T16:59:22.442519Z", + "finished_at": "2026-10-29T16:59:24.583556Z" + }, + "agent_setup": { + "started_at": "2026-10-29T16:59:24.583596Z", + "finished_at": "2026-10-29T16:59:42.995006Z" + }, + "agent_execution": { + "started_at": "2026-10-29T16:59:42.995200Z", + "finished_at": "2026-10-29T17:00:05.990293Z" + }, + "verifier": { + "started_at": "2026-10-29T17:00:06.224244Z", + "finished_at": "2026-10-29T17:00:21.571280Z" + } + }, + "agent_seconds": 22.995093, + "usage": { + "n_input_tokens": 19433, + "n_cache_tokens": 16512, + "n_output_tokens": 3564, + "cost_usd": 0.005252171999999999, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 4, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 44, + "tool_calls": 3, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/result.json": "5facd50971ed4abecd885aacc32a44ded8debd1813010bf628278a3bfdfa9e4b", + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/manifest.json": "0acff64ec34570c28cbc51b872e37818174ebfa341f9c64255eb851b9acc3bff", + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029-input/config.json": "8696255e70167eb24e3fe0b181f89bd12fe093848db6808ea0bbb5b784919917", + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/pi.txt": "fa44829a46bc62207103497b888ceeaddb1f52aa1f6edbe44d4b91f93f6579e0", + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/test-stdout.txt": "2d642cd113db961ceddee1759cc8daf06cf0c15ecc4a60cc4170ce596011a1c8", + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/verifier/retry-summary.json": "5737855016ac64a38c7947d0a09fafcf499fcd2defeed10c1f13a1274de7381c", + "jobs/supervised-source/jobs/pi-flash-dev34-constraints-scheduling-supervised-20261029/constraints-scheduling__XqZSbiP/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/custom-memory-heap-crash", + "task_ref": "sha256:61e021e0304c9818a9be2c9173c8a0938ce5d136e95c5422e099f1fdb2dc080e", + "job_name": "pi-flash-dev34-custom-memory-heap-crash-supervised-20261029", + "trial": "custom-memory-heap-crash__UKhh5Pf", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 6, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 6, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793293713.4074707, + "stop": 1793293716.455697 + }, + "verifier_attempts": { + "timeout_sec": 1800.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 18.45516760200553 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T17:06:18.485322Z", + "finished_at": "2026-10-29T17:06:20.701514Z" + }, + "agent_setup": { + "started_at": "2026-10-29T17:06:20.701571Z", + "finished_at": "2026-10-29T17:06:41.406716Z" + }, + "agent_execution": { + "started_at": "2026-10-29T17:06:41.406837Z", + "finished_at": "2026-10-29T17:08:17.620808Z" + }, + "verifier": { + "started_at": "2026-10-29T17:08:17.862082Z", + "finished_at": "2026-10-29T17:08:36.509421Z" + } + }, + "agent_seconds": 96.213971, + "usage": { + "n_input_tokens": 298321, + "n_cache_tokens": 273664, + "n_output_tokens": 10097, + "cost_usd": 0.021155484, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 17, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 276, + "tool_calls": 28, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/result.json": "b93b51681e1e8a69f36f813c5bf39b64f9689c58abe7e6abd64715a5f9893648", + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/manifest.json": "9c0ab477fcc23b995c03b118c97f2ede8d9b2296661da20213850c25803aa41e", + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029-input/config.json": "1bc62edf083d5204f4e804592fdf26bf392258ec080fdbd7068726132c5c0a63", + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/pi.txt": "3e6985e0be6984c0c94503d23011886171a069edc3486ddf6f95d24ba821979d", + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/test-stdout.txt": "dd49fbab1cf0c35f05a52036e29277dd4d11e830580c728a766bb28eed05c8bb", + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/verifier/retry-summary.json": "dda00a29b5ee453eaed55618736007ea4014d5ce0183f92d8dfc835360dab645", + "jobs/supervised-source/jobs/pi-flash-dev34-custom-memory-heap-crash-supervised-20261029/custom-memory-heap-crash__UKhh5Pf/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/dna-assembly", + "task_ref": "sha256:e41a8e94d86019949b08d3b5f88a85f6d943ba0fd85d5e1d5ebb95cb8f66223f", + "job_name": "pi-flash-dev34-dna-assembly-supervised-20261029", + "trial": "dna-assembly__a33iNAA", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-dna-assembly-supervised-20261029.json", + "native_reward": 0.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 0.0 + }, + "analytical_classification": "failed", + "analytical_score": 0.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 0, + "failed": 1, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 0, + "failed": 1, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793294396.7519155, + "stop": 1793294396.810893 + }, + "verifier_attempts": { + "timeout_sec": 1800.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 14.948949562996859 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T17:06:18.507951Z", + "finished_at": "2026-10-29T17:06:20.693912Z" + }, + "agent_setup": { + "started_at": "2026-10-29T17:06:20.693973Z", + "finished_at": "2026-10-29T17:06:58.240673Z" + }, + "agent_execution": { + "started_at": "2026-10-29T17:06:58.240867Z", + "finished_at": "2026-10-29T17:19:41.509719Z" + }, + "verifier": { + "started_at": "2026-10-29T17:19:41.745749Z", + "finished_at": "2026-10-29T17:19:56.884965Z" + } + }, + "agent_seconds": 763.268852, + "usage": { + "n_input_tokens": 2458945, + "n_cache_tokens": 2411392, + "n_output_tokens": 56079, + "cost_usd": 0.096029052, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 51, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 577, + "tool_calls": 57, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "The forward/reverse primer melting temperatures differed by 5.4112089999999995 degrees C, exceeding the 5-degree limit.", + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/result.json": "8b0d37e25e0420b131d1c105b449aa34e921caae9a74fc115271aa0ee1979729", + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/manifest.json": "fd242f7e3d7dcf272f903e2f651b555522ff8fda7b8465f54f2816d2247d8ec5", + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029-input/config.json": "fd1b2e493cf4983544ca2e6f693412568bb1795c5329b9e2c8f079eccc91623e", + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/pi.txt": "e4ceca47f1b1894c95570b5796dd77685869d2bec48cb12de7593f10a7759099", + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/test-stdout.txt": "edcf5638abea37a344092e747053bc314a2f80c447f5e142f88381371bdb4ec5", + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/verifier/retry-summary.json": "c4a0ba2d33e097e744c23315d69866f662116e0360d51e6c3ae9ed5426c5f756", + "jobs/supervised-source/jobs/pi-flash-dev34-dna-assembly-supervised-20261029/dna-assembly__a33iNAA/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/llm-inference-batching-scheduler", + "task_ref": "sha256:a3bf47589118daec5124fe3689b4439802c805b2f4ed9d402a48ae73ca2fea67", + "job_name": "pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029", + "trial": "llm-inference-batching-scheduler__ubFeu2f", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 6, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 6, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793294821.7847266, + "stop": 1793294821.874118 + }, + "verifier_attempts": { + "timeout_sec": 1800.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 16.31038071399962 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T17:20:42.990463Z", + "finished_at": "2026-10-29T17:20:45.188418Z" + }, + "agent_setup": { + "started_at": "2026-10-29T17:20:45.188459Z", + "finished_at": "2026-10-29T17:21:07.450369Z" + }, + "agent_execution": { + "started_at": "2026-10-29T17:21:07.450637Z", + "finished_at": "2026-10-29T17:26:45.197428Z" + }, + "verifier": { + "started_at": "2026-10-29T17:26:45.429973Z", + "finished_at": "2026-10-29T17:27:01.953990Z" + } + }, + "agent_seconds": 337.746791, + "usage": { + "n_input_tokens": 671108, + "n_cache_tokens": 646784, + "n_output_tokens": 31211, + "cost_usd": 0.048631104, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 24, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 255, + "tool_calls": 25, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": true, + "credential_tool_call_ids": [ + "call_00_Wkxt1JlaPXL1Q0XAFVa45672" + ], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/result.json": "11358254845ba02d43b2d3be41ec8b72a268110b4d94be29244636d7b7cf3a82", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/manifest.json": "1d3b21608b6a16219cd05fc58e899f43f8af27f47d6c25ccd5d48a691c549b80", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029-input/config.json": "26d8838951211379f94d51547dac4b90244f7dc74f8a8179b37fa40d72fd5410", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/pi.txt": "4d90e51f4f840f48df1747fa6ae8c7d8ea7f7d2fea399d7c866e0b0c10d0a330", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/test-stdout.txt": "15f4c60f8c924888b2d5e7008aad87993bb6edbf11dc6b54de9f8ec693d6792e", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/verifier/retry-summary.json": "1ddf03e2d3080934b3844b3a63f14466630af002e02da82cd66b7a3c966c8f6e", + "jobs/supervised-source/jobs/pi-flash-dev34-llm-inference-batching-scheduler-supervised-20261029/llm-inference-batching-scheduler__ubFeu2f/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/path-tracing", + "task_ref": "sha256:cf56094c881a488b27e9f204a638a7e78ed7d55e12dc3064108c93357190314c", + "job_name": "pi-flash-dev34-path-tracing-supervised-20261029", + "trial": "path-tracing__QdtVmqq", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-path-tracing-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 5, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 5, + "passed": 5, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793294617.626552, + "stop": 1793294625.7244127 + }, + "verifier_attempts": { + "timeout_sec": 1800.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 28.331578092009295 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T17:20:42.996388Z", + "finished_at": "2026-10-29T17:20:45.186344Z" + }, + "agent_setup": { + "started_at": "2026-10-29T17:20:45.186390Z", + "finished_at": "2026-10-29T17:21:04.015016Z" + }, + "agent_execution": { + "started_at": "2026-10-29T17:21:04.015144Z", + "finished_at": "2026-10-29T17:23:17.065474Z" + }, + "verifier": { + "started_at": "2026-10-29T17:23:17.295907Z", + "finished_at": "2026-10-29T17:23:45.822587Z" + } + }, + "agent_seconds": 133.05033, + "usage": { + "n_input_tokens": 777422, + "n_cache_tokens": 719744, + "n_output_tokens": 16425, + "cost_usd": 0.041331864, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 23, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 250, + "tool_calls": 22, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/result.json": "2148544e350d5478ba70ac6a7e5e1b1e0fb2f17c058bc50550da9216d4dd889f", + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/manifest.json": "f9e1c5c83d4ab2753389dc88b58249597970df0bea7cfa914ea26a869130b47f", + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029-input/config.json": "64764c35b73f57b89e27e49286861defaadb084059eb5f90fd7b77168d964b7c", + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/pi.txt": "16d56fdcf52bebc2c87e966dbb178ac66d61c51ff9944108f105c19d2cd1df01", + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/test-stdout.txt": "b13f927b31a51ad71b1d2080a2ce2a19aed72df32531a67e077091e69ad4f215", + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/verifier/retry-summary.json": "7b09a9f2803c09d83ba55d8336b4ad54f51371be3a16038618b8d138d761823a", + "jobs/supervised-source/jobs/pi-flash-dev34-path-tracing-supervised-20261029/path-tracing__QdtVmqq/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/schemelike-metacircular-eval", + "task_ref": "sha256:58130c2166c3115276dc8592f358e326ff2d81ea852e3d88636c82fd1dff57e6", + "job_name": "pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029", + "trial": "schemelike-metacircular-eval__n3CPpMo", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 1, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 1, + "passed": 1, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793295699.5200686, + "stop": 1793295744.9518137 + }, + "verifier_attempts": { + "timeout_sec": 2400.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 58.27502875799837 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T17:27:41.997301Z", + "finished_at": "2026-10-29T17:27:44.044713Z" + }, + "agent_setup": { + "started_at": "2026-10-29T17:27:44.044757Z", + "finished_at": "2026-10-29T17:28:06.172359Z" + }, + "agent_execution": { + "started_at": "2026-10-29T17:28:06.172479Z", + "finished_at": "2026-10-29T17:41:26.296517Z" + }, + "verifier": { + "started_at": "2026-10-29T17:41:26.545504Z", + "finished_at": "2026-10-29T17:42:25.024815Z" + } + }, + "agent_seconds": 800.124038, + "usage": { + "n_input_tokens": 1460805, + "n_cache_tokens": 1426944, + "n_output_tokens": 40664, + "cost_usd": 0.06751676399999999, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 31, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 388, + "tool_calls": 40, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/result.json": "2c5e4caffbc25203b5983b7509ff39a78009cee60b68409fb0055cc139bb5cb3", + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/manifest.json": "1d3ba56b4dd728666985cbc4c24742c5e3123b9ba580df8fe57d53a23726f9b7", + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029-input/config.json": "340fe0b2c493bb10f391c50f544faeb9571526457c93e825ea479ab4329b9ddd", + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/pi.txt": "f8d730262523fa98c2f9c08c5f533a38ffc5e0513dd438c682b42ea11514811f", + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/test-stdout.txt": "d9db5d0a4415bfd14bb840016d2be0137b26ac8b9a6ee4b8f9d95a09f1dd9ed5", + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/verifier/retry-summary.json": "3c8fe969d29212f2db2dcc35671e4ff1e33a758df69d93590c234a5ac1635f7e", + "jobs/supervised-source/jobs/pi-flash-dev34-schemelike-metacircular-eval-supervised-20261029/schemelike-metacircular-eval__n3CPpMo/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/caffe-cifar-10", + "task_ref": "sha256:7b0045106d7d5af724efe96b610ba64f7893f5c88528401c573c4d47e384e2bf", + "job_name": "pi-flash-dev34-caffe-cifar-10-supervised-20261029", + "trial": "caffe-cifar-10__tdZV692", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-caffe-cifar-10-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": "AgentTimeoutError", + "standard_summary_status": "execution_error", + "standard_summary_rewards": null, + "analytical_classification": "passed_after_confirmed_agent_timeout", + "analytical_score": 1.0, + "boundary_status": "confirmed_cancelled", + "process_supervision": { + "status": "cancelled", + "processes_stopped": true, + "stopped_processes": 13 + }, + "pytest": { + "passed": 6, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 6, + "passed": 6, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793298501.1300986, + "stop": 1793298539.2519102 + }, + "verifier_attempts": { + "timeout_sec": 1200.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 59.61115111199615 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T17:27:41.808482Z", + "finished_at": "2026-10-29T17:27:43.969904Z" + }, + "agent_setup": { + "started_at": "2026-10-29T17:27:43.969947Z", + "finished_at": "2026-10-29T17:27:58.793494Z" + }, + "agent_execution": { + "started_at": "2026-10-29T17:27:58.793632Z", + "finished_at": "2026-10-29T18:27:59.206363Z" + }, + "verifier": { + "started_at": "2026-10-29T18:27:59.503202Z", + "finished_at": "2026-10-29T18:28:59.351022Z" + } + }, + "agent_seconds": 3600.412731, + "usage": { + "n_input_tokens": 1755242, + "n_cache_tokens": 1709824, + "n_output_tokens": 31881, + "cost_usd": 0.062141544, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 53, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": true, + "cost_is_partial": true + }, + "audit_usage_is_partial": true, + "audit_cost_is_partial": true + }, + "operational_metadata": { + "event_count": 664, + "tool_calls": 65, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": "AgentTimeoutError at the native 3600-second limit. Supervision acknowledged cancellation with 13 stopped processes before verification; all 6 official tests passed. Preserve the native exception and execution_error summary alongside analytical score 1.", + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/result.json": "f7dc6cd62f4f9a06293659a85471067e0d749fab717a0a838811c898ea101a92", + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/manifest.json": "8c862da886aa3b3a6f784b91f50a5ef7034c7d3c5d3c8a6e429982cf74da6687", + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029-input/config.json": "0fe88d22fc834020c454f69c8157b732d1617ccce74073b116634097e1f9f159", + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/pi.txt": "5ebd8135080341ace675a23f03d3385019f77e05191172fdbb3dec37bee16981", + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/test-stdout.txt": "9223c44a71f80eb25e1da5b9a823a3fa84d31408337b7712a45e1ccd3647c4ff", + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/verifier/retry-summary.json": "bc74cec712d83796a276dee68ee95df83489dc2ffd76ae380cb3b09529b4eb84", + "jobs/supervised-source/jobs/pi-flash-dev34-caffe-cifar-10-supervised-20261029/caffe-cifar-10__tdZV692/agent/process-supervision.json": "01eab71f63ee2e1ff88b2c1f8dc9b5bb768f267c88f23df412a9ea02c4e5fed8" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/circuit-fibsqrt", + "task_ref": "sha256:9bcffe1054bb33249aa578a9a2a74f3c8cca66b0cb7aa1328233f1d31822aae3", + "job_name": "pi-flash-dev34-circuit-fibsqrt-supervised-20261029", + "trial": "circuit-fibsqrt__mnDzrnD", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-circuit-fibsqrt-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 3, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793298850.585929, + "stop": 1793298851.4181633 + }, + "verifier_attempts": { + "timeout_sec": 3600.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 13.6434142569924 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T18:29:38.566105Z", + "finished_at": "2026-10-29T18:29:40.781576Z" + }, + "agent_setup": { + "started_at": "2026-10-29T18:29:40.781643Z", + "finished_at": "2026-10-29T18:29:58.356008Z" + }, + "agent_execution": { + "started_at": "2026-10-29T18:29:58.356133Z", + "finished_at": "2026-10-29T18:33:57.395128Z" + }, + "verifier": { + "started_at": "2026-10-29T18:33:57.629236Z", + "finished_at": "2026-10-29T18:34:11.485780Z" + } + }, + "agent_seconds": 239.038995, + "usage": { + "n_input_tokens": 275234, + "n_cache_tokens": 262656, + "n_output_tokens": 26623, + "cost_usd": 0.037296935999999996, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 12, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 128, + "tool_calls": 11, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/result.json": "c8d610ab678b91c24ca6086784d4dfef2330ddb4f6389c961683063082cc47a0", + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/manifest.json": "f4434b014eba820d59dbb4e14e52fb0201c54faab7b3ee4ce51e4000dac19dec", + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029-input/config.json": "a2e7a359321a123fa1953373ea53b6fc3b87921ec339f606f441e8b77b243714", + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/pi.txt": "3d4484e8861723c5d2a1255afbbc8f5b2a8d63a45871fd6daf1a189641d7915b", + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/test-stdout.txt": "7ec35520b3c07ad66f80d9d90716c6cfa1297dfddc469c53a405ce2165ecba91", + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/verifier/retry-summary.json": "0dfdd27207b913689972147e73bc1d48356efc199af4a343e27a78de22437f93", + "jobs/supervised-source/jobs/pi-flash-dev34-circuit-fibsqrt-supervised-20261029/circuit-fibsqrt__mnDzrnD/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/mteb-leaderboard", + "task_ref": "sha256:484f6d7008a05b5b8640fc6618a384b8c9447cd76f85416c8a595028d29bff9c", + "job_name": "pi-flash-dev34-mteb-leaderboard-supervised-20261029", + "trial": "mteb-leaderboard__22sRwKk", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-mteb-leaderboard-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 2, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 2, + "passed": 2, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793299274.1293857, + "stop": 1793299274.170302 + }, + "verifier_attempts": { + "timeout_sec": 3600.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 13.599283546995139 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T18:29:38.667633Z", + "finished_at": "2026-10-29T18:29:40.780245Z" + }, + "agent_setup": { + "started_at": "2026-10-29T18:29:40.780307Z", + "finished_at": "2026-10-29T18:29:52.479423Z" + }, + "agent_execution": { + "started_at": "2026-10-29T18:29:52.479648Z", + "finished_at": "2026-10-29T18:41:00.132884Z" + }, + "verifier": { + "started_at": "2026-10-29T18:41:00.430843Z", + "finished_at": "2026-10-29T18:41:14.242750Z" + } + }, + "agent_seconds": 667.653236, + "usage": { + "n_input_tokens": 2685424, + "n_cache_tokens": 2600576, + "n_output_tokens": 22326, + "cost_usd": 0.06784905599999999, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 61, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 1013, + "tool_calls": 76, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/result.json": "235ba97614b25344dbcc206e4ccb52774ef1cf7e4dc8dbb9aa5caba6b7318b7e", + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/manifest.json": "20a97bbcfb54332d8782d695f5b49f719dcda95077c53094d9b5cb28c389d770", + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029-input/config.json": "ded6f45b014a2be3f6332a78455f198be5fb1beabef225342247038f2b1e9a93", + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/pi.txt": "5a2b8ba48a04eb6378ab83bf730add971fc6670cc851e01b13570821daffd0b2", + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/test-stdout.txt": "1367d9e0635a3e4bf7b7167f5f427711e80ec241ce6f01c9265fa1980328c217", + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/verifier/retry-summary.json": "684e04c1382a7d36dd9d4f2986aecd4ca5adcb8d3ab519fb3c7ad47be1998c77", + "jobs/supervised-source/jobs/pi-flash-dev34-mteb-leaderboard-supervised-20261029/mteb-leaderboard__22sRwKk/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + }, + { + "task": "terminal-bench/regex-chess", + "task_ref": "sha256:e763e0ac1c9759081af0a4a82ba51b8cf9ae5485a93de3bbe42d7d344597bd78", + "job_name": "pi-flash-dev34-regex-chess-supervised-20261029", + "trial": "regex-chess__TmQJBXt", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-regex-chess-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 4, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 4, + "passed": 4, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793300972.105137, + "stop": 1793301087.8280725 + }, + "verifier_attempts": { + "timeout_sec": 3600.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 132.72550347200013 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T18:41:54.546449Z", + "finished_at": "2026-10-29T18:41:56.750922Z" + }, + "agent_setup": { + "started_at": "2026-10-29T18:41:56.750967Z", + "finished_at": "2026-10-29T18:42:16.527610Z" + }, + "agent_execution": { + "started_at": "2026-10-29T18:42:16.527803Z", + "finished_at": "2026-10-29T19:09:14.765699Z" + }, + "verifier": { + "started_at": "2026-10-29T19:09:15.003060Z", + "finished_at": "2026-10-29T19:11:27.932787Z" + } + }, + "agent_seconds": 1618.237896, + "usage": { + "n_input_tokens": 2352352, + "n_cache_tokens": 2325632, + "n_output_tokens": 72690, + "cost_usd": 0.109197792, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 36, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 445, + "tool_calls": 35, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/result.json": "8ba32037813aa41478afc3d95bd009ec6dd1ede28cd687503269122575ae1dfd", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/manifest.json": "c5a0266fd97b4eb085276f8a94db0526edf5d9bf1a8c0ce5d45d81299921dc27", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029-input/config.json": "175bc8df37e279761a8a5cd4aad8f366a2e9517d1f5cd3ac4602836608547666", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/pi.txt": "41d577a7678652235bc0417169e7d083a6557fb1d58f82ca727e818f8b1f7c11", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/test-stdout.txt": "5258f81d8857f0b0413236d91def3712b91f8d921b866c5679d0390e344a6b33", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/verifier/retry-summary.json": "70c885ea3232bbf4cc55d881d6f26f4b1bfe37ebb828da774e9a46a0baa38cd2", + "jobs/supervised-source/jobs/pi-flash-dev34-regex-chess-supervised-20261029/regex-chess__TmQJBXt/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 0.0 + }, + { + "task": "terminal-bench/build-pov-ray", + "task_ref": "sha256:4f08dab602fe2e9e2f42051cc6aef684a88cf2822e8b499b3ca19e670759fddb", + "job_name": "pi-flash-dev34-build-pov-ray-supervised-20261029", + "trial": "build-pov-ray__oxrWfXD", + "arm": "pi_supervised", + "source_ref": "29db8d7abbbe5f1660b8fb4d7caeeb0d183dc64d", + "native_root": "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD", + "record": "jobs/pi-deepseek-flash-dev34-20261029-supervision-v7-orchestration/pi-flash-dev34-build-pov-ray-supervised-20261029.json", + "native_reward": 1.0, + "native_exception_type": null, + "standard_summary_status": "scored", + "standard_summary_rewards": { + "reward": 1.0 + }, + "analytical_classification": "passed", + "analytical_score": 1.0, + "boundary_status": "confirmed_completed", + "process_supervision": { + "status": "completed", + "return_code": 0 + }, + "pytest": { + "passed": 3, + "failed": 0, + "errors": 0, + "skipped": 0 + }, + "ctrf_summary": { + "tests": 3, + "passed": 3, + "failed": 0, + "skipped": 0, + "pending": 0, + "other": 0, + "start": 1793299620.8555977, + "stop": 1793299627.520424 + }, + "verifier_attempts": { + "timeout_sec": 12000.0, + "max_attempts": 3, + "retry_dependency_errors": false, + "attempts": [ + { + "attempt": 1, + "status": "completed", + "return_code": 0, + "duration_seconds": 27.708205570001155 + } + ] + }, + "timing": { + "environment_setup": { + "started_at": "2026-10-29T18:41:54.537369Z", + "finished_at": "2026-10-29T18:41:56.740340Z" + }, + "agent_setup": { + "started_at": "2026-10-29T18:41:56.740402Z", + "finished_at": "2026-10-29T18:42:12.106423Z" + }, + "agent_execution": { + "started_at": "2026-10-29T18:42:12.106642Z", + "finished_at": "2026-10-29T18:46:39.523300Z" + }, + "verifier": { + "started_at": "2026-10-29T18:46:39.759879Z", + "finished_at": "2026-10-29T18:47:07.668876Z" + } + }, + "agent_seconds": 267.416658, + "usage": { + "n_input_tokens": 396202, + "n_cache_tokens": 363520, + "n_output_tokens": 6121, + "cost_usd": 0.019330919999999998, + "raw_pi_metadata": { + "version": "1.4.0", + "usage_records": 20, + "missing_usage_records": 0, + "assistant_errors": [], + "cost_is_estimated": true, + "usage_is_partial": false, + "cost_is_partial": false + }, + "audit_usage_is_partial": false, + "audit_cost_is_partial": false + }, + "operational_metadata": { + "event_count": 247, + "tool_calls": 24, + "post_agent_end_messages": 0, + "messages_during_or_after_verifier_start": 0 + }, + "audit_flags": { + "external_benchmark_material": false, + "external_benchmark_tool_call_ids": [], + "provider_key_in_tool_output": false, + "credential_tool_call_ids": [], + "verifier_log_directory_probe": false + }, + "diagnosis": null, + "evidence_sha256": { + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/result.json": "41b225157a865f0a6f8ed41667973a4e850f6e0c952d6cbb2e80cff14403654b", + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/manifest.json": "c296476ec612eeabb11916fc760674c21e41676e60c1960731c84cd26863173d", + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029-input/config.json": "08ca16ead9d0174e6b0535182398464f61524bfffaf87ebf3aacba7456411aab", + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/pi.txt": "5332ecc33747f33cd48d8ab103fdee24ec2bbe1a29f4c3d41b296356819c68be", + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/test-stdout.txt": "e1e92e63509e7e0411e2afe54fb5815be89f3bc0f69549be40b9457be525c745", + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/verifier/retry-summary.json": "08729d1e30969c59bb969a8771ba38e7e6d8ce2a2d5120a6e83452a5cff4d911", + "jobs/supervised-source/jobs/pi-flash-dev34-build-pov-ray-supervised-20261029/build-pov-ray__oxrWfXD/agent/process-supervision.json": "c86e08060391f52429e8b442add8a03983fa0caf8f19eb62ad580208cc0bd7af" + }, + "historical_nanopy_raw_reward": 1.0 + } + ] +} diff --git a/benchmarks/harbor/scripts/cache_image.py b/benchmarks/harbor/scripts/cache_image.py index 062bb1d..dbdf1d2 100644 --- a/benchmarks/harbor/scripts/cache_image.py +++ b/benchmarks/harbor/scripts/cache_image.py @@ -2,7 +2,8 @@ For slow or interrupted Docker pulls, create an OCI archive, then optionally load it into Docker's containerd image store. No layers are edited or rebuilt. -Only single-platform Linux images with SHA256 and gzip layers are supported. +Linux amd64 images with SHA256 and gzip layers are supported. OCI indexes and +Docker manifest lists retain their original bytes while loading that platform. """ import argparse @@ -17,8 +18,9 @@ import re import subprocess import tarfile +import threading import time -from urllib.error import URLError +from urllib.error import HTTPError, URLError from urllib.parse import urlsplit from urllib.request import HTTPRedirectHandler, Request, build_opener @@ -27,6 +29,10 @@ "application/vnd.docker.distribution.manifest.v2+json", "application/vnd.oci.image.manifest.v1+json", ) +INDEX_TYPES = ( + "application/vnd.docker.distribution.manifest.list.v2+json", + "application/vnd.oci.image.index.v1+json", +) LAYER_TYPES = { "application/vnd.docker.image.rootfs.diff.tar.gzip", "application/vnd.oci.image.layer.v1.tar+gzip", @@ -43,6 +49,14 @@ def redirect_request(self, request, response, code, message, headers, url): return redirected +class RegistryHTTPError(RuntimeError): + """Report an HTTP status without leaking signed URLs or request headers.""" + + def __init__(self, status): + self.status = status + super().__init__(f"Registry request failed: HTTP {status}") + + def get(url, headers=None, limit=4 * 1024 * 1024): # Never include exception URLs: blob redirects may contain signed queries. for attempt in range(3): @@ -52,12 +66,55 @@ def get(url, headers=None, limit=4 * 1024 * 1024): if len(data) > limit: raise ValueError("Response exceeds requested size") return r.status, dict(r.headers.items()), data + except HTTPError as error: + if error.code == 401 or attempt == 2: + raise RegistryHTTPError(error.code) from None + time.sleep(attempt + 1) except (URLError, TimeoutError, OSError, IncompleteRead) as error: if attempt == 2: raise RuntimeError(f"Registry request failed: {type(error).__name__}") from None time.sleep(attempt + 1) +class RegistryAuth: + """Refresh anonymous pull tokens during long, parallel range downloads.""" + + def __init__(self, repository): + self.url = "https://auth.docker.io/token?service=registry.docker.io&scope=repository:" + repository + ":pull" + self._token = None + self._expires_at = 0 + self._lock = threading.Lock() + + def token(self, rejected=None): + with self._lock: + if rejected is not None and rejected == self._token: + self._expires_at = 0 + if self._token is None or time.monotonic() >= self._expires_at: + _, _, raw = get(self.url) + data = json.loads(raw) + token = data.get("token") or data.get("access_token") + if not isinstance(token, str) or not token: + raise ValueError("Registry did not return a pull token") + self._token = token + self._expires_at = time.monotonic() + max(1, float(data.get("expires_in", 300)) - 30) + return self._token + + def get(self, url, headers=None, limit=4 * 1024 * 1024): + rejected = None + for attempt in range(2): + token = self.token(rejected) + request_headers = { + "Accept": ", ".join((*MANIFEST_TYPES, *INDEX_TYPES)), **(headers or {}), + "Authorization": "Bearer " + token, + } + try: + return get(url, request_headers, limit) + except RegistryHTTPError as error: + if error.status != 401 or attempt: + raise + rejected = token + + def sha(data): return "sha256:" + hashlib.sha256(data).hexdigest() @@ -80,7 +137,8 @@ def verify(path, descriptor): raise ValueError("Blob SHA256 mismatch") -def fetch_blob(cache, descriptor, url, headers, workers=8, chunk_size=1024 * 1024): +def fetch_blob(cache, descriptor, url, headers, workers=8, chunk_size=1024 * 1024, *, request=None): + request = request or get path = blob_path(cache, descriptor) if path.exists(): verify(path, descriptor) @@ -96,7 +154,7 @@ def fetch(start): if part.exists() and part.stat().st_size == end - start + 1: return part for attempt in range(3): - status, response_headers, data = get( + status, response_headers, data = request( url, {**headers, "Range": f"bytes={start}-{end}"}, end - start + 1, ) response_headers = {k.lower(): v for k, v in response_headers.items()} @@ -146,7 +204,58 @@ def validate_layers(cache, manifest): return config -def write_archive(cache, image, raw_manifest, output): +def native_descriptor(index): + """Select exactly one explicit Linux amd64 child without changing its index.""" + if index.get("schemaVersion") != 2 or index.get("mediaType") not in INDEX_TYPES: + raise ValueError("Expected a schema 2 OCI index or Docker manifest list") + matches = [item for item in index.get("manifests", []) if ( + item.get("platform", {}).get("os") == "linux" + and item.get("platform", {}).get("architecture") == "amd64" + and item.get("platform", {}).get("variant") in {None, "", "v1"} + )] + if len(matches) != 1 or matches[0].get("mediaType") not in MANIFEST_TYPES: + raise ValueError("Image index requires exactly one Linux amd64 image manifest") + blob_path(Path("."), matches[0]) + return matches[0] + + +def native_attestations(index): + subject = native_descriptor(index)["digest"] + matches = [item for item in index.get("manifests", []) if ( + item.get("annotations", {}).get("vnd.docker.reference.type") == "attestation-manifest" + and item.get("annotations", {}).get("vnd.docker.reference.digest") == subject + )] + for item in matches: + if item.get("mediaType") not in MANIFEST_TYPES: + raise ValueError("Unsupported native attestation manifest type") + blob_path(Path("."), item) + return matches + + +def attestation_blobs(raw): + manifest = json.loads(raw) + if (manifest.get("schemaVersion") != 2 or manifest.get("mediaType") not in MANIFEST_TYPES + or any(item.get("mediaType") != "application/vnd.in-toto+json" for item in manifest.get("layers", []))): + raise ValueError("Expected a schema 2 in-toto attestation manifest") + return [manifest["config"], *manifest.get("layers", [])] + + +def fetch_manifest(cache, auth, base, descriptor): + path = blob_path(cache, descriptor) + if path.exists(): + verify(path, descriptor) + return path.read_bytes() + _, headers, raw = auth.get(base + "/manifests/" + descriptor["digest"]) + headers = {k.lower(): v for k, v in headers.items()} + if (len(raw) != descriptor["size"] or sha(raw) != descriptor["digest"] + or headers.get("docker-content-digest") != descriptor["digest"]): + raise ValueError("Image manifest size or digest differs from its index") + path.parent.mkdir(parents=True, exist_ok=True) + path.write_bytes(raw) + return raw + + +def write_archive(cache, image, raw_manifest, output, *, root_index=None): manifest = json.loads(raw_manifest) if manifest.get("schemaVersion") != 2 or manifest.get("mediaType") not in MANIFEST_TYPES: raise ValueError("Only single-platform schema 2 image manifests are supported") @@ -158,8 +267,27 @@ def write_archive(cache, image, raw_manifest, output): manifest_path = blob_path(cache, manifest_descriptor) manifest_path.write_bytes(raw_manifest) descriptors.append(manifest_descriptor) + reference = manifest_descriptor + if root_index is not None: + index = json.loads(root_index) + selected = native_descriptor(index) + if selected["digest"] != sha(raw_manifest) or selected["size"] != len(raw_manifest): + raise ValueError("Selected image manifest differs from its original index") + reference = {"mediaType": index["mediaType"], "size": len(root_index), "digest": sha(root_index)} + blob_path(cache, reference).write_bytes(root_index) + descriptors.append(reference) + # Docker's platform filter also retains attestations whose subject is + # the selected image. Their original config and in-toto payloads are + # metadata, not executable filesystem layers. + for attestation in native_attestations(index): + path = blob_path(cache, attestation) + verify(path, attestation) + metadata = attestation_blobs(path.read_bytes()) + for item in metadata: + verify(blob_path(cache, item), item) + descriptors.extend([attestation, *metadata]) index = {"schemaVersion": 2, "manifests": [{ - **manifest_descriptor, "annotations": { + **reference, "annotations": { "io.containerd.image.name": "docker.io/" + image, "org.opencontainers.image.ref.name": image.rsplit(":", 1)[1], }, @@ -187,32 +315,45 @@ def cache_image(image, cache, workers=8, load=False): raise ValueError("Expected a public Docker Hub owner/repository:tag") cache.mkdir(parents=True, exist_ok=True) repo, tag = image.rsplit(":", 1) - _, _, raw_token = get("https://auth.docker.io/token?service=registry.docker.io&scope=repository:" + repo + ":pull") - headers = {"Authorization": "Bearer " + json.loads(raw_token)["token"], "Accept": ", ".join(MANIFEST_TYPES)} + auth = RegistryAuth(repo) base = "https://registry-1.docker.io/v2/" + repo - _, response_headers, raw = get(base + "/manifests/" + tag, headers) + _, response_headers, raw = auth.get(base + "/manifests/" + tag) response_headers = {k.lower(): v for k, v in response_headers.items()} if response_headers.get("docker-content-digest") != sha(raw): raise ValueError("Registry manifest digest mismatch") manifest = json.loads(raw) + root_index = None + root_digest = sha(raw) + if manifest.get("mediaType") in INDEX_TYPES: + root_index = raw + selected = native_descriptor(manifest) + raw = fetch_manifest(cache, auth, base, selected) + manifest = json.loads(raw) if manifest.get("mediaType") not in MANIFEST_TYPES or manifest.get("schemaVersion") != 2: - raise ValueError("Only single-platform schema 2 image manifests are supported") - lock = {"image": image, "manifest_digest": sha(raw), "config_digest": manifest["config"]["digest"]} + raise ValueError("Only schema 2 image manifests and native-platform indexes are supported") + lock = {"image": image, "manifest_digest": root_digest, "config_digest": manifest["config"]["digest"]} lock_path = cache / (image.replace("/", "_").replace(":", "_") + ".json") if lock_path.exists() and json.loads(lock_path.read_text()) != lock: raise ValueError("Image tag changed since the first cache request") lock_path.write_text(json.dumps(lock, indent=2) + "\n") print(json.dumps(lock), flush=True) + if root_index is not None: + for item in native_attestations(json.loads(root_index)): + metadata = fetch_manifest(cache, auth, base, item) + for node in attestation_blobs(metadata): + fetch_blob(cache, node, base + "/blobs/" + node["digest"], {}, workers, request=auth.get) + print(json.dumps({"verified_native_attestation": item["digest"]}), flush=True) for descriptor in [manifest["config"], *manifest["layers"]]: - fetch_blob(cache, descriptor, base + "/blobs/" + descriptor["digest"], headers, workers) + fetch_blob(cache, descriptor, base + "/blobs/" + descriptor["digest"], {}, workers, request=auth.get) print(json.dumps({"verified_blob": descriptor["digest"], "bytes": descriptor["size"]}), flush=True) output = cache / (lock_path.stem + ".oci.tar") - write_archive(cache, image, raw, output) + write_archive(cache, image, raw, output, root_index=root_index) if load: driver = subprocess.check_output(["docker", "info", "--format", "{{json .DriverStatus}}"], text=True) if "io.containerd.snapshotter.v1" not in driver: raise ValueError("OCI import requires Docker's containerd image store") - subprocess.run(["docker", "load", "--quiet", "--input", str(output)], check=True) + platform = ["--platform", "linux/amd64"] if root_index is not None else [] + subprocess.run(["docker", "load", "--quiet", *platform, "--input", str(output)], check=True) installed = json.loads(subprocess.check_output(["docker", "image", "inspect", image], text=True))[0] if not any(ref.endswith("@" + lock["manifest_digest"]) for ref in installed["RepoDigests"]): raise ValueError("Imported image does not retain the original manifest digest") diff --git a/benchmarks/harbor/scripts/run_comparison.py b/benchmarks/harbor/scripts/run_comparison.py index 87ff60e..57891f7 100644 --- a/benchmarks/harbor/scripts/run_comparison.py +++ b/benchmarks/harbor/scripts/run_comparison.py @@ -11,7 +11,94 @@ import hashlib import json from pathlib import Path +import shutil import subprocess +import sys + + +class BatchImages: + """Restore pinned images per batch and evict only this run's new cache.""" + + def __init__(self, root, plan, record): + self.policy = plan.get("batch_images") + self.record = record + if not self.policy: + return + evidence = json.loads((root / self.policy["manifest"]).read_text()) + self.images = {item["task"]: item for item in evidence["images"]} + inventory = record / "initial-docker-images.json" + if not inventory.exists(): + ids = subprocess.check_output(["docker", "image", "ls", "--quiet", "--no-trunc"], text=True).splitlines() + inventory.write_text(json.dumps(sorted(set(ids)), indent=2) + "\n") + self.protected = set(json.loads(inventory.read_text())) + self.root = root + + def inspect(self, image): + result = subprocess.run(["docker", "image", "inspect", image], text=True, capture_output=True) + if result.returncode: + if "No such image" in result.stderr or "No such object" in result.stderr: + return None + raise RuntimeError(f"Cannot inspect Docker image: {result.stderr}") + return json.loads(result.stdout)[0] + + def entries(self, jobs): + result = {} + for job in jobs: + config = json.loads((self.root / job["config"]).read_text()) + for task in config["tasks"]: + entry = self.images[task["name"]] + result[entry["image"]] = entry + return list(result.values()) + + def event(self, action, entry): + with (self.record / "image-cache.jsonl").open("a") as stream: + stream.write(json.dumps({"at": datetime.now(timezone.utc).isoformat(), "action": action, "image": entry["image"], "image_id": entry["image_id"]}) + "\n") + + def prepare(self, jobs): + if not self.policy: + return + free = shutil.disk_usage(self.policy["storage_path"]).free + if free < self.policy["minimum_free_disk_bytes"]: + raise RuntimeError("Insufficient Docker storage before image restoration") + for entry in self.entries(jobs): + installed = self.inspect(entry["image"]) + if installed is None: + reference = entry["repo_digests"][0] + with (self.record / "image-downloads.log").open("a") as log: + if self.policy.get("cache_dir"): + subprocess.run([ + sys.executable, str(self.root / "benchmarks/harbor/scripts/cache_image.py"), + entry["image"], "--cache", self.policy["cache_dir"], "--load", + ], check=True, stdout=log, stderr=subprocess.STDOUT) + else: + subprocess.run(["docker", "pull", reference], check=True, stdout=log, stderr=subprocess.STDOUT) + subprocess.run(["docker", "tag", reference, entry["image"]], check=True, stdout=log, stderr=subprocess.STDOUT) + installed = self.inspect(entry["image"]) + self.event("restored", entry) + if (installed["Id"] != entry["image_id"] + or not set(entry["repo_digests"]) <= set(installed["RepoDigests"])): + raise ValueError(f"Docker image differs from registration: {entry['image']}") + self.event("verified", entry) + + def release(self, jobs): + if not self.policy: + return + for entry in self.entries(jobs): + if entry["image_id"] in self.protected: + continue + installed = self.inspect(entry["image"]) + if installed is None: + continue + if installed["Id"] != entry["image_id"]: + raise ValueError("Refusing to evict a different image") + # No force: Docker preserves images referenced by any remaining + # container. Keep both successful and failed task files/logs. + with (self.record / "image-downloads.log").open("a") as log: + for reference in [entry["image"], *entry["repo_digests"]]: + if self.inspect(reference) is None: + continue + subprocess.run(["docker", "image", "rm", reference], check=True, stdout=log, stderr=subprocess.STDOUT) + self.event("evicted", entry) def digest(path): @@ -57,11 +144,16 @@ def validate_plan(root, plan_path, plan): ["git", "diff", "--exit-code", "HEAD", "--", "src", "benchmarks/harbor/src"], cwd=source, check=True, stdout=subprocess.DEVNULL, ) - expected = plan_path.parent / arm["prompt_file"] - if digest(expected) != arm["prompt_sha256"]: - raise ValueError(f"Prompt file changed: {expected}") - if prompt_from_source(source / "src/nanopycodeagent/agent.py") != expected.read_text(): - raise ValueError(f"Source prompt does not match the plan: {source}") + if arm.get("agent") == "pi": + from harbor_adapter.pi import distribution_digest, distribution_files + if distribution_digest(distribution_files(Path(arm["distribution_path"]))) != arm["distribution_sha256"]: + raise ValueError("Registered Pi distribution changed") + else: + expected = plan_path.parent / arm["prompt_file"] + if digest(expected) != arm["prompt_sha256"]: + raise ValueError(f"Prompt file changed: {expected}") + if prompt_from_source(source / "src/nanopycodeagent/agent.py") != expected.read_text(): + raise ValueError(f"Source prompt does not match the plan: {source}") for task in plan["tasks"]: cache = Path.home() / ".cache/harbor/tasks/packages" / task["name"] / task["ref"].split(":")[1] for name, expected in task["task_file_sha256"].items(): @@ -84,6 +176,9 @@ def run_job(root, plan, job, record, install_only): if previous["status"] == "finished": return previous raise RuntimeError(f"Previously started job requires inspection; refusing to retry: {status_path}") + minimum = plan.get("minimum_free_disk_bytes", 0) + if shutil.disk_usage(root).free < minimum: + raise RuntimeError(f"Insufficient free disk space: this plan requires {minimum} bytes") source = root / plan["arms"][job["arm"]]["source_root"] command = [ "uv", "run", "--project", "benchmarks/harbor", "python", "-m", "harbor_adapter", @@ -108,6 +203,11 @@ def run_job(root, plan, job, record, install_only): return state +def finished_job(record, job): + path = record / (job["job_name"] + ".json") + return path.exists() and json.loads(path.read_text()).get("status") == "finished" + + def preflight_passed(states, expected_counts): return bool(states) and len(states) == len(expected_counts) and all( s.get("exit_code") == 0 @@ -166,23 +266,37 @@ def checked(path, expected_hash): if config.get("install_only") is not True or manifest["commit"] != arm["source_ref"]: raise ValueError("Accepted installation used a different mode or source revision") expected_sources = {} - for prefix, package in (("src", "nanopycodeagent"), ("benchmarks/harbor/src", "harbor_adapter")): + packages = [("benchmarks/harbor/src", "harbor_adapter")] + if arm.get("agent") != "pi": + packages.append(("src", "nanopycodeagent")) + for prefix, package in packages: for path in (source / prefix / package).rglob("*.py"): expected_sources[str(path.relative_to(source / prefix))] = digest(path) if not expected_sources or manifest["source_sha256"] != expected_sources: raise ValueError("Accepted installation source snapshot differs from the frozen source") - wheel = Path(result["config"]["agent"]["kwargs"]["wheel_path"]).resolve() - if not wheel.is_relative_to((inputs / "dist").resolve()): - raise ValueError("Accepted installation wheel must belong to its recorded inputs") - if not (digest(wheel) == manifest["wheel_sha256"] == bootstrap["wheel_sha256"]): - raise ValueError("Accepted installation wheel digest differs from its provenance") + if arm.get("agent") == "pi": + pin = arm["distribution_sha256"] + pi_input, = manifest["pi_inputs"] + if not (pi_input["distribution_sha256"] == bootstrap["distribution_sha256"] == pin): + raise ValueError("Accepted Pi distribution differs from its provenance") + from harbor_adapter.pi import distribution_digest + if distribution_digest(pi_input["files"]) != pin: + raise ValueError("Accepted Pi file manifest differs from its provenance") + if digest(inputs / "pi-models.json") != pi_input["model_config_sha256"]: + raise ValueError("Accepted Pi model config differs from its provenance") + else: + wheel = Path(result["config"]["agent"]["kwargs"]["wheel_path"]).resolve() + if not wheel.is_relative_to((inputs / "dist").resolve()): + raise ValueError("Accepted installation wheel must belong to its recorded inputs") + if not (digest(wheel) == manifest["wheel_sha256"] == bootstrap["wheel_sha256"]): + raise ValueError("Accepted installation wheel digest differs from its provenance") return True def main(): parser = argparse.ArgumentParser(description=__doc__) parser.add_argument("--plan", type=Path, required=True) - parser.add_argument("--phase", choices=["preflight", "model"], required=True) + parser.add_argument("--phase", choices=["preflight", "model", "all"], required=True) args = parser.parse_args() root, plan_path = Path.cwd(), args.plan.resolve() plan = json.loads(plan_path.read_text()) @@ -201,13 +315,24 @@ def main(): raise ValueError("The registered plan is immutable") else: registration.write_text(json.dumps(current, indent=2) + "\n") + images = BatchImages(root, plan, record) expected_counts = [ len(json.loads((root / job["config"]).read_text())["tasks"]) for job in plan["preflight"] ] - if args.phase == "preflight": - states = [run_job(root, plan, j, record, True) for j in plan["preflight"]] - return 0 if accepted or preflight_passed(states, expected_counts) else 1 + if args.phase in {"preflight", "all"}: + states = [] + for job in plan["preflight"]: + completed = finished_job(record, job) + if not completed: + images.prepare([job]) + states.append(run_job(root, plan, job, record, True)) + if not completed: + images.release([job]) + if not accepted and not preflight_passed(states, expected_counts): + return 1 + if args.phase == "preflight": + return 0 if not accepted: preflight = [json.loads((record / (j["job_name"] + ".json")).read_text()) for j in plan["preflight"]] if not preflight_passed(preflight, expected_counts): @@ -216,8 +341,11 @@ def main(): with ThreadPoolExecutor(max_workers=2) as pool: for offset in range(0, len(plan["queue"]), 2): batch = plan["queue"][offset:offset + 2] + pending = [job for job in batch if not finished_job(record, job)] + images.prepare(pending) futures = [pool.submit(run_job, root, plan, job, record, False) for job in batch] states.extend(f.result() for f in futures) + images.release(pending) return int(any(s["exit_code"] != 0 for s in states)) diff --git a/benchmarks/harbor/src/harbor_adapter/__init__.py b/benchmarks/harbor/src/harbor_adapter/__init__.py index 5e8c83c..b27118f 100644 --- a/benchmarks/harbor/src/harbor_adapter/__init__.py +++ b/benchmarks/harbor/src/harbor_adapter/__init__.py @@ -1,6 +1,7 @@ """Public import for the repository-local nanoPyCodeAgent Harbor adapter.""" from .adapter import NanoPyCodeAgent +from .pi import PiCodeAgent from .verifier import RetryingVerifier -__all__ = ["NanoPyCodeAgent", "RetryingVerifier"] +__all__ = ["NanoPyCodeAgent", "PiCodeAgent", "RetryingVerifier"] diff --git a/benchmarks/harbor/src/harbor_adapter/__main__.py b/benchmarks/harbor/src/harbor_adapter/__main__.py index 11eed91..57baead 100644 --- a/benchmarks/harbor/src/harbor_adapter/__main__.py +++ b/benchmarks/harbor/src/harbor_adapter/__main__.py @@ -2,6 +2,7 @@ import argparse from datetime import datetime, timezone +import errno import hashlib import json import os @@ -9,11 +10,16 @@ import shutil import subprocess import sys +import tempfile from zipfile import ZipFile from harbor.models.job.config import JobConfig from .bootstrap import AptCache, default_cache_dir +from .pi import distribution_digest, distribution_files, read_model_config + +_NANO = "harbor_adapter:NanoPyCodeAgent" +_PI = "harbor_adapter:PiCodeAgent" def standard_config(raw: dict) -> JobConfig: @@ -50,22 +56,69 @@ def standard_config(raw: dict) -> JobConfig: raise ValueError("Select at least one agent and one trial attempt") for agent in config.agents: identifiers = [value for value in (agent.import_path, agent.name) if value] - if not identifiers or any( - value != "harbor_adapter:NanoPyCodeAgent" for value in identifiers - ): - raise ValueError("The standard workflow requires harbor_adapter:NanoPyCodeAgent") - if agent.env or set(agent.kwargs) - { - "max_turns", "max_tokens", "time_budget_seconds", - }: + if not identifiers or len(set(identifiers)) != 1 or identifiers[0] not in {_NANO, _PI}: + raise ValueError("The standard workflow requires a repository benchmark adapter") + allowed = ({"max_turns", "max_tokens", "time_budget_seconds"} + if identifiers[0] == _NANO else { + "distribution_path", "distribution_sha256", "model_config_path", "thinking", + }) + if agent.env or set(agent.kwargs) - allowed: raise ValueError( "Pass model credentials through the environment; " - "agent kwargs may only set budgets" + "agent kwargs must use the adapter's supported public settings" ) + if identifiers[0] == _PI: + required = {"distribution_path", "distribution_sha256", "model_config_path"} + if not required <= agent.kwargs.keys(): + raise ValueError("Pi requires a hash-pinned distribution and public model config") + read_model_config(Path(agent.kwargs["model_config_path"])) if not agent.model_name: raise ValueError("Each agent requires an explicit model_name") return config +def snapshot_pi(agent, cache: Path, record: Path) -> dict: + """Share one content-addressed binary snapshot across the full task pool.""" + original = Path(agent.kwargs["distribution_path"]).expanduser().resolve() + files = distribution_files(original) + digest = distribution_digest(files) + if digest != agent.kwargs["distribution_sha256"]: + raise ValueError("Pi distribution differs from its registered SHA256") + destination = cache / "pi-distributions" / digest + if not destination.exists(): + destination.parent.mkdir(parents=True, exist_ok=True) + with tempfile.TemporaryDirectory(prefix=digest + ".tmp-", dir=destination.parent) as temporary: + staged = Path(temporary) / "distribution" + shutil.copytree(original, staged) + if distribution_files(staged) != files: + raise ValueError("Pi distribution changed while snapshotting") + try: + staged.rename(destination) + except OSError as error: + if error.errno not in {errno.EEXIST, errno.ENOTEMPTY}: + raise + # Another invocation published first. Validate its bytes below + # before reusing it; never overwrite a corrupt cache entry. + if distribution_files(destination) != files: + raise ValueError("Cached Pi distribution is corrupt") + public_models = read_model_config(Path(agent.kwargs["model_config_path"])) + model_bytes = (json.dumps(public_models, indent=2) + "\n").encode() + model_digest = hashlib.sha256(model_bytes).hexdigest() + models = record / "pi-models.json" + if models.exists() and models.read_bytes() != model_bytes: + models = record / f"pi-models-{model_digest}.json" + if models.exists(): + if models.read_bytes() != model_bytes: + raise ValueError("Recorded Pi model configuration is corrupt") + else: + models.write_bytes(model_bytes) + agent.kwargs.update(distribution_path=str(destination), model_config_path=str(models)) + return { + "distribution_sha256": digest, "files": files, + "model_config_file": models.name, "model_config_sha256": model_digest, + } + + def summarize(job_dir: Path, *, install_only: bool = False) -> dict: trials = [] for path in sorted(job_dir.glob("*/result.json")): @@ -111,16 +164,23 @@ def run(args) -> int: record.chmod(0o700) source = record / "source" shutil.copytree(root / "benchmarks/harbor/src/harbor_adapter", source / "harbor_adapter", ignore=shutil.ignore_patterns("__pycache__")) - shutil.copytree(root / "src/nanopycodeagent", source / "nanopycodeagent", ignore=shutil.ignore_patterns("__pycache__")) - subprocess.run(["uv", "build", "--wheel", "--out-dir", str(record / "dist")], check=True) - wheel, = (record / "dist").glob("*.whl") - with ZipFile(wheel) as archive: - for path in (source / "nanopycodeagent").rglob("*.py"): - if archive.read(path.relative_to(source).as_posix()) != path.read_bytes(): - raise ValueError("Source changed during preparation; prepare a new job") cache = args.cache_dir.expanduser().resolve() + wheel = None + if any((agent.import_path or agent.name) == _NANO for agent in config.agents): + shutil.copytree(root / "src/nanopycodeagent", source / "nanopycodeagent", ignore=shutil.ignore_patterns("__pycache__")) + subprocess.run(["uv", "build", "--wheel", "--out-dir", str(record / "dist")], check=True) + wheel, = (record / "dist").glob("*.whl") + with ZipFile(wheel) as archive: + for path in (source / "nanopycodeagent").rglob("*.py"): + if archive.read(path.relative_to(source).as_posix()) != path.read_bytes(): + raise ValueError("Source changed during preparation; prepare a new job") + pi_inputs = [] for agent in config.agents: - agent.kwargs.update(wheel_path=str(wheel), bootstrap_cache=str(cache)) + if (agent.import_path or agent.name) == _PI: + pi_inputs.append(snapshot_pi(agent, cache, record)) + else: + agent.kwargs["wheel_path"] = str(wheel) + agent.kwargs["bootstrap_cache"] = str(cache) config_path = record / "config.json" config_path.write_text(config.model_dump_json(indent=2) + "\n") manifest = { @@ -129,9 +189,11 @@ def run(args) -> int: "branch": subprocess.check_output(["git", "branch", "--show-current"], text=True).strip(), "source_kind": "working tree including uncommitted files", "source_sha256": {str(p.relative_to(source)): hashlib.sha256(p.read_bytes()).hexdigest() for p in sorted(source.rglob("*.py"))}, - "wheel_sha256": hashlib.sha256(wheel.read_bytes()).hexdigest(), + "wheel_sha256": hashlib.sha256(wheel.read_bytes()).hexdigest() if wheel else None, "cache_dir": str(cache), } + if pi_inputs: + manifest["pi_inputs"] = pi_inputs (record / "manifest.json").write_text(json.dumps(manifest, indent=2) + "\n") (record / "working-tree.patch").write_bytes(subprocess.check_output(["git", "diff", "HEAD"])) print(f"Prepared {record}", flush=True) diff --git a/benchmarks/harbor/src/harbor_adapter/_agent_runner.py b/benchmarks/harbor/src/harbor_adapter/_agent_runner.py new file mode 100644 index 0000000..18ad136 --- /dev/null +++ b/benchmarks/harbor/src/harbor_adapter/_agent_runner.py @@ -0,0 +1,153 @@ +"""Supervise one Linux agent command, including detached tool subprocesses.""" + +import ctypes +import json +import os +from pathlib import Path +import signal +import subprocess +import sys +import time + + +def processes(): + result = {} + for path in Path("/proc").iterdir(): + if not path.name.isdigit(): + continue + try: + fields = (path / "stat").read_text().rsplit(")", 1)[1].split() + result[int(path.name)] = (int(fields[1]), int(fields[19]), fields[0]) + except (OSError, ValueError, IndexError): + continue + return result + + +def descendants(root, table): + owned = {root} + while True: + updated = owned | {pid for pid, (parent, _, _) in table.items() if parent in owned} + if updated == owned: + return {pid: table[pid][1] for pid in owned - {root} if pid in table} + owned = updated + + +def signal_same_process(pid, started, number): + entry = processes().get(pid) + if entry is not None and entry[1] == started and entry[2] != "Z": + try: + os.kill(pid, number) + except ProcessLookupError: + pass + + +def stop_descendants(): + """Freeze parents before killing the forest, including new sessions.""" + owned = {} + for _ in range(4): + current = descendants(os.getpid(), processes()) + previous = set(owned) + owned.update(current) + for pid, started in current.items(): + signal_same_process(pid, started, signal.SIGSTOP) + if set(owned) == previous: + break + deadline = time.monotonic() + 5 + while True: + owned.update(descendants(os.getpid(), processes())) + for pid, started in owned.items(): + signal_same_process(pid, started, signal.SIGKILL) + while True: + try: + pid, _ = os.waitpid(-1, os.WNOHANG) + except ChildProcessError: + break + if pid == 0: + break + table = processes() + alive = [pid for pid, started in owned.items() + if pid in table and table[pid][1] == started and table[pid][2] != "Z"] + if not alive and not descendants(os.getpid(), table): + return len(owned) + if time.monotonic() >= deadline: + raise RuntimeError("Agent subprocesses did not stop") + time.sleep(.02) + + +def write_json(path, value): + temporary = path.with_name(path.name + ".tmp") + temporary.write_text(json.dumps(value) + "\n") + temporary.replace(path) + + +def run(control, record, command): + control, record = Path(control), Path(record) + cancelled = control.with_suffix(".cancelled") + libc = ctypes.CDLL(None, use_errno=True) + if libc.prctl(36, 1, 0, 0, 0) != 0: # PR_SET_CHILD_SUBREAPER + raise OSError(ctypes.get_errno(), "Cannot supervise orphaned agent tools") + stopping = False + + def interrupted(signum, frame): + if not stopping: + raise InterruptedError("Agent execution cancelled") + + signal.signal(signal.SIGTERM, interrupted) + signal.signal(signal.SIGINT, interrupted) + write_json(control, {"pid": os.getpid(), "started": processes()[os.getpid()][1]}) + try: + if cancelled.exists(): + write_json(record, {"status": "cancelled", "processes_stopped": True, "before_start": True}) + return 130 + write_json(record, {"status": "running"}) + child = subprocess.Popen(["bash", "-c", command], stdin=subprocess.DEVNULL, start_new_session=True) + code = child.wait() + if code: + stopping = True + stop_descendants() + # Successful solutions may intentionally leave a server running. + write_json(record, {"status": "completed", "return_code": code}) + return code + except BaseException as error: + stopping = True + try: + count = stop_descendants() + write_json(record, {"status": "cancelled" if isinstance(error, InterruptedError) else "error_stopped", + "processes_stopped": True, "stopped_processes": count}) + except BaseException: + write_json(record, {"status": "cleanup_failed", "processes_stopped": False}) + raise + if isinstance(error, InterruptedError): + return 130 + raise + finally: + control.unlink(missing_ok=True) + + +def cancel(control, record): + control, record = Path(control), Path(record) + control.with_suffix(".cancelled").touch() + try: + owner = json.loads(control.read_text()) + except FileNotFoundError: + return # A later startup observes the marker before spawning the agent. + signal_same_process(owner["pid"], owner["started"], signal.SIGTERM) + deadline = time.monotonic() + 8 + while time.monotonic() < deadline: + if record.exists(): + state = json.loads(record.read_text()) + if state.get("status") == "completed" or ( + state.get("status") in {"cancelled", "error_stopped"} and state.get("processes_stopped") is True + ): + return + if state.get("status") == "cleanup_failed": + raise RuntimeError("Agent subprocess cleanup failed") + time.sleep(.02) + raise RuntimeError("Agent supervisor did not confirm cancellation") + + +if __name__ == "__main__": + if sys.argv[1] == "cancel": + cancel(sys.argv[2], sys.argv[3]) + else: + raise SystemExit(run(sys.argv[2], sys.argv[3], sys.argv[4])) diff --git a/benchmarks/harbor/src/harbor_adapter/pi.py b/benchmarks/harbor/src/harbor_adapter/pi.py new file mode 100644 index 0000000..9a42ba2 --- /dev/null +++ b/benchmarks/harbor/src/harbor_adapter/pi.py @@ -0,0 +1,249 @@ +"""Run an unmodified, hash-pinned Pi distribution with shared Harbor bootstrap.""" + +import asyncio +import hashlib +import json +from pathlib import Path +import shlex +import uuid + +from harbor.agents.installed.base import CliFlag, with_prompt_template +from harbor.agents.model_connection import ModelConnectionSpec + +from .adapter import NanoPyCodeAgent + +_RUNNER = Path(__file__).with_name("_agent_runner.py").read_text() + + +def distribution_files(root: Path) -> dict: + """Include executable permissions and reject links outside the snapshot.""" + files = {} + for path in sorted(root.rglob("*")): + if path.is_symlink(): + raise ValueError("Pi distributions must not contain symbolic links") + if path.is_file(): + with path.open("rb") as stream: + digest = hashlib.file_digest(stream, "sha256").hexdigest() + files[path.relative_to(root).as_posix()] = { + "sha256": digest, "mode": path.stat().st_mode & 0o777, + } + if "pi" not in files or "package.json" not in files: + raise ValueError("Pi distribution requires pi and package.json") + return files + + +def distribution_digest(files: dict) -> str: + return hashlib.sha256(json.dumps(files, sort_keys=True).encode()).hexdigest() + + +def read_model_config(path: Path) -> dict: + """Accept a public DeepSeek connection; credentials stay in the environment.""" + config = json.loads(path.read_text()) + if set(config) != {"providers"} or set(config["providers"]) != {"deepseek"}: + raise ValueError("Pi benchmark models must configure only DeepSeek") + provider = config["providers"]["deepseek"] + if (set(provider) - {"api", "apiKey", "baseUrl", "models"} + or provider.get("apiKey") != "${DEEPSEEK_API_KEY}" + or provider.get("api") != "openai-completions" + or provider.get("baseUrl") != "https://api.deepseek.com"): + raise ValueError("Pi models must use the official API and an environment key reference") + if not provider.get("models"): + raise ValueError("Pi models must explicitly register at least one model") + return config + + +def validate_model_registration(stdout: str, stderr: str, model_name: str) -> None: + """Pi can exit zero after rejecting models.json and using built-in models.""" + requested = tuple(model_name.split("/", 1)) + rows = {tuple(line.split()[:2]) for line in stdout.splitlines()} + if stderr.strip() or requested not in rows: + raise ValueError( + f"Pi model registration failed for {model_name}: " + f"{stderr.strip() or 'requested provider/model missing from listing'}" + ) + + +class PiCodeAgent(NanoPyCodeAgent): + """Share verified dependencies/preflight, replacing Nano's install and run.""" + + SUPPORTS_ATIF = False + MODEL_CONNECTION = ModelConnectionSpec(api_key_envs=("DEEPSEEK_API_KEY",)) + CLI_FLAGS = [CliFlag("thinking", cli="--thinking", type="enum", choices=[ + "off", "minimal", "low", "medium", "high", "xhigh", "max", + ])] + + def __init__(self, *args, distribution_path: str, distribution_sha256: str, + model_config_path: str, **kwargs): + self._distribution = Path(distribution_path).resolve() + self._distribution_sha256 = distribution_sha256 + self._model_config = read_model_config(Path(model_config_path)) + package = json.loads((self._distribution / "package.json").read_text()) + self._pi_version = package["version"] + super().__init__(*args, version=self._pi_version, **kwargs) + provider, separator, model = (self.model_name or "").partition("/") + if provider != "deepseek" or not separator or model not in { + item["id"] for item in self._model_config["providers"][provider]["models"] + }: + raise ValueError("Pi model_name must identify a registered deepseek/model") + + @staticmethod + def name() -> str: + return "pi" + + def get_version_command(self) -> str: + return "/opt/harbor-pi/pi --version" + + def parse_version(self, stdout: str) -> str: + return stdout.strip().splitlines()[-1] + + async def _install_agent(self, environment) -> None: + self.logs_dir.mkdir(parents=True, exist_ok=True) + (self.logs_dir / "process-supervision-required").write_text("pi\n") + actual = distribution_digest(distribution_files(self._distribution)) + if actual != self._distribution_sha256: + raise ValueError("Pi distribution changed after registration") + await environment.upload_dir(self._distribution, "/opt/harbor-pi") + await self.exec_as_root(environment, command="chmod 755 /opt/harbor-pi/pi") + result = await self.exec_as_agent(environment, command=self.get_version_command()) + if self.parse_version(result.stdout) != self._pi_version: + raise ValueError("Installed Pi version differs from its distribution") + self._bootstrap_record.update( + pi_version=self._pi_version, distribution_sha256=actual, + ) + await self.exec_as_agent(environment, command="mkdir -p /tmp/harbor-pi-agent") + await self._upload_config_text( + environment, content=json.dumps(self._model_config), + remote_path="/tmp/harbor-pi-agent/models.json", filename="models.json", + ) + probe = f"/tmp/harbor-pi-supervision-probe-{uuid.uuid4().hex}" + await self.exec_as_agent(environment, command=shlex.join([ + "python3", "-I", "-c", _RUNNER, "run", probe + ".json", probe + "-result.json", "true", + ])) + self._bootstrap_record["process_supervision_probe"] = "passed" + # Validate the real binary and provider configuration without a model call. + _, model = self.model_name.split("/", 1) + listing = await self.exec_as_agent( + environment, + command=f"/opt/harbor-pi/pi --offline --list-models {shlex.quote(model)}", + env={"PI_CODING_AGENT_DIR": "/tmp/harbor-pi-agent", "DEEPSEEK_API_KEY": "preflight-only"}, + ) + validate_model_registration(listing.stdout, listing.stderr, self.model_name) + + @with_prompt_template + async def run(self, instruction, environment, context) -> None: + key = self.model_connection.api_key + if not key: + raise ValueError("DEEPSEEK_API_KEY is required for Pi model work") + _, model = self.model_name.split("/", 1) + # Keep completed events and native sessions; streaming deltas duplicate + # their contents and can consume substantial disk space. + event_filter = ( + "import json,sys\n" + "for line in sys.stdin:\n" + " try: skip=json.loads(line).get('type') == 'message_update'\n" + " except (ValueError,AttributeError): skip=False\n" + " if not skip: print(line,end='',flush=True)\n" + ) + command = ( + "set -euo pipefail; /opt/harbor-pi/pi --print --mode json --offline " + "--session-dir /logs/agent/pi/sessions " + f"--provider deepseek --model {shlex.quote(model)} " + f"{self.build_cli_flags()} -- {shlex.quote(instruction)} " + "/logs/agent/pi-stderr.txt | " + f"python3 -I -u -c {shlex.quote(event_filter)} | tee /logs/agent/pi.txt" + ) + control = f"/tmp/harbor-pi-{uuid.uuid4().hex}.json" + record = "/logs/agent/process-supervision.json" + try: + await self.exec_as_agent( + environment, + command=shlex.join(["python3", "-I", "-c", _RUNNER, "run", control, record, command]), + env={"PI_CODING_AGENT_DIR": "/tmp/harbor-pi-agent", "DEEPSEEK_API_KEY": key}, + ) + except BaseException: + cleanup = await asyncio.shield(asyncio.wait_for(environment.exec( + command=shlex.join(["python3", "-I", "-c", _RUNNER, "cancel", control, record]), user="root", + ), timeout=12)) + if cleanup.return_code != 0: + raise RuntimeError("Pi subprocess cleanup did not confirm cancellation") + raise + # Pi may finish its event stream after an API error with exit code zero. + # Do not misclassify that as an ordinary scored task failure. + check = ( + "import json\n" + "last={}; assistant={}\n" + "for line in open('/logs/agent/pi.txt'):\n" + " try: event=json.loads(line)\n" + " except ValueError: continue\n" + " last=event\n" + " if event.get('type')=='message_end' and event.get('message',{}).get('role')=='assistant': assistant=event['message']\n" + "print(json.dumps({'settled': last.get('type')=='agent_settled' and not last.get('aborted'), 'stop_reason': assistant.get('stopReason')}))\n" + ) + output = await self.exec_as_agent(environment, command=f"python3 -I -c {shlex.quote(check)}") + status = json.loads(output.stdout) + if not status["settled"] or status["stop_reason"] != "stop": + raise RuntimeError(f"Pi did not finish successfully: {status}") + + def populate_context_post_run(self, context) -> None: + context.metadata = context.metadata or {} + context.metadata["bootstrap"] = self._bootstrap_record + supervision = None + supervision_path = self.logs_dir / "process-supervision.json" + if supervision_path.exists(): + try: + supervision = json.loads(supervision_path.read_text()) + except (OSError, ValueError): + supervision = {"status": "invalid"} + context.metadata["process_supervision"] = supervision + interrupted = supervision is not None and ( + not isinstance(supervision, dict) or supervision.get("status") != "completed" + or supervision.get("return_code") != 0 + ) + output = self.logs_dir / "pi.txt" + if not output.exists(): + return + usages, missing = [], 0 + errors = [] + for line in output.read_text().split("\n"): + try: + event = json.loads(line) + except ValueError: + continue + if not isinstance(event, dict): + continue + message = event.get("message", {}) + if event.get("type") == "message_end" and message.get("role") == "assistant": + if message.get("stopReason") in {"error", "aborted"}: + errors.append(message.get("stopReason")) + usage = message.get("usage") + # Failed streams can retain Pi's initialized all-zero usage + # when the connection closes before the provider usage chunk. + unknown_failure_usage = message.get("stopReason") in {"error", "aborted"} and ( + not usage or not any(usage.get(key, 0) for key in ( + "input", "output", "cacheRead", "cacheWrite", "totalTokens", + )) + ) + if usage and not unknown_failure_usage: + usages.append(usage) + else: + missing += 1 + elif event.get("type") == "compaction_end" and event.get("result"): + usage = event["result"].get("usage") + if usage: + usages.append(usage) + else: + missing += 1 + context.metadata["pi"] = { + "version": self._pi_version, "usage_records": len(usages), + "missing_usage_records": missing, "assistant_errors": errors, + "cost_is_estimated": True, + "usage_is_partial": not usages or missing > 0 or interrupted, + "cost_is_partial": not usages or missing > 0 or interrupted, + } + if usages: + context.n_input_tokens = sum(u.get("input", 0) + u.get("cacheRead", 0) + u.get("cacheWrite", 0) for u in usages) + context.n_output_tokens = sum(u.get("output", 0) for u in usages) + context.n_cache_tokens = sum(u.get("cacheRead", 0) for u in usages) + costs = [u["cost"]["total"] for u in usages if "total" in u.get("cost", {})] + context.cost_usd = sum(costs) if costs else None + context.metadata["pi"]["cost_is_partial"] = missing > 0 or len(costs) != len(usages) or interrupted diff --git a/benchmarks/harbor/src/harbor_adapter/verifier.py b/benchmarks/harbor/src/harbor_adapter/verifier.py index 155f02f..5e9d79c 100644 --- a/benchmarks/harbor/src/harbor_adapter/verifier.py +++ b/benchmarks/harbor/src/harbor_adapter/verifier.py @@ -24,6 +24,27 @@ class VerifierCollectionError(RuntimeError): """Authorized pytest collection recovery exhausted its attempt budget.""" +class AgentQuiescenceError(RuntimeError): + """The installed agent did not establish a frozen execution boundary.""" + + +def check_agent_quiescence(agent_dir, *, required=False): + agent_dir = Path(agent_dir) + if not required and not (agent_dir / "process-supervision-required").exists(): + return + try: + state = json.loads((agent_dir / "process-supervision.json").read_text()) + except (OSError, ValueError) as error: + raise AgentQuiescenceError("Missing or invalid agent supervision record") from error + if not isinstance(state, dict): + raise AgentQuiescenceError("Invalid agent supervision record") + if state.get("status") == "completed" and type(state.get("return_code")) is int: + return + if state.get("status") in {"cancelled", "error_stopped"} and state.get("processes_stopped") is True: + return + raise AgentQuiescenceError("Agent subprocesses have not confirmed a frozen boundary") + + class _RetryEnvironment: """Delegate Harbor operations, wrapping only the test-script execution.""" @@ -93,11 +114,14 @@ class RetryingVerifier(Verifier): """ def __init__(self, *, max_attempts=3, retry_dependency_errors=False, - retry_collection_errors=False, **kwargs): + retry_collection_errors=False, require_agent_quiescence=False, + **kwargs): if type(max_attempts) is not int or not 1 <= max_attempts <= 3: raise ValueError("max_attempts must be an integer between 1 and 3") if type(retry_dependency_errors) is not bool: raise ValueError("retry_dependency_errors must be a boolean") + if type(require_agent_quiescence) is not bool: + raise ValueError("require_agent_quiescence must be a boolean") if type(retry_collection_errors) is not bool: raise ValueError("retry_collection_errors must be a boolean") super().__init__(**kwargs) @@ -105,12 +129,15 @@ def __init__(self, *, max_attempts=3, retry_dependency_errors=False, raise ValueError("RetryingVerifier supports single-step Linux tasks") self._max_attempts = max_attempts self._retry_dependency_errors = retry_dependency_errors + self._require_agent_quiescence = require_agent_quiescence self._retry_collection_errors = retry_collection_errors self._attempt_timeout = self.task.config.verifier.timeout_sec if not math.isfinite(self._attempt_timeout) or self._attempt_timeout <= 0: raise ValueError("The task's verifier timeout must be finite and positive") async def verify(self): + check_agent_quiescence(self.trial_paths.agent_dir, + required=getattr(self, "_require_agent_quiescence", False)) # Harbor's outer watchdog covers the entire custom verifier. Reject an # insufficient budget instead of silently losing the promised retries. config = TrialConfig.model_validate_json(self.trial_paths.config_path.read_text()) diff --git a/benchmarks/harbor/tests/test_agent_supervision.py b/benchmarks/harbor/tests/test_agent_supervision.py new file mode 100644 index 0000000..e21c0c3 --- /dev/null +++ b/benchmarks/harbor/tests/test_agent_supervision.py @@ -0,0 +1,149 @@ +"""A timed-out agent must stop before any verifier can observe its output.""" + +import json +import os +from pathlib import Path +import shlex +import signal +import subprocess +import sys +import time +from types import SimpleNamespace + +import pytest + +from harbor_adapter import _agent_runner as runner +from harbor_adapter.verifier import AgentQuiescenceError, RetryingVerifier, check_agent_quiescence + + +def wait_for(predicate, timeout=5): + deadline = time.monotonic() + timeout + while time.monotonic() < deadline: + if predicate(): + return + time.sleep(.01) + raise AssertionError("Timed out waiting for the supervision fixture") + + +def alive(pid): + entry = runner.processes().get(pid) + return entry is not None and entry[2] != "Z" + + +def start(tmp_path, command): + control, record = tmp_path / "control.json", tmp_path / "supervision.json" + child = subprocess.Popen( + [sys.executable, runner.__file__, "run", str(control), str(record), command], + stdin=subprocess.DEVNULL, stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, + ) + return child, control, record + + +def command_with_detached_writer(tmp_path, *, exit_code=None): + pidfile, counter = tmp_path / "writer.pid", tmp_path / "counter" + writer = ( + f"import os,time; from pathlib import Path; Path({str(pidfile)!r}).write_text(str(os.getpid()))\n" + f"with open({str(counter)!r},'a',buffering=1) as stream:\n" + " while True: stream.write('x'); stream.flush(); time.sleep(.01)\n" + ) + launcher = ( + "import subprocess,sys,time; " + f"subprocess.Popen([sys.executable,'-c',{writer!r}],start_new_session=True, " + "stdin=subprocess.DEVNULL,stdout=subprocess.DEVNULL,stderr=subprocess.DEVNULL); " + ) + if exit_code is None: + launcher += "time.sleep(60)" + else: + launcher += "time.sleep(.15); sys.exit(" + str(exit_code) + ")" + return shlex.join([sys.executable, "-c", launcher]), pidfile, counter + + +def cleanup(child, pidfile): + if child.poll() is None: + child.kill() + child.wait(timeout=5) + if pidfile.exists(): + pid = int(pidfile.read_text()) + if alive(pid): + os.kill(pid, signal.SIGKILL) + + +def test_cancellation_stops_detached_tools_and_freezes_output(tmp_path): + command, pidfile, counter = command_with_detached_writer(tmp_path) + child, control, record = start(tmp_path, command) + try: + wait_for(lambda: pidfile.exists() and counter.exists() and counter.stat().st_size > 2) + subprocess.run([sys.executable, runner.__file__, "cancel", str(control), str(record)], check=True, timeout=12) + assert child.wait(timeout=5) == 130 + assert not alive(int(pidfile.read_text())) + value = counter.read_bytes() + time.sleep(.1) + assert counter.read_bytes() == value + state = json.loads(record.read_text()) + assert state["status"] == "cancelled" and state["processes_stopped"] is True + finally: + cleanup(child, pidfile) + + +def test_cancellation_before_launch_prevents_agent_work(tmp_path): + control, record, output = tmp_path / "control.json", tmp_path / "supervision.json", tmp_path / "output" + runner.cancel(control, record) + command = shlex.join([sys.executable, "-c", f"from pathlib import Path; Path({str(output)!r}).touch()"]) + child, _, _ = start(tmp_path, command) + assert child.wait(timeout=5) == 130 + assert not output.exists() + assert json.loads(record.read_text())["before_start"] is True + + +def test_successful_agent_keeps_intentional_background_service(tmp_path): + command, pidfile, _ = command_with_detached_writer(tmp_path, exit_code=0) + child, _, record = start(tmp_path, command) + try: + assert child.wait(timeout=5) == 0 + assert alive(int(pidfile.read_text())) + assert json.loads(record.read_text()) == {"status": "completed", "return_code": 0} + finally: + cleanup(child, pidfile) + + +def test_nonzero_agent_stops_detached_tool_and_preserves_exit_code(tmp_path): + command, pidfile, _ = command_with_detached_writer(tmp_path, exit_code=7) + child, _, record = start(tmp_path, command) + try: + assert child.wait(timeout=5) == 7 + assert not alive(int(pidfile.read_text())) + assert json.loads(record.read_text())["return_code"] == 7 + finally: + cleanup(child, pidfile) + + +@pytest.mark.parametrize("state", [None, {"status": "running"}, {"status": "cleanup_failed"}, + {"status": "cancelled", "processes_stopped": False}, {"status": "completed"}]) +def test_verifier_refuses_an_unconfirmed_agent_boundary(tmp_path, state): + (tmp_path / "process-supervision-required").touch() + if state is not None: + (tmp_path / "process-supervision.json").write_text(json.dumps(state)) + with pytest.raises(AgentQuiescenceError): + check_agent_quiescence(tmp_path) + + +@pytest.mark.parametrize("state", [{"status": "completed", "return_code": 0}, + {"status": "cancelled", "processes_stopped": True}]) +def test_verifier_accepts_a_confirmed_agent_boundary(tmp_path, state): + (tmp_path / "process-supervision-required").touch() + (tmp_path / "process-supervision.json").write_text(json.dumps(state)) + check_agent_quiescence(tmp_path) + + +def test_registered_guard_cannot_be_disabled_by_a_missing_marker(tmp_path): + with pytest.raises(AgentQuiescenceError): + check_agent_quiescence(tmp_path, required=True) + + +def test_verifier_checks_the_boundary_before_accessing_task_assertions(tmp_path): + import asyncio + verifier = object.__new__(RetryingVerifier) + verifier.trial_paths = SimpleNamespace(agent_dir=tmp_path) + verifier._require_agent_quiescence = True + with pytest.raises(AgentQuiescenceError): + asyncio.run(verifier.verify()) diff --git a/benchmarks/harbor/tests/test_comparison.py b/benchmarks/harbor/tests/test_comparison.py index fc2f5d6..59614b1 100644 --- a/benchmarks/harbor/tests/test_comparison.py +++ b/benchmarks/harbor/tests/test_comparison.py @@ -57,6 +57,79 @@ def test_interrupted_job_cannot_silently_restart(tmp_path): comparison.run_job(tmp_path, {}, job, tmp_path, False) +def test_low_disk_space_stops_before_recording_or_starting_a_trial(tmp_path, monkeypatch): + monkeypatch.setattr(comparison.shutil, "disk_usage", lambda path: SimpleNamespace(free=1)) + with pytest.raises(RuntimeError, match="Insufficient free disk"): + comparison.run_job(tmp_path, {"minimum_free_disk_bytes": 2}, {"job_name": "new"}, tmp_path, False) + assert not (tmp_path / "new.json").exists() + + +@pytest.fixture +def batch_images(tmp_path, monkeypatch): + entry = {"task": "org/task", "image": "owner/task:tag", "image_id": "sha256:expected", + "repo_digests": ["owner/task@sha256:expected"]} + (tmp_path / "images.json").write_text(json.dumps({"images": [entry]})) + (tmp_path / "config.json").write_text(json.dumps({"tasks": [{"name": "org/task"}]})) + monkeypatch.setattr(comparison.subprocess, "check_output", lambda *a, **k: "sha256:protected\n") + monkeypatch.setattr(comparison.shutil, "disk_usage", lambda p: SimpleNamespace(free=100)) + calls = [] + def run(command, **kwargs): + calls.append(command) + return SimpleNamespace(returncode=0, stdout="", stderr="") + monkeypatch.setattr(comparison.subprocess, "run", run) + plan = {"batch_images": {"manifest": "images.json", "storage_path": str(tmp_path), + "minimum_free_disk_bytes": 50}} + manager = comparison.BatchImages(tmp_path, plan, tmp_path) + return manager, [{"config": "config.json"}], entry, calls + + +def test_batch_restores_by_digest_and_verifies_identity(batch_images, monkeypatch): + manager, jobs, entry, calls = batch_images + sequence = iter([None, {"Id": entry["image_id"], "RepoDigests": entry["repo_digests"]}]) + monkeypatch.setattr(manager, "inspect", lambda image: next(sequence)) + manager.prepare(jobs) + assert calls == [["docker", "pull", entry["repo_digests"][0]], + ["docker", "tag", entry["repo_digests"][0], entry["image"]]] + + +def test_batch_can_use_existing_verified_range_cache(batch_images, monkeypatch): + manager, jobs, entry, calls = batch_images + manager.policy["cache_dir"] = "/cache/images" + sequence = iter([None, {"Id": entry["image_id"], "RepoDigests": entry["repo_digests"]}]) + monkeypatch.setattr(manager, "inspect", lambda image: next(sequence)) + manager.prepare(jobs) + assert len(calls) == 1 + assert calls[0][1].endswith("scripts/cache_image.py") + assert calls[0][2:] == [entry["image"], "--cache", "/cache/images", "--load"] + + +def test_batch_rejects_a_changed_tag_before_running_jobs(batch_images, monkeypatch): + manager, jobs, _, _ = batch_images + monkeypatch.setattr(manager, "inspect", lambda image: {"Id": "sha256:different", "RepoDigests": []}) + with pytest.raises(ValueError, match="differs from registration"): + manager.prepare(jobs) + + +def test_batch_preserves_preexisting_images_and_never_forces_eviction(batch_images, monkeypatch): + manager, jobs, entry, calls = batch_images + monkeypatch.setattr(manager, "inspect", lambda image: {"Id": entry["image_id"]}) + manager.release(jobs) + assert calls == [["docker", "image", "rm", entry["image"]], + ["docker", "image", "rm", entry["repo_digests"][0]]] + calls.clear() + manager.protected.add(entry["image_id"]) + manager.release(jobs) + assert calls == [] + + +def test_batch_checks_actual_docker_storage_before_downloading(batch_images, monkeypatch): + manager, jobs, _, calls = batch_images + monkeypatch.setattr(comparison.shutil, "disk_usage", lambda path: SimpleNamespace(free=1)) + with pytest.raises(RuntimeError, match="Insufficient Docker storage"): + manager.prepare(jobs) + assert calls == [] + + def test_registered_transport_files_cannot_change(tmp_path): path = tmp_path / "apt-cache.conf" path.write_text("original configuration") diff --git a/benchmarks/harbor/tests/test_image_cache.py b/benchmarks/harbor/tests/test_image_cache.py index 28de6bc..9fa15bf 100644 --- a/benchmarks/harbor/tests/test_image_cache.py +++ b/benchmarks/harbor/tests/test_image_cache.py @@ -152,3 +152,192 @@ def test_redirect_does_not_forward_registry_authorization(): assert redirected.get_header("Range") == "bytes=0-2" with pytest.raises(ValueError, match="HTTPS"): cache_image.SafeRedirect().redirect_request(request, None, 307, "", {}, "http://cdn.docker.com/blob") + + +def test_registry_refreshes_expiring_token_before_later_ranges(monkeypatch): + clock = [0] + tokens = [] + def get(url, headers=None, limit=None): + if url.startswith("https://auth.docker.io/"): + token = f"token-{len(tokens)}" + tokens.append(token) + return 200, {}, json.dumps({"token": token, "expires_in": 300}).encode() + return 206, {}, headers["Authorization"].encode() + monkeypatch.setattr(cache_image, "get", get) + monkeypatch.setattr(cache_image.time, "monotonic", lambda: clock[0]) + auth = cache_image.RegistryAuth("example/task") + assert auth.get("blob")[2] == b"Bearer token-0" + clock[0] = 271 + assert auth.get("blob")[2] == b"Bearer token-1" + assert len(tokens) == 2 + + +def test_registry_refreshes_once_on_unauthorized_and_preserves_range(monkeypatch): + calls = [] + tokens = iter(["old", "new"]) + def get(url, headers=None, limit=None): + if url.startswith("https://auth.docker.io/"): + return 200, {}, json.dumps({"token": next(tokens)}).encode() + calls.append(headers.copy()) + if headers["Authorization"] == "Bearer old": + raise cache_image.RegistryHTTPError(401) + return 206, {}, b"abc" + monkeypatch.setattr(cache_image, "get", get) + assert cache_image.RegistryAuth("example/task").get("blob", {"Range": "bytes=0-2"})[2] == b"abc" + assert [call["Range"] for call in calls] == ["bytes=0-2", "bytes=0-2"] + + +def test_registry_does_not_refresh_indefinitely_or_on_rate_limits(monkeypatch): + for status, expected_calls in [(401, 2), (429, 1)]: + calls = [] + def get(url, headers=None, limit=None): + if url.startswith("https://auth.docker.io/"): + return 200, {}, b'{"token":"token"}' + calls.append(url) + raise cache_image.RegistryHTTPError(status) + monkeypatch.setattr(cache_image, "get", get) + with pytest.raises(cache_image.RegistryHTTPError) as error: + cache_image.RegistryAuth("example/task").get("blob") + assert error.value.status == status + assert len(calls) == expected_calls + + +def test_http_diagnostic_does_not_expose_signed_url(monkeypatch): + secret_url = "https://cdn.example/blob?signature=private" + class Opener: + def open(self, *args, **kwargs): + raise cache_image.HTTPError(secret_url, 401, "unauthorized", {}, None) + monkeypatch.setattr(cache_image, "build_opener", lambda *args: Opener()) + with pytest.raises(cache_image.RegistryHTTPError) as error: + cache_image.get(secret_url) + assert str(error.value) == "Registry request failed: HTTP 401" + assert "private" not in str(error.value) + + +def fixture_index(raw, platforms=None): + child = descriptor(raw, cache_image.MANIFEST_TYPES[0]) + platforms = platforms or [{"os": "linux", "architecture": "amd64"}, {"os": "linux", "architecture": "arm64"}] + return json.dumps({"schemaVersion": 2, "mediaType": cache_image.INDEX_TYPES[1], + "manifests": [{**child, "platform": platform} for platform in platforms]}).encode() + + +def test_index_archive_preserves_parent_child_and_native_layers(tmp_path): + raw = fixture_manifest(tmp_path) + index = fixture_index(raw) + output = tmp_path / "indexed.tar" + cache_image.write_archive(tmp_path, "example/task:version", raw, output, root_index=index) + with tarfile.open(output) as archive: + reference = json.load(archive.extractfile("index.json"))["manifests"][0] + assert reference["digest"] == cache_image.sha(index) + assert archive.extractfile("blobs/sha256/" + cache_image.sha(index).split(":")[1]).read() == index + assert archive.extractfile("blobs/sha256/" + cache_image.sha(raw).split(":")[1]).read() == raw + for item in json.loads(raw)["layers"]: + assert cache_image.sha(archive.extractfile("blobs/sha256/" + item["digest"].split(":")[1]).read()) == item["digest"] + + +@pytest.mark.parametrize("platforms", [ + [{"os": "linux", "architecture": "arm64"}], + [{"os": "linux", "architecture": "amd64"}] * 2, + [{"os": "linux", "architecture": "amd64", "variant": "v3"}], + [{"os": "windows", "architecture": "amd64"}], +]) +def test_index_rejects_missing_or_ambiguous_native_platform(platforms): + with pytest.raises(ValueError, match="exactly one Linux amd64"): + cache_image.native_descriptor(json.loads(fixture_index(b"child", platforms))) + + +def test_index_rejects_a_different_native_manifest(tmp_path): + raw = fixture_manifest(tmp_path) + with pytest.raises(ValueError, match="differs from its original index"): + cache_image.write_archive(tmp_path, "example/task:version", raw, tmp_path / "image.tar", + root_index=fixture_index(b"different manifest")) + assert not (tmp_path / "image.tar").exists() + + +def test_registry_index_loads_selected_child_by_digest_and_keeps_parent_pin(tmp_path, monkeypatch): + raw = fixture_manifest(tmp_path) + index = fixture_index(raw) + requests = [] + def get(url, headers=None, limit=None): + requests.append(url) + if url.startswith("https://auth.docker.io/"): + return 200, {}, b'{"token":"anonymous-token"}' + body = index if url.endswith("/version") else raw + return 200, {"Docker-Content-Digest": cache_image.sha(body)}, body + monkeypatch.setattr(cache_image, "get", get) + cache_image.cache_image("example/task:version", tmp_path) + assert requests[-1].endswith("/manifests/" + cache_image.sha(raw)) + lock = json.loads((tmp_path / "example_task_version.json").read_text()) + assert lock["manifest_digest"] == cache_image.sha(index) + with tarfile.open(tmp_path / "example_task_version.oci.tar") as archive: + assert json.load(archive.extractfile("index.json"))["manifests"][0]["digest"] == cache_image.sha(index) + + +def fixture_attested_index(cache, native_raw): + index = json.loads(fixture_index(native_raw)) + index["manifests"][1].update(digest=cache_image.sha(b"absent ARM manifest"), size=19) + config = put(cache, b'{"architecture":"unknown","os":"unknown","rootfs":{"type":"layers","diff_ids":[]}}', "application/vnd.oci.image.config.v1+json") + payload = put(cache, b'{"_type":"https://in-toto.io/Statement/v0.1"}', "application/vnd.in-toto+json") + raw = json.dumps({"schemaVersion": 2, "mediaType": cache_image.MANIFEST_TYPES[1], + "config": config, "layers": [payload]}).encode() + attestation = put(cache, raw, cache_image.MANIFEST_TYPES[1]) + attestation.update(platform={"os": "unknown", "architecture": "unknown"}, + annotations={"vnd.docker.reference.type": "attestation-manifest", + "vnd.docker.reference.digest": cache_image.sha(native_raw)}) + absent = {**attestation, "digest": cache_image.sha(b"absent ARM attestation"), + "annotations": {"vnd.docker.reference.type": "attestation-manifest", + "vnd.docker.reference.digest": index["manifests"][1]["digest"]}} + index["manifests"].extend([attestation, absent]) + return json.dumps(index).encode(), attestation, [config, payload] + + +def test_index_preserves_native_attestations_without_fetching_other_platforms(tmp_path): + raw = fixture_manifest(tmp_path) + index, attestation, metadata = fixture_attested_index(tmp_path, raw) + assert cache_image.native_attestations(json.loads(index)) == [attestation] + output = tmp_path / "attested.tar" + cache_image.write_archive(tmp_path, "example/task:version", raw, output, root_index=index) + with tarfile.open(output) as archive: + for item in [attestation, *metadata]: + blob = archive.extractfile("blobs/sha256/" + item["digest"].split(":")[1]).read() + assert cache_image.sha(blob) == item["digest"] + for item in [json.loads(index)["manifests"][1], json.loads(index)["manifests"][3]]: + assert "blobs/sha256/" + item["digest"].split(":")[1] not in archive.getnames() + + +def test_index_refuses_corrupt_attestation_payload(tmp_path): + raw = fixture_manifest(tmp_path) + index, _, metadata = fixture_attested_index(tmp_path, raw) + cache_image.blob_path(tmp_path, metadata[1]).write_bytes(b"corrupt") + with pytest.raises(ValueError, match="size mismatch|SHA256 mismatch"): + cache_image.write_archive(tmp_path, "example/task:version", raw, tmp_path / "bad.tar", root_index=index) + + +def test_attestation_metadata_is_never_treated_as_an_executable_image(tmp_path): + raw = fixture_manifest(tmp_path) + with pytest.raises(ValueError, match="in-toto"): + cache_image.attestation_blobs(raw) + + +def test_registry_fetches_native_attestation_graph_and_skips_arm_graph(tmp_path, monkeypatch): + raw = fixture_manifest(tmp_path) + index, attestation, metadata = fixture_attested_index(tmp_path, raw) + bodies = {cache_image.sha(raw): raw} + for item in [attestation, *metadata]: + path = cache_image.blob_path(tmp_path, item) + bodies[item["digest"]] = path.read_bytes() + path.unlink() + requests = [] + def get(url, headers=None, limit=None): + requests.append(url) + if url.startswith("https://auth.docker.io/"): + return 200, {}, b'{"token":"anonymous-token"}' + body = index if url.endswith("/version") else bodies[url.rsplit("/", 1)[1]] + return 200, {"Docker-Content-Digest": cache_image.sha(body)}, body + monkeypatch.setattr(cache_image, "get", get) + cache_image.cache_image("example/task:version", tmp_path) + assert any(url.endswith("/manifests/" + attestation["digest"]) for url in requests) + for item in metadata: + cache_image.verify(cache_image.blob_path(tmp_path, item), item) + for item in [json.loads(index)["manifests"][1], json.loads(index)["manifests"][3]]: + assert not any(url.endswith(item["digest"]) for url in requests) diff --git a/benchmarks/harbor/tests/test_pi.py b/benchmarks/harbor/tests/test_pi.py new file mode 100644 index 0000000..500d9ca --- /dev/null +++ b/benchmarks/harbor/tests/test_pi.py @@ -0,0 +1,352 @@ +"""Pi provenance, native execution, and usage accounting contracts.""" + +import asyncio +from concurrent.futures import ThreadPoolExecutor +import errno +import hashlib +import json +from pathlib import Path +from threading import Barrier +from types import SimpleNamespace + +import pytest +from harbor.models.agent.context import AgentContext + +from harbor_adapter import PiCodeAgent +from harbor_adapter import __main__ as workflow +from harbor_adapter.__main__ import snapshot_pi, standard_config +from harbor_adapter.pi import distribution_digest, distribution_files, read_model_config + + +@pytest.fixture +def inputs(tmp_path): + distribution = tmp_path / "distribution" + distribution.mkdir() + (distribution / "pi").write_text("binary") + (distribution / "pi").chmod(0o755) + (distribution / "package.json").write_text(json.dumps({"version": "1.4.0"})) + models = tmp_path / "models.json" + models.write_text(json.dumps({"providers": {"deepseek": { + "api": "openai-completions", "baseUrl": "https://api.deepseek.com", + "apiKey": "${DEEPSEEK_API_KEY}", "models": [{"id": "deepseek-flash"}], + }}})) + return { + "distribution_path": str(distribution), + "distribution_sha256": distribution_digest(distribution_files(distribution)), + "model_config_path": str(models), "thinking": "high", + } + + +def adapter(tmp_path, inputs): + return PiCodeAgent(logs_dir=tmp_path, model_name="deepseek/deepseek-flash", **inputs) + + +class Environment: + default_user = None + + def __init__(self): + self.calls = [] + self.uploads = [] + self.completion = {"settled": True, "stop_reason": "stop"} + self.model_listing = "provider model\ndeepseek deepseek-flash\n" + self.model_warning = "" + + async def exec(self, **kwargs): + self.calls.append(kwargs) + command = kwargs["command"] + stdout = "" + if command.endswith("--version"): + stdout = "1.4.0\n" + elif "; python3 -c " in command or "; python3 -I -c " in command: + stdout = json.dumps(self.completion) + elif "--list-models" in command: + return SimpleNamespace(return_code=0, stdout=self.model_listing, stderr=self.model_warning) + return SimpleNamespace(return_code=0, stdout=stdout, stderr="") + + async def upload_dir(self, source, target): + self.uploads.append((source, target)) + + async def upload_file(self, source, target): + self.uploads.append((Path(source).read_text(), target)) + + +def test_public_pi_config_is_accepted_without_allowing_credentials(tmp_path, inputs): + raw = {"tasks": [{"name": "org/task"}], "agents": [{ + "name": "harbor_adapter:PiCodeAgent", "model_name": "deepseek/deepseek-flash", + "kwargs": inputs, + }]} + assert standard_config(raw).agents[0].kwargs["thinking"] == "high" + raw["agents"][0]["kwargs"] = inputs | {"api_key": "private"} + with pytest.raises(ValueError, match="credentials"): + standard_config(raw) + models = Path(inputs["model_config_path"]) + models.write_text(models.read_text().replace("${DEEPSEEK_API_KEY}", "private")) + with pytest.raises(ValueError, match="environment key"): + read_model_config(models) + + +def test_snapshot_reuses_identical_binary_but_detects_corruption(tmp_path, inputs): + cache = tmp_path / "cache" + agent = SimpleNamespace(kwargs=inputs.copy()) + first, second = tmp_path / "first", tmp_path / "second" + first.mkdir() + second.mkdir() + manifest = snapshot_pi(agent, cache, first) + assert snapshot_pi(SimpleNamespace(kwargs=inputs.copy()), cache, second) == manifest + destination = Path(agent.kwargs["distribution_path"]) + assert destination != Path(inputs["distribution_path"]) + (destination / "pi").write_text("corrupted") + with pytest.raises(ValueError, match="corrupt"): + snapshot_pi(SimpleNamespace(kwargs=inputs.copy()), cache, second) + + +def test_changed_executable_permissions_and_symlinks_are_rejected(tmp_path, inputs): + root = Path(inputs["distribution_path"]) + (root / "pi").chmod(0o644) + assert distribution_digest(distribution_files(root)) != inputs["distribution_sha256"] + (root / "link").symlink_to(tmp_path / "outside") + with pytest.raises(ValueError, match="symbolic links"): + distribution_files(root) + + +def test_snapshot_preserves_distinct_model_settings_in_one_job(tmp_path, inputs): + first = SimpleNamespace(kwargs=inputs.copy()) + alternate = tmp_path / "alternate-models.json" + models = json.loads(Path(inputs["model_config_path"]).read_text()) + models["providers"]["deepseek"]["models"][0]["maxTokens"] = 1000 + alternate.write_text(json.dumps(models)) + second = SimpleNamespace(kwargs=inputs | {"model_config_path": str(alternate)}) + record = tmp_path / "job-input" + record.mkdir() + manifests = [snapshot_pi(agent, tmp_path / "cache", record) for agent in [first, second]] + paths = [Path(agent.kwargs["model_config_path"]) for agent in [first, second]] + assert paths[0] != paths[1] + assert "maxTokens" not in json.loads(paths[0].read_text())["providers"]["deepseek"]["models"][0] + assert json.loads(paths[1].read_text())["providers"]["deepseek"]["models"][0]["maxTokens"] == 1000 + for path, manifest in zip(paths, manifests): + assert hashlib.sha256(path.read_bytes()).hexdigest() == manifest["model_config_sha256"] + assert record / manifest["model_config_file"] == path + third = SimpleNamespace(kwargs=inputs | {"model_config_path": str(alternate)}) + snapshot_pi(third, tmp_path / "cache", record) + assert third.kwargs["model_config_path"] == str(paths[1]) + + +@pytest.mark.parametrize("same_record_name", [False, True]) +def test_concurrent_distribution_publication_reuses_verified_winner(tmp_path, inputs, monkeypatch, same_record_name): + barrier = Barrier(2) + copytree = workflow.shutil.copytree + original = Path(inputs["distribution_path"]) + + def stage_together(source, target, *args, **kwargs): + result = copytree(source, target, *args, **kwargs) + if Path(source) == original: + barrier.wait(timeout=5) + return result + + monkeypatch.setattr(workflow.shutil, "copytree", stage_together) + records = [tmp_path / "a" / "job-input", tmp_path / "b" / ("job-input" if same_record_name else "other-input")] + for record in records: + record.mkdir(parents=True) + agents = [SimpleNamespace(kwargs=inputs.copy()) for _ in records] + with ThreadPoolExecutor(max_workers=2) as pool: + futures = [pool.submit(snapshot_pi, agent, tmp_path / "cache", record) for agent, record in zip(agents, records)] + manifests = [future.result(timeout=10) for future in futures] + assert manifests[0] == manifests[1] + destination = Path(agents[0].kwargs["distribution_path"]) + assert destination == Path(agents[1].kwargs["distribution_path"]) + assert distribution_digest(distribution_files(destination)) == inputs["distribution_sha256"] + assert list(destination.parent.iterdir()) == [destination] + + +def test_concurrent_cache_winner_is_rejected_if_corrupt(tmp_path, inputs, monkeypatch): + cache = tmp_path / "cache" + record = tmp_path / "job-input" + record.mkdir() + destination = cache / "pi-distributions" / inputs["distribution_sha256"] + + def publish_corrupt_winner(source, target): + target.mkdir() + (target / "pi").write_text("corrupt") + (target / "package.json").write_text('{"version": "1.4.0"}') + raise OSError(errno.ENOTEMPTY, "Directory not empty") + + monkeypatch.setattr(Path, "rename", publish_corrupt_winner) + with pytest.raises(ValueError, match="Cached Pi distribution is corrupt"): + snapshot_pi(SimpleNamespace(kwargs=inputs.copy()), cache, record) + assert list(destination.parent.iterdir()) == [destination] + + +def test_install_verifies_binary_and_provider_without_calling_model(tmp_path, inputs): + agent = adapter(tmp_path, inputs) + env = Environment() + asyncio.run(agent.install(env)) + record = json.loads((tmp_path / "bootstrap.json").read_text()) + assert record["status"] == "ready" + assert record["distribution_sha256"] == inputs["distribution_sha256"] + assert record["verifier_preflight"]["status"] == "not_configured" + assert env.calls[-1]["command"].endswith("--list-models deepseek-flash") + assert env.calls[-1]["env"]["DEEPSEEK_API_KEY"] == "preflight-only" + assert not any("--print" in call["command"] for call in env.calls) + + +@pytest.mark.parametrize("listing,warning", [ + ("provider model\ndeepseek deepseek-flash\n", "Warning: errors loading models.json: maxTokens must be number"), + ("No models found", ""), + ("provider model\nother deepseek-flash\n", ""), + ("provider model\ndeepseek other-model\n", ""), +]) +def test_install_rejects_model_fallback_or_missing_registration(tmp_path, inputs, listing, warning): + environment = Environment() + environment.model_listing = listing + environment.model_warning = warning + with pytest.raises(ValueError, match="Pi model registration failed"): + asyncio.run(adapter(tmp_path, inputs).install(environment)) + assert json.loads((tmp_path / "bootstrap.json").read_text())["status"] == "failed" + + +def test_install_probes_the_requested_model_instead_of_a_fixed_id(tmp_path, inputs): + models = Path(inputs["model_config_path"]) + models.write_text(models.read_text().replace('deepseek-flash', 'registered-alternate')) + agent = PiCodeAgent(logs_dir=tmp_path, model_name="deepseek/registered-alternate", **inputs) + environment = Environment() + environment.model_listing = "provider model\ndeepseek registered-alternate\n" + asyncio.run(agent.install(environment)) + assert environment.calls[-1]["command"].endswith("--list-models registered-alternate") + + +def test_install_refuses_modified_binary(tmp_path, inputs): + agent = adapter(tmp_path, inputs) + Path(inputs["distribution_path"], "pi").write_text("changed") + with pytest.raises(ValueError, match="changed after registration"): + asyncio.run(agent.install(Environment())) + assert json.loads((tmp_path / "bootstrap.json").read_text())["status"] == "failed" + + +def test_native_run_preserves_failure_status_and_uses_only_environment_key(tmp_path, inputs, monkeypatch): + monkeypatch.setenv("DEEPSEEK_API_KEY", "test-secret") + agent = adapter(tmp_path, inputs) + env = Environment() + asyncio.run(agent.run("Handle '$HOME' and `literal`", env, AgentContext())) + call = env.calls[0] + assert "set -euo pipefail" in call["command"] + assert "--thinking high" in call["command"] + assert "--session-dir /logs/agent/pi/sessions" in call["command"] + assert "test-secret" not in call["command"] + assert call["env"]["DEEPSEEK_API_KEY"] == "test-secret" + assert "--system-prompt" not in call["command"] + + +def test_usage_includes_compaction_and_cache_without_pricing_missing_calls_as_zero(tmp_path, inputs): + agent = adapter(tmp_path, inputs) + usage = {"input": 10, "output": 5, "cacheRead": 20, "cacheWrite": 3, "cost": {"total": 0.2}} + (tmp_path / "pi.txt").write_text("\n".join(json.dumps(event) for event in [ + {"type": "message_end", "message": {"role": "assistant", "usage": usage}}, + {"type": "compaction_end", "result": {"usage": usage}}, + {"type": "message_end", "message": {"role": "assistant"}}, + ])) + context = AgentContext() + agent.populate_context_post_run(context) + assert context.n_input_tokens == 66 + assert context.n_output_tokens == 10 + assert context.n_cache_tokens == 40 + assert context.cost_usd == 0.4 + assert context.metadata["pi"]["missing_usage_records"] == 1 + assert context.metadata["pi"]["cost_is_estimated"] is True + assert context.metadata["pi"]["cost_is_partial"] is True + + +@pytest.mark.parametrize("completion", [ + {"settled": True, "stop_reason": "error"}, + {"settled": False, "stop_reason": "stop"}, + {"settled": True, "stop_reason": None}, +]) +def test_zero_exit_with_failed_or_missing_response_is_an_execution_error(tmp_path, inputs, monkeypatch, completion): + monkeypatch.setenv("DEEPSEEK_API_KEY", "test-secret") + environment = Environment() + environment.completion = completion + with pytest.raises(RuntimeError, match="did not finish successfully"): + asyncio.run(adapter(tmp_path, inputs).run("test", environment, AgentContext())) + + +def test_missing_output_keeps_usage_unknown(tmp_path, inputs): + context = AgentContext() + adapter(tmp_path, inputs).populate_context_post_run(context) + assert context.n_input_tokens is None + assert context.cost_usd is None + + +@pytest.mark.parametrize("stop_reason", ["error", "aborted"]) +def test_failed_stream_zero_usage_stays_partial_after_successful_retry(tmp_path, inputs, stop_reason): + (tmp_path / "process-supervision.json").write_text(json.dumps({"status": "completed", "return_code": 0})) + zero = {key: 0 for key in ["input", "output", "cacheRead", "cacheWrite", "totalTokens"]} + zero["cost"] = {"total": 0} + known = {"input": 10, "output": 5, "cacheRead": 20, "cost": {"total": .2}} + events = [{"type": "message_end", "message": { + "role": "assistant", "stopReason": reason, + "content": [{"type": "text", "text": "Streamed text"}], "usage": usage, + }} for reason, usage in [(stop_reason, zero), ("stop", known)]] + (tmp_path / "pi.txt").write_text("\n".join(map(json.dumps, events))) + context = AgentContext() + adapter(tmp_path, inputs).populate_context_post_run(context) + assert context.n_input_tokens == 30 + assert context.n_output_tokens == 5 + assert context.n_cache_tokens == 20 + assert context.cost_usd == .2 + assert context.metadata["pi"]["usage_records"] == 1 + assert context.metadata["pi"]["missing_usage_records"] == 1 + assert context.metadata["pi"]["usage_is_partial"] is True + assert context.metadata["pi"]["cost_is_partial"] is True + + +def test_failed_stream_only_keeps_token_and_cost_totals_unknown(tmp_path, inputs): + (tmp_path / "pi.txt").write_text(json.dumps({"type": "message_end", "message": { + "role": "assistant", "stopReason": "error", "content": [{"type": "text", "text": "Partial"}], + "usage": {"input": 0, "output": 0, "cost": {"total": 0}}, + }})) + context = AgentContext() + adapter(tmp_path, inputs).populate_context_post_run(context) + assert context.n_input_tokens is None + assert context.cost_usd is None + assert context.metadata["pi"]["usage_is_partial"] is True + assert context.metadata["pi"]["cost_is_partial"] is True + + +def test_missing_cost_keeps_the_reported_cost_subtotal(tmp_path, inputs): + events = [{"type": "message_end", "message": {"role": "assistant", "usage": usage}} for usage in [ + {"input": 10, "output": 5, "cost": {"total": .2}}, {"input": 10, "output": 5}, + ]] + (tmp_path / "pi.txt").write_text("\n".join(map(json.dumps, events))) + context = AgentContext() + adapter(tmp_path, inputs).populate_context_post_run(context) + assert context.n_input_tokens == 20 + assert context.cost_usd == .2 + assert context.metadata["pi"]["usage_is_partial"] is False + assert context.metadata["pi"]["cost_is_partial"] is True + + +def test_transport_cancellation_waits_for_root_process_cleanup(tmp_path, inputs, monkeypatch): + monkeypatch.setenv("DEEPSEEK_API_KEY", "test-secret") + class CancelledEnvironment(Environment): + async def exec(self, **kwargs): + self.calls.append(kwargs) + if len(self.calls) == 1: + raise asyncio.CancelledError() + return SimpleNamespace(return_code=0, stdout="", stderr="") + environment = CancelledEnvironment() + with pytest.raises(asyncio.CancelledError): + asyncio.run(adapter(tmp_path, inputs).run("test", environment, AgentContext())) + assert environment.calls[-1]["user"] == "root" + assert "cancel" in environment.calls[-1]["command"] + assert environment.calls[-1].get("env") is None + + +def test_cancelled_run_marks_known_usage_and_cost_as_partial(tmp_path, inputs): + (tmp_path / "process-supervision.json").write_text(json.dumps({"status": "cancelled", "processes_stopped": True})) + (tmp_path / "pi.txt").write_text(json.dumps({"type": "message_end", "message": { + "role": "assistant", "usage": {"input": 10, "output": 5, "cost": {"total": .1}}, + }})) + context = AgentContext() + adapter(tmp_path, inputs).populate_context_post_run(context) + assert context.cost_usd == .1 + assert context.metadata["pi"]["usage_is_partial"] is True + assert context.metadata["pi"]["cost_is_partial"] is True diff --git a/docs/changelogs/0.10.x.md b/docs/changelogs/0.10.x.md index 951071b..cb7ecf6 100644 --- a/docs/changelogs/0.10.x.md +++ b/docs/changelogs/0.10.x.md @@ -8,6 +8,9 @@ All notable changes in the **0.10.x** release series are documented here. - Add retained-workspace Harbor verification, reviewed dependency preparation with independent deadlines, and opt-in collection-error recovery while preserving native verifier limits and prior evidence. +- Add pinned Pi agent baselines to the standard Harbor workflow, with offline + model-registration checks, verified image caching, process supervision before + grading, and explicit partial usage and cost accounting. ### Changed - Clarify headless task contracts, output verification, and permitted references.