diff --git a/skills/agent-dispatch/SKILL.md b/skills/agent-dispatch/SKILL.md index fd9331d1..28a62981 100644 --- a/skills/agent-dispatch/SKILL.md +++ b/skills/agent-dispatch/SKILL.md @@ -152,48 +152,41 @@ ran it, and the lead passed it on. When run, it mistyped week dates and `+HH:MM` crashed on year-1 offsets. The same claim had already been disproved on 10-15: the Python version needed a regex and grew to 74 lines. -**The verification is one self-dispatched agent-log COPY that runs both sides.** The inputs and -the signature are declared once, in the first CTE. The dispatched statement is the agent-log COPY, -so both sides run as typed columns inside it and land as signed rows. The receipt says only -whether the write happened, and the readback reads the rows as columns (compendium: "each -statement writes them and the next stage reads the files"). +**The verification is one self-dispatched query that runs both sides.** The inputs are declared +once, as rows in the first CTE. Each input gets its DuckDB answer and renders the proposal as a +one-input Python statement, dispatched beside it, so both sides come back in the receipts, nothing +written to read back. Each receipt stays whole (`receipt.*`, body included), each row it returned is +one `result_row`, and a failed dispatch stays as one row with a NULL `result_row`. The last SELECT +names the compared field with one `->>` (RULES.md 5). ```sql -WITH verification AS ( -- 1. plain values, declared once - SELECT [] AS raws, to_json(raws)::VARCHAR AS raws_json, - '' AS agent, '' AS session_id, '' AS skill, - $n$$n$ AS notes, '' AS log_dir -), sides AS ( -- 2. the program under test, rendered once as a column - SELECT *, tera_render($tera$ UNION ALL BY NAME $tera$, v::JSON, autoescape := false) AS both_sides - FROM verification AS v -), rendered AS ( -- 3. the agent-log COPY: both_sides stored as text and run as a subquery - SELECT tera_render($tera$COPY (SELECT uuidv7() AS row_id, 'verification' AS type, $v${{ agent }}$v$ AS agent, - $v${{ session_id }}$v$ AS session_id, $v${{ agent }}-{{ session_id }}$v$ AS agent_signature, - $v${{ skill }}$v$ AS skill, $query${{ both_sides }}$query$ AS query_was_ran, r.*, - $n${{ notes }}$n$ AS markdown_notes, now() AS ts - FROM ({{ both_sides }}) AS r) - TO $v${{ log_dir }}$v$ (FORMAT parquet, PARTITION_BY (type, agent_signature), OVERWRITE_OR_IGNORE true, FILENAME_PATTERN '{uuid}')$tera$, - s::JSON, autoescape := false) AS sql - FROM sides AS s -), dispatched AS ( -- 4. post it - SELECT array_agg(selfdispatch(sql)) AS receipts FROM rendered +WITH intent AS ( -- the inputs, declared once as rows + SELECT unnest(['2026-10-18', '2026-02-30', $v$O'Brien$v$]) AS raw +), rendered AS ( -- per input: the DuckDB answer, and the proposal as a one-input statement + SELECT raw, try_strptime(raw, '%Y-%m-%d')::DATE::VARCHAR AS duckdb_answer, + concat($py$SELECT * FROM read_json($cmd$/opt/homebrew/bin/uv run --quiet --no-project python - <<'PY' +import json, datetime +print(json.dumps(dict(python_answer=datetime.date.fromisoformat($py$, to_json(raw), $py$).isoformat()))) +PY +|$cmd$, format := 'newline_delimited')$py$) AS sql + FROM intent +), dispatched AS ( -- run them, each receipt beside its input + SELECT raw, duckdb_answer, array_agg(selfdispatch(sql)) AS receipts FROM rendered GROUP BY ALL +), result_rows AS ( -- every receipt, and each row it returned + SELECT * EXCLUDE (receipts, receipt), receipt.* + FROM dispatched + CROSS JOIN UNNEST(receipts) AS t(receipt) + CROSS JOIN UNNEST(coalesce(try_cast(receipt.body AS JSON[]), [NULL::JSON])) AS r(result_row) ) -SELECT receipt.* FROM dispatched CROSS JOIN UNNEST(receipts) AS t(receipt); +SELECT raw, duckdb_answer, result_row ->> 'python_answer' AS python_answer, status, body, + row(duckdb_answer, 200) IS DISTINCT FROM row(python_answer, status) AS differs +FROM result_rows +ORDER BY differs DESC, raw ``` -`` is ` UNION ALL BY NAME ` over the same declared list: -- **DuckDB side:** `SELECT 'duckdb' AS side, raw, AS - FROM (SELECT unnest($raws${{ raws_json }}$raws$::JSON::VARCHAR[]) AS raw)`. -- **Python side:** `SELECT 'python' AS side, * FROM read_json($cmd$/opt/homebrew/bin/uv run --quiet - --no-project python - <<'PY' … for raw in {{ raws_json }}: … PY |$cmd$, format := 'newline_delimited')`. - The script prints one JSON line per input, and for an input that raises it prints the exception - type in `error`, so every input is a row on both sides. The reader infers the columns. - -The program is rendered once, so the stored `query_was_ran` is exactly what ran, and the signature -is built from the declared agent and session id. Values render inside dollar quotes (`$v$`, -`$raws$`, `$n$`), so inputs and notes may contain a single quote. A worker testing ponytail's or -Thermos's proposal writes it as the Python (or bash) side. Worked example (verified on dev -2026-10-18, an input and the notes containing `'`): `skill = 'T4-check-dispatched-v3'` under -`~/.duck/agent_log/skill_trial_2026_10_17/type=verification`. -Readback: `query/references/useful_queries.sql`, "Verification: both sides". +Swap in the real inputs, the DuckDB expression and the proposal (`fromisoformat` here). There is no +loop and no `try`: an input the proposal raises on comes back as its own failed receipt (`status` +500, the error in `body`) and counts as `differs`. A worker testing ponytail's or Thermos's proposal +writes it as the Python (or bash) statement. Verified on dev 2026-10-19: three rows; `2026-10-18` +agrees, `2026-02-30` and `O'Brien` differ (Python raised). ## 4. Evidence is false-first diff --git a/skills/query/references/useful_queries.sql b/skills/query/references/useful_queries.sql index 929a81fa..56cefa11 100644 --- a/skills/query/references/useful_queries.sql +++ b/skills/query/references/useful_queries.sql @@ -241,22 +241,3 @@ WITH c AS ( SELECT array_agg(selfdispatch(sql)) AS receipts FROM rendered ) SELECT receipt.* FROM dispatched CROSS JOIN UNNEST(receipts) AS t(receipt); - --- ## Verification: both sides of a DuckDB-vs-Python claim, joined on the input (Opus 5.5, 2026-10-17) --- when: reading back an agent-log `type=verification` row whose query was UNION ALL BY NAME (agent-dispatch section 3). --- uses: read_parquet (core). --- returns: one row per input: raw, duckdb_row and python_row (each side's whole logged row as a STRUCT, query_was_ran and markdown_notes included), missing_duckdb, missing_python, differs BOOLEAN; a missing_* row is an input only one side produced. is the output both sides print (value_type in the example). --- tested: T4 fromisoformat proposal vs the INF-1535 classifier, 19 inputs, 3 disagreements (all Python errors). -WITH logged AS ( - SELECT * FROM read_parquet('/type=verification/**/*.parquet', hive_partitioning := true, union_by_name := true) - WHERE skill = '' -), duckdb_side AS ( - SELECT * FROM logged WHERE side = 'duckdb' -), python_side AS ( - SELECT * FROM logged WHERE side = 'python' -) -SELECT raw, duckdb_row, python_row, - duckdb_row.side IS NULL AS missing_duckdb, python_row.side IS NULL AS missing_python, - duckdb_row. IS DISTINCT FROM python_row. AS differs -FROM duckdb_side AS duckdb_row FULL JOIN python_side AS python_row USING (raw) -ORDER BY differs DESC, raw;