From 23802ec38e00ec050df1c7b3bd43af974912f990 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 13:35:43 -0600 Subject: [PATCH 01/20] Add a generic agent-skills host with an access-aware preflight MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The kumo-data-science skill shipped only to Snowflake CoCo, and its "portable" core assumed Snowflake throughout. This adds a `generic` host for coding agents beside the user (Claude Code, Codex) that generates code for any of the six kumo.engine connectors. - Preflight §0: before inferring structure, the agent proves it can read source_scope with a read-only tool it already has. If not, it says what it checked and asks for access or for the structure, never for a credential. - hosts/generic/: adapter (source forms, access checks per system, destinations) and runtime (config cell with placeholders, connect step, destinations and read-back per connector). - Core no longer names a host; source and destination forms are host-defined. The SDK reference gains §3.4 (non-Snowflake connectors) and §9.2 (destinations by connector). - package_skill.py builds `generic`; tests check that the core names no host, that each package ships only its own host, and that every adapter covers what the core defers. Co-Authored-By: Claude Opus 5.5 --- agent-skills/DESIGN.md | 16 ++ agent-skills/README.md | 16 +- agent-skills/scripts/package_skill.py | 6 + agent-skills/skills/hosts/generic/adapter.md | 134 +++++++++++++++ agent-skills/skills/hosts/generic/runtime.md | 161 ++++++++++++++++++ .../skills/hosts/snowflake-coco/adapter.md | 18 ++ .../skills/kumo-data-science/SKILL.md | 40 +++-- .../accept/batch_inference.md | 15 +- .../accept/intake_contract.md | 16 +- .../kumo-data-science/accept/preflight.md | 55 +++++- .../validate/examples/bad_request.md | 2 +- .../skills/kumo-model-explanations/SKILL.md | 2 +- .../skills/kumo-model-tuning/SKILL.md | 2 +- .../skills/kumo-training-diagnostics/SKILL.md | 2 +- agent-skills/skills/kumoai-reference/SKILL.md | 6 +- .../references/graph_construction.md | 19 ++- .../references/kumo_finetuning_sdk.md | 109 ++++++++++-- .../kumoai-reference/references/outputs.md | 6 +- .../references/task_journeys.md | 2 +- .../references/troubleshooting_signals.md | 2 +- agent-skills/tests/test_package_skill.py | 29 ++++ 21 files changed, 582 insertions(+), 76 deletions(-) create mode 100644 agent-skills/skills/hosts/generic/adapter.md create mode 100644 agent-skills/skills/hosts/generic/runtime.md diff --git a/agent-skills/DESIGN.md b/agent-skills/DESIGN.md index 78f4ca2f8..10aaf11d0 100644 --- a/agent-skills/DESIGN.md +++ b/agent-skills/DESIGN.md @@ -42,5 +42,21 @@ rules; this is why they exist. | Predict in notebook | one submission call, no execution | the predict call is part of the artifact (router Step 2) | user decision 2026-09-23 | | Snowflake ML Jobs | no scheduling | the artifact may run as an ML Job on request; a recurring run is proposed and created only on explicit confirmation (`hosts/snowflake-coco/ml_job.md`) | spec feature 7 (2026-10-13). How Kumo reaches the job container is pending; the skill names it only as host values (`adapter.md` §10) | +| Preflight source | read-only SQL against the user's source | on the generic host, whatever read tool the agent has; with none, the structure the user pastes (`preflight.md` §0) | added 2026-10-20 with the generic host. A coding agent beside the user often has no warehouse session; asking for access or details keeps the gate instead of guessing | + Accommodation, not deviation: the spec has the user select the table mode. Nobody does, so it defaults to `all-connected` and is confirmed at Node 1c. + +## Hosts and the access check + +The core says *what* intake and code generation need; a host says *how* in its `adapter.md` +(intake) and `runtime.md` (Step 2). A package ships one host, so the agent never weighs another +host's rules, and the core names it only as `hosts//` (`tests/test_package_skill.py`). + +Preflight §0 asks the agent to prove it can read the source before inferring anything from it. +CoCo always can — its SQL is the read tool — so §0 only adds the grant-or-details ask-back for an +unreadable scope. A coding agent may have an authenticated CLI, a tool server, Python with Kumo, +or nothing. Without the check it falls back on its priors, the failure seen when CoCo read an +empty skill tree. The agent's access is never the artifact's: the generated code authenticates +where it runs, so §0 changes no gate and no generated line. Credentials stay with the user — +the agent may ask them to authenticate a tool, never for a value. diff --git a/agent-skills/README.md b/agent-skills/README.md index c47199823..f52a24faf 100644 --- a/agent-skills/README.md +++ b/agent-skills/README.md @@ -1,8 +1,12 @@ # Agent skills: `kumo-data-science` Agent skill for building, training, and running inference with Kumo models. One shared source tree -serves every host; host-specific behavior lives under `skills/hosts/`. Currently shipped to -Snowflake Cortex Code (CoCo). +serves every host; host-specific behavior lives under `skills/hosts/`. Two hosts: + +| Host | Package | For | +|---|---|---| +| `hosts/snowflake-coco/` | `coco` | Snowflake Cortex Code (CoCo), inside Snowflake | +| `hosts/generic/` | `generic` | a coding agent beside the user — Claude Code, Codex — generating code for any of the six Kumo connectors | This directory is deliberately not under `.agents/skills/` or `.claude/skills/`: Codex, Claude Code and the CoCo CLI load those for repo developers, and this skill triggers on "kumo" and "train a @@ -19,7 +23,7 @@ are sub-skills and references it loads: |---|---| | `kumo-data-science/` | Router: intake, validation, preflight, notebook generation | | `kumoai-reference/` | SDK reference the generated code is checked against | -| `hosts/` | Host-specific behavior (currently Snowflake CoCo) | +| `hosts/` | Host-specific behavior: access, source and destination forms, preflight, connect step | | `kumo-model-explanations/` | Explaining predictions | | `kumo-training-diagnostics/` | Diagnosing training runs | | `kumo-model-persistence/` | Saving and reloading trained models | @@ -34,9 +38,15 @@ path. Build a host package with: ```bash python3 agent-skills/scripts/package_skill.py coco # -> agent-skills/dist/coco/ +python3 agent-skills/scripts/package_skill.py generic # -> agent-skills/dist/generic/skills/ python3 agent-skills/scripts/package_skill.py --check # build every host, verify references ``` +A package carries exactly one host, and the core reaches it only as `hosts//`; tests fail if +a core file names a host. To use the `generic` package, copy +`dist/generic/skills/kumo-data-science/` into the agent's user-level skills directory (for Claude +Code, `~/.claude/skills/`) — not into this repository, for the reason above. + The build stamps the router's `SKILL-BUILD` marker with the source commit (`-`, `-dirty` for uncommitted edits); never edit it by hand. Every build fails if a markdown link or a backticked package path does not resolve inside the diff --git a/agent-skills/scripts/package_skill.py b/agent-skills/scripts/package_skill.py index e837536c6..fc67661d7 100755 --- a/agent-skills/scripts/package_skill.py +++ b/agent-skills/scripts/package_skill.py @@ -5,6 +5,7 @@ """Build the kumo-data-science skill package for one host and check that it resolves. python3 agent-skills/scripts/package_skill.py coco # build into agent-skills/dist/ + python3 agent-skills/scripts/package_skill.py generic # Claude Code, Codex, other agents python3 agent-skills/scripts/package_skill.py coco --out DIR # build somewhere else python3 agent-skills/scripts/package_skill.py --check # build every host in a temp dir @@ -49,6 +50,11 @@ hosts=['snowflake-coco/adapter.md', 'snowflake-coco/runtime.md', 'snowflake-coco/ml_job.md'], capability=True), + # A coding agent outside any data platform. The package root holds the one skill directory, + # ready to copy into the agent's skills directory. + 'generic': dict(prefix='skills', + hosts=['generic/adapter.md', 'generic/runtime.md'], + capability=True), } MARKER = re.compile(r'SKILL-BUILD: [^`*\s]+') diff --git a/agent-skills/skills/hosts/generic/adapter.md b/agent-skills/skills/hosts/generic/adapter.md new file mode 100644 index 000000000..c30b6f383 --- /dev/null +++ b/agent-skills/skills/hosts/generic/adapter.md @@ -0,0 +1,134 @@ +# Host Adapter — Generic coding agent + +For a coding agent working beside the user — Claude Code, Codex, or any agent with a shell and +file tools — rather than inside a data platform. The agent produces code that runs in **any** +Python environment where Kumo is present, against whichever of the six Kumo connectors the data +calls for. The agent does not know where that code will run; the user does. + +Kumo is already present in the environment that runs the artifact; never raise environment setup +with the user. Generating the artifact — connect step, credentials, destinations and read-back — +is in `runtime.md` in this directory; read it at Step 2. + +This host has no job file: it does not submit, schedule or monitor jobs. When asked for one, say +so and offer the notebook or script. + +--- + +## 1. Source scope + +`source_scope` names a system and a location in it. Map it to its connector +(`kumoai-reference/references/kumo_finetuning_sdk.md` §3, §3.4): + +| The user names | Connector | +|---|---| +| A local directory of CSV or Parquet files | `FileConnector` | +| `s3://bucket/prefix` | `S3Connector` | +| `gs://bucket/prefix` | `GCSConnector` | +| A BigQuery project and dataset | `BigQueryConnector` | +| A Databricks catalog and schema | `DatabricksConnector` | +| A Snowflake database and schema | `SnowflakeConnector` | + +- A dotted name with no system named (`SALES.EVENTS`) fits Snowflake, BigQuery and Databricks + alike. Ask which system holds it — that one question, nothing else. A system already named in + any turn, or evident from the user's tools or files, settles it. +- A connector name on its own is not a location on this host, which keeps no registry of named + connectors. Ask what system and location it points at. +- A Snowflake semantic view resolves through `kumo-data-science/accept/semantic_view.md`, which + needs Snowflake SQL (§2). Without it, the §2 details are the view's base tables, their keys and + its relationships. +- Values only the runtime needs — a Snowflake warehouse, a Databricks SQL warehouse or cluster, a + `connections.toml` entry — are not part of `source_scope` and are never asked for at intake. + `runtime.md` §1 handles them. + +ADLS has no Kumo connector. Data there is outside this host's reach: say so. + +--- + +## 2. Access + +Implements `kumo-data-science/accept/preflight.md` §0. Check, in this order, and stop at the +first that reads `source_scope`: + +1. **A tool in this session** that reads the system — a server or connector tool for that + warehouse or storage. Use only its read operations. +2. **A command-line tool on the `PATH`** that is already authenticated (table below). Find it + with `command -v `, then run its check command against `source_scope` itself — a + general "who am I" call proves the login, not access to the data. +3. **Python in this session** in which `import kumo.engine` works and the provider's ambient + credentials resolve. Build the connector exactly as the artifact would (`runtime.md` §2) and + read `connector.table_names()` and `connector[''].columns`, which gives each column's + name and source type. Catalog reads only: no `Table`, `Graph` or snapshot. + +| Source | Tool | Check, inside `source_scope` | Columns and types | Declared keys | §3 probe | +|---|---|---|---|---|---| +| Local files | your file tools; `duckdb`, or Python with `pyarrow` | list the directory | CSV header line; Parquet schema (`duckdb -c "DESCRIBE SELECT * FROM ''"`) | none | `duckdb`, or `pyarrow` | +| S3 | `aws` | `aws s3 ls s3:////` | file names only; for columns, a reader of `s3://` with the same credentials (`duckdb`), else §2 details | none | the same reader, else skip | +| GCS | `gcloud` | `gcloud storage ls gs:////` | as S3, with `gs://` | none | as S3 | +| BigQuery | `bq` | `bq ls :` | `bq show --schema --format=prettyjson :.
` | `INFORMATION_SCHEMA.TABLE_CONSTRAINTS` and `KEY_COLUMN_USAGE` of the dataset | `bq query --nouse_legacy_sql` | +| Databricks | `databricks` | `databricks tables list ` | `databricks tables get ..
` | the same output's table constraints | only through a SQL warehouse the user named; else skip | +| Snowflake | `snow` | `snow sql -q ""` | the preflight §2 query | `SHOW PRIMARY KEYS` / `SHOW IMPORTED KEYS IN SCHEMA .` | `snow sql -q` | + +A skipped probe leaves its candidates `[inferred, unverified]`, as preflight §0 says. A BigQuery +probe is billed by the bytes it scans; probe only the candidate columns. + +**What a check may do.** Read catalog metadata and run the preflight §3 probe — nothing else. +Never run a command that writes, creates, copies or exports data; never sign in, switch a +profile or account, or edit a tool's configuration; never add or update software to get a +reader. Tool permission prompts are the user's to answer; a refused prompt means that path is +unavailable. + +**When nothing reads the source**, send the preflight §0 ask-back. State what you checked and +what happened — "no BigQuery tool in this session; `bq ls my-proj:sales` returned `Access +Denied`" — then offer: + +1. **Access**: authenticate the named command-line tool as an identity that can read the scope, + or connect a tool for that system to this session, and tell you when it is done. The user + does this; you never handle the credential. +2. **Details**: paste the structure. No rows are needed. Useful forms: DDL, `DESCRIBE` output, + `bq show --schema` JSON, a Parquet schema, CSV header lines — with any declared keys. + +--- + +## 3. Preflight + +Answer `kumo-data-science/accept/preflight.md` §1-§6 with the tool §2 found. Write table and +column names exactly as that tool returns them; the artifact must use the same spelling +(`kumoai-reference/references/kumo_finetuning_sdk.md` §3.2). Probe results are evidence for the +proposal only: never written, never saved to a file, never fed to the artifact. + +For files, a table is a file or a directory of parts in one format, named by its path relative to +the source directory — the name `FileConnector` uses. + +--- + +## 4. Destinations + +Any destination a Kumo connector writes +(`kumoai-reference/references/kumo_finetuning_sdk.md` §9.2): a table in Snowflake, BigQuery or +Databricks, or a path in a local directory, S3 or GCS. The destination may be in a different +system from the source; it gets its own connector. Not destinations here: ADLS, and BigQuery for +anything but predictions and embeddings. + +--- + +## 5. Runtime column constraints + +None known for this host. Do not exclude text columns pre-emptively. If a run fails while loading +a text encoder, report the error and the column; that finding belongs at Node 1c. + +--- + +## 6. Capability skills + +The capability skills were written for, and measured on, Snowflake. Their Kumo calls hold for +every connector. Where a step names a Snowflake registry, stage, session or table, follow it only +when the source or destination is Snowflake. For any other connector, say that the step is not +covered on this host; do not adapt it. + +--- + +## 7. Other deployment modes — out of scope + +Hosted Kumo — `kumo.init(...)`, an API key, `KUMO_API_KEY` / `KUMO_API_ENDPOINT`, Kumo Studio — and +the Snowflake Native App are not this SDK. If the user is on one of them, say so and stop. Never +blend their setup into generated code. diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md new file mode 100644 index 000000000..2f63ce2dd --- /dev/null +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -0,0 +1,161 @@ +# Host Runtime — Generic coding agent + +Read at Step 2, when generating the artifact: where it runs, the config cell, the connect step, +credentials, destinations and read-back. Intake is in `adapter.md`. + +--- + +## 0. Where the code runs + +Unknown to you, and it does not matter: the artifact must run unchanged in any Python environment +where `import kumo.engine` works and the source's credentials resolve — a workstation, a VM, a +hosted notebook. So it assumes no platform: + +- no Snowpark session from `get_active_session()`, no `dbutils`, no SPCS token, no notebook + magics, no platform secret store; +- no path outside the ones confirmed at intake, except Kumo's scratch default + (`kumoai-reference/references/kumo_finetuning_sdk.md` §2.1). + +Your own session's access (`adapter.md` §2) says nothing about the runtime's. The connect step +resolves credentials where the code runs. + +Write the artifact to the path the user names, else to the current working directory, named after +the task (`churn_30d_train.ipynb`). Write only that file. Creating it is not running it: Step 3 +still waits for an explicit request. + +--- + +## 1. Config cell + +The first code cell holds the imports and every location as a named constant: the confirmed +source, the confirmed destination, and any population table — each written verbatim. + +Values only the runtime knows — a Snowflake warehouse or `connections.toml` entry, a Databricks +SQL warehouse or cluster — are constants here too. Write the user's value if they gave one; +otherwise a placeholder of the form `''`, and list each placeholder in +your reply as something to fill in before running. A placeholder never stands in for a confirmed +value, and never for a secret. + +End the cell with a guard, so a run with an unfilled value stops before reading anything: + +```python +CONFIG = dict(SOURCE_DIR=SOURCE_DIR, OUTPUT_DIR=OUTPUT_DIR) # every constant in this cell +unfilled = [k for k, v in CONFIG.items() if isinstance(v, str) and v.startswith('<')] +assert not unfilled, f'Fill in before running: {unfilled}' +``` + +--- + +## 2. Connect step + +One connector per system, built from the confirmed `source_scope` with ambient credentials +(`kumoai-reference/references/kumo_finetuning_sdk.md` §3, §3.4). Signatures are there; do not +add arguments the user did not confirm. + +```python +# Local files +connector = kumo.FileConnector(root_dir=SOURCE_DIR) +# S3 / GCS — rooted at the confirmed prefix, so table_names() works +connector = kumo.S3Connector(root_dir=SOURCE_URI) # 's3://bucket/prefix/' +connector = kumo.GCSConnector(root_dir=SOURCE_URI) # 'gs://bucket/prefix/' +# BigQuery +connector = kumo.BigQueryConnector(project_id=PROJECT_ID, dataset_id=DATASET_ID) +# Databricks — exactly one of warehouse_id / cluster_id +connector = kumo.DatabricksConnector( + catalog=CATALOG, schema_name=SCHEMA, identity=kumo.DatabricksIdentity(), + warehouse_id=DATABRICKS_WAREHOUSE_ID) +# Snowflake — outside Snowflake's own runtime, through a connections.toml entry +connector = kumo.SnowflakeConnector.from_connection_toml( + connection_name=SNOWFLAKE_CONNECTION, warehouse=SNOWFLAKE_WAREHOUSE, + database=DATABASE, schema_name=SCHEMA) # lower case: keep the quotes, §3 + +assert connector.table_names(), 'Connector reached no tables.' +``` + +Emit only the line for the confirmed system. Never emit a rootless S3 or GCS connector for a +source: it cannot list tables. + +**Credentials.** The ambient forms above name no secret. Use an explicit identity (§3.4) only +when the user asks for one, and then read each secret at run time from an environment variable +whose **name** is a config-cell constant — never a literal, never a value asked for in chat: + +```python +identity = kumo.S3Identity(access_key_id=os.environ[AWS_KEY_ID_VAR], + secret_access_key=os.environ[AWS_SECRET_VAR]) +``` + +The Snowflake factory `from_spcs_env` and Snowflake's notebook secret store belong to +Snowflake's own runtime. Emit them only when the user says the code runs there. + +### 2.1 Storage + +Kumo's scratch directory defaults to `~/.cache/kumo-engine`. Emit `kumo.configure(scratch_dir=...)` +only for a scratch path the user named. A `FileConnector` source or destination must not overlap +it (§3.4). Scratch is disposable: anything that must outlive the run is written to the confirmed +destination. + +--- + +## 3. Destinations and read-back + +Build the output connector for the destination's own system, and the `PredictionOutputConfig` in +that connector's form (`kumoai-reference/references/kumo_finetuning_sdk.md` §9.2). Then read the +destination back and compare its row count to `prediction_result.num_rows` — the only proof the +write worked. + +```python +# Local files, S3, GCS — directory or single-file output +import pyarrow.dataset as ds +written = ds.dataset(OUTPUT_LOCATION, format='parquet').count_rows() # full path or URI + +# BigQuery +from google.cloud import bigquery +written = next(iter(bigquery.Client(project=PROJECT_ID).query( + f'SELECT COUNT(*) FROM `{PROJECT_ID}.{OUTPUT_DATASET}.{OUTPUT_TABLE}`').result()))[0] + +# Databricks — with the confirmed SQL warehouse +from databricks.sdk import WorkspaceClient +response = WorkspaceClient().statement_execution.execute_statement( + warehouse_id=DATABRICKS_WAREHOUSE_ID, wait_timeout='30s', + statement=f'SELECT COUNT(*) FROM {CATALOG}.{OUTPUT_SCHEMA}.{OUTPUT_TABLE}') +written = int(response.result.data_array[0][0]) + +# Snowflake — through the same connections.toml entry +import snowflake.connector +with snowflake.connector.connect(connection_name=SNOWFLAKE_CONNECTION) as conn: + written = conn.cursor().execute( + f'SELECT COUNT(*) FROM {OUTPUT_DATABASE}.{OUTPUT_SCHEMA}.{OUTPUT_TABLE}').fetchone()[0] + +assert written == prediction_result.num_rows, (written, prediction_result.num_rows) +``` + +- `OUTPUT_LOCATION` for files is the destination as written: a rooted connector's relative path + joined to its root. +- A Databricks destination is passed to Kumo as `'__kumo__
'` and read back as + `..
`. With a cluster and no SQL warehouse, there is no read-back here: + say so in the artifact and in your reply. +- Report the destination in its own form, unchanged — `DB.SCHEMA.TABLE`, or the path. + +> **Not verified:** none of these read-backs, and no connector other than Snowflake, has been run +> end to end with this skill. Say so when you hand over an artifact for one of them. + +### 3.1 Static scoring population + +When Node 1d confirmed a population table (`kumoai-reference/references/kumo_finetuning_sdk.md` +§7.1), stage it in the destination's system: + +- **Files, S3, GCS:** read the entity key column from the source with `pyarrow.dataset`, rename it + `ENTITY1`, and write one Parquet file at the confirmed population path. Build the + `PredictionTable` from a connector rooted at that path's directory. +- **Snowflake:** the §7.1 SQL, run through the read-back connection above. +- **BigQuery, Databricks:** not covered on this host. Say so at Node 1d and offer the rows whose + target is NULL instead. + +--- + +## 4. Step 3 — running + +Run only on explicit request (router Step 3), in this session's own Python, and only if +`import kumo.engine` works here and the connect step resolves credentials. If either fails, report +it and stop: making this session able to run Kumo is not your job. `fit` blocks for as long as +training takes — tell the user before starting. diff --git a/agent-skills/skills/hosts/snowflake-coco/adapter.md b/agent-skills/skills/hosts/snowflake-coco/adapter.md index 62262bb8b..2be48c5dc 100644 --- a/agent-skills/skills/hosts/snowflake-coco/adapter.md +++ b/agent-skills/skills/hosts/snowflake-coco/adapter.md @@ -14,6 +14,24 @@ schedules and cost — is in `ml_job.md`; read it only when the user asks for a --- +## 0. Source scope, access and destinations + +**Source scope.** A Snowflake database and schema (warehouse optional), a fully qualified semantic +view (`DB.SCHEMA.VIEW`, resolved by `kumo-data-science/accept/semantic_view.md`), or a connector +name. + +**Access** (`kumo-data-science/accept/preflight.md` §0). The read tool is CoCo's own SQL, run as +the session's role; the check is the §6.1 scope query. A schema missing from `INFORMATION_SCHEMA` +may not exist or may not be visible to this role — say both. On an authorization error, or a +scope the role cannot see, offer the two ways forward: a role with `USAGE` on the database and +schema and `SELECT` on its tables, granted by the user or their administrator; or the structure +pasted in. Never run a `GRANT` yourself. + +**Destinations.** Snowflake tables only: a connector plus a table name, or `DB.SCHEMA.TABLE`. An +`s3://`, `gs://` or file path is not a destination on this host. + +--- + ## 1.4 Runtime-dependent text features In measured Snowflake CoCo runs, retaining a column resolved as `Stype.text` made text-encoder initialization fail with `NoCredentialsError` while resolving GloVe assets. The remedy is the diff --git a/agent-skills/skills/kumo-data-science/SKILL.md b/agent-skills/skills/kumo-data-science/SKILL.md index 7ba0152ae..592c919e9 100644 --- a/agent-skills/skills/kumo-data-science/SKILL.md +++ b/agent-skills/skills/kumo-data-science/SKILL.md @@ -1,6 +1,6 @@ --- name: kumo-data-science -description: "Use for any work with KumoAI predictive models on Snowflake data: train a model, score data with a trained one, or answer questions about an existing model or run — explain its predictions, diagnose its training, publish, reload or retain it, train or score on custom labels or rows, or tune it. Covers churn, propensity, fraud, recommendation, forecasting and link prediction; feature attribution and column cohorts; metrics, trials and learning curves; registry publishing and artifact retention; caller-supplied labels, weights and splits; encoders, run mode and experiments. For a new model it checks that the request names the data (a schema, connector, or semantic view), the outcome and the time window, confirms the inferred tables and the task, then generates one notebook or script; it runs the job only when asked, in the notebook or as a Snowflake ML Job on a compute pool. Triggers: kumo, predict, churn, recommend, forecast, fraud, train a model, run inference, explain a prediction, feature importance, model metrics, publish or load a model, tune a model, semantic view, ML Job, compute pool." +description: "Use for any work with KumoAI predictive models on relational data in a warehouse, an object store or local files: train a model, score data with a trained one, or answer questions about an existing model or run — explain its predictions, diagnose its training, publish, reload or retain it, train or score on custom labels or rows, or tune it. Covers churn, propensity, fraud, recommendation, forecasting and link prediction; feature attribution and column cohorts; metrics, trials and learning curves; registry publishing and artifact retention; caller-supplied labels, weights and splits; encoders, run mode and experiments. For a new model it checks that the request names the data (a schema, dataset, folder of files, connector, or semantic view), the outcome and the time window, confirms the inferred tables and the task, then generates one notebook or script; it runs the job only when asked, in the notebook or, on Snowflake, as an ML Job on a compute pool. Triggers: kumo, predict, churn, recommend, forecast, fraud, train a model, run inference, explain a prediction, feature importance, model metrics, publish or load a model, tune a model, semantic view, ML Job, compute pool." --- # Data Science Agent — Router Skill @@ -34,10 +34,12 @@ for what the selected workflow is missing. If the host has not provided the refe report that limitation rather than inventing its workflow. **New training requests still follow the gates below**, unchanged. -A request to run the artifact as a **Snowflake ML Job** — on a compute pool, unattended, or on a -schedule — or a question about a job already submitted (status, logs, cost) also reads -`hosts/snowflake-coco/ml_job.md`. It adds settings to the gates below and changes where Step 3 -runs; it never replaces the gates. A request that does not mention a job stays in the notebook. +A request to run the artifact as a **job** — a Snowflake ML Job on a compute pool, unattended, or +on a schedule — or a question about a job already submitted (status, logs, cost) also reads the +host's job file, when the host adapter names one. It adds settings to the gates below and changes +where Step 3 runs; it never replaces the gates. If the adapter names no job file, say that this +host does not run jobs and offer the notebook. A request that does not mention a job stays in the +notebook. ## Purpose Take a plain-language ML request, get it to a **confirmed** task and graph through a multi-turn @@ -68,7 +70,7 @@ trainer, and `predict` runs against an explicitly rebuilt graph snapshot (see St - Retry or resume jobs, or keep state across sessions. The one execution the agent performs is Step 3, and only when the user asks. - Schedule anything on its own initiative (a recurring run is proposed and created only on explicit - confirmation; see `hosts/snowflake-coco/ml_job.md` §8) + confirmation, and only on a host whose job file covers it) - RFM / zero-shot workflows - Audit the user's schema for leakage at intake beyond the mandatory availability guards in Node 1c. The codegen guard in `kumoai-reference/SKILL.md` (the target never appears as a @@ -99,7 +101,7 @@ Reference: `accept/intake_contract.md` First decide the operation: `train` or `batch_inference`. **`train` — required (ask if missing):** -- `source_scope` — connector, database + schema, or a semantic view +- `source_scope` — where the tables live, in a form the host accepts (`accept/intake_contract.md`) - `prediction_entity` — what gets one prediction - `prediction_target` — the outcome or interaction, with the positive-label condition - `time_window` — `specified` / `not_critical` (static task) / `missing` → ask only when `missing`. @@ -128,7 +130,7 @@ implements a field they already named** — infer the mapping and show it `[infe | Condition | Transition | |---|---| | Any required field missing | ASK BACK — list exactly the missing fields | -| `source_scope` given but does not resolve (schema or semantic view not found / unreadable) | ASK BACK — ask for a valid source; do not guess a similar name | +| `source_scope` given but does not resolve (location or semantic view not found) | ASK BACK — ask for a valid source; do not guess a similar name | | All required present | proceed to Node 1b | ### Node 1b — Atomic question check @@ -144,7 +146,8 @@ Reference: `validate/atomic_question_check.md` ### Node 1c — Preflight & graph gate Reference: `accept/preflight.md` -Run read-only preflight inside `source_scope`. +First establish that you can read `source_scope` (`accept/preflight.md` §0). Then run read-only +preflight inside it. Present the graph proposal — seeds, tables, primary keys, time columns, links, `table_mode: all-connected [default]` — each line tagged @@ -191,6 +194,7 @@ leaving them unmentioned, because each one changes whether the task is expressib | Condition | Transition | |---|---| +| You cannot read `source_scope` | ASK BACK — say what you checked; offer access or the structure in the user's own words (`accept/preflight.md` §0) | | Seeds cannot be connected without transforming data | REJECT — explain; generate nothing | | A semantic view fails a check in `accept/semantic_view.md` | ASK BACK — as that file says; do not pick a schema, drop tables, or substitute a link | | User corrects a line | re-present the proposal with the correction `[supplied]` | @@ -245,7 +249,7 @@ Source: shipped package (`kumo_core.model_plan`) ## Step 2 — Generate via KumoAI -Read the host's `runtime.md` (`hosts/snowflake-coco/runtime.md`) before generating. +Read the host's `runtime.md` (`hosts//runtime.md`) before generating. After ACCEPT, load `kumoai-reference/SKILL.md` and hand off the confirmed payload. Produce **one** artifact. @@ -282,18 +286,24 @@ metadata, links, destination. No invented link, horizon, or table. Default is generate-only. Run only when the user says to run it, in the same or a later turn. Then, per `kumoai-reference/references/kumo_finetuning_sdk.md`: - Run the notebook. `fit` blocks until training finishes; there is no Kumo job to poll. When the - user asked for a Snowflake ML Job, running means submitting it and waiting for its Snowflake - status instead (`hosts/snowflake-coco/ml_job.md` §6). + user asked for a job, running means submitting it and waiting for its status as the host's job + file describes. - Report the outcome: the training metrics, or the exception and the cell that raised it - Let the notebook's own predict step run; predictions land at the confirmed destination — **the - name is written verbatim**, no suffix, no case change; report it as `DB.SCHEMA.TABLE` + name is written verbatim**, no suffix, no case change; report it in the destination's own + form (`DB.SCHEMA.TABLE`, or a path) - Include embeddings if they were confirmed at Node 1d --- ## Host adapter -Snowflake-specific runtime facts, including how to run preflight SQL, live in -`hosts/snowflake-coco/adapter.md` (intake), and `hosts/snowflake-coco/runtime.md` (read at Step 2). +This package carries exactly one host: the single directory under `hosts/`. Everything specific to +where you run and where the data lives is there — never assume another host's rules. + +- `hosts//adapter.md` — read at intake: how to check access to the source, the source and + destination forms the host accepts, how to run preflight, and runtime column constraints +- `hosts//runtime.md` — read at Step 2: how the generated code connects, authenticates, + writes and verifies --- diff --git a/agent-skills/skills/kumo-data-science/accept/batch_inference.md b/agent-skills/skills/kumo-data-science/accept/batch_inference.md index 492dbd195..215162615 100644 --- a/agent-skills/skills/kumo-data-science/accept/batch_inference.md +++ b/agent-skills/skills/kumo-data-science/accept/batch_inference.md @@ -9,7 +9,8 @@ wants it to score data. A training request never needs it. It adds to `SKILL.md` **`batch_inference` — required (ask if missing):** - `model_address` — the model the user names: a name or ID, a registry location with a version, or a retained local bundle directory -- `output_location` — a Snowflake table; a connector plus a table name is complete +- `output_location` — a destination the active host can write (`hosts//adapter.md`, + *Destinations*); a connector plus a table name is complete **`batch_inference` — resolved, never asked** (`accept/intake_contract.md`): - `source_scope`: a source the user named, else the connector named for this request, else the @@ -31,7 +32,7 @@ the user already said, or defaulted — and shown at the gates for confirmation. | Field | What it must include | |---|---| | **`model_address`** | The model the user names, in their words: a model name or ID ("trainingjob-benchmark-001", "the benchmark model"), a registry location with a version, or a retained local bundle directory | -| **`output_location`** | Where scores are written: a Snowflake table. A connector plus a table name ("connector `benchmark_sf`, table `MODEL_SCORES`") is complete, because the connector supplies the database and schema. A fully qualified `DB.SCHEMA.TABLE` is also complete. | +| **`output_location`** | Where scores are written, in a form the active host can write (`hosts//adapter.md`, *Destinations*). A connector plus a table name ("connector `benchmark_sf`, table `MODEL_SCORES`") is complete, because the connector supplies the database and schema. A fully qualified table (`DB.SCHEMA.TABLE`), or a path on a host that writes files, is also complete. | Both pass the quote test through earlier turns too. "Run the benchmark model again and put the preds in the scores table", after a turn that named the model and the table, supplies both. @@ -41,7 +42,7 @@ preds in the scores table", after a turn that named the model and the table, sup | Field | How it is obtained | Shown as | |---|---|---| | **task** — entity, outcome, window, task type | Reused from the model unchanged. Never reopened. | restated at 1d | -| **`source_scope`** | In order: a source the user named → the connector the user named for this request (one Snowflake connector reads the source and writes the scores unless the user separates them) → the source scope recorded with the model at publication. Ask only if none of the three exists. | `[supplied]` / `[inferred]` at 1c | +| **`source_scope`** | In order: a source the user named → the connector the user named for this request (one connector reads the source and writes the scores unless the user separates them) → the source scope recorded with the model at publication. Ask only if none of the three exists. | `[supplied]` / `[inferred]` at 1c | | **graph** | Rebuilt from the model's recorded graph definition inside `source_scope`, then confirmed | `[inferred]` at 1c | | **`scoring_population`** | Default: every entity of the trained entity table, anchored at the latest timestamp in the rebuilt snapshot ("current predictions"). A static task has no anchor. If the user names a subset or an anchor date, use theirs. | `[default]` / `[supplied]` at 1d | | **model version** | The version the user named. If the address resolves to more than one version and none was named, ask which one. Never pick the latest yourself. | `[supplied]` at 1d | @@ -62,7 +63,7 @@ tagged, so the user can correct it at confirmation. | Condition | Response | |---|---| | `model_address` does not resolve in the registry or as a bundle directory, or the user calls it rejected/invalid | Name it as unusable and ask for a corrected model identifier. Do not guess a similarly named model. | -| `output_location` is not a Snowflake table (an `s3://`, `gs://` or file path) | Name it as unusable here, since scores are written to Snowflake tables, and ask for a Snowflake destination. Do not silently swap in another one. | +| `output_location` is a kind the host cannot write (the adapter's *Destinations* lists what it can) | Name it as unusable here, say what the host writes instead, and ask for such a destination. Do not silently swap in another one. | When one field is invalid and the other is valid, ask only about the invalid one. When one is missing and the other is present, ask only for the missing one. Do not also list the resolved @@ -71,7 +72,7 @@ fields as questions. | Condition | Transition | |---|---| | `batch_inference`: `model_address` does not resolve, or the user calls it rejected | ASK BACK — name it as unusable, ask for a corrected identifier; do not guess | -| `batch_inference`: `output_location` is not a Snowflake table (`s3://`, file path) | ASK BACK — name it as unusable, ask for a Snowflake destination; do not substitute | +| `batch_inference`: `output_location` is a kind the host cannot write | ASK BACK — name it as unusable, ask for a destination the host writes; do not substitute | ## Node 1c — rebuilt graph @@ -148,11 +149,13 @@ With "…export the entity and target-item embeddings" added, the gate also carr accounts at the latest snapshot timestamp, the default), and the task. The task is never reopened. -> "Where should the scores be written? A Snowflake table, e.g. a connector plus a table name, or +> "Where should the scores be written? For example a connector plus a table name, or > `DB.SCHEMA.TABLE`." ### Destination unusable → ASK BACK +On a host whose *Destinations* are Snowflake tables only: + > "Score with `trainingjob-abc123` and write predictions to `s3://tmp/scores`." - `model_address` ✓ · `output_location` invalid: not a Snowflake table diff --git a/agent-skills/skills/kumo-data-science/accept/intake_contract.md b/agent-skills/skills/kumo-data-science/accept/intake_contract.md index 11ca3e54d..f18406721 100644 --- a/agent-skills/skills/kumo-data-science/accept/intake_contract.md +++ b/agent-skills/skills/kumo-data-science/accept/intake_contract.md @@ -53,7 +53,7 @@ Ask only if the request genuinely reads both ways. | Field | What it must include | |---|---| -| **`source_scope`** | A connector name, **or** a Snowflake database + schema (warehouse optional), **or** a fully qualified semantic view (`DB.SCHEMA.VIEW`). This is where tables are enumerated; a semantic view contributes exactly its base tables, which must share one schema (`semantic_view.md`). Credentials are never requested in chat. | +| **`source_scope`** | Where the tables live, in a form the active host accepts (`hosts//adapter.md`, *Source scope*): for example a database + schema, a dataset, a catalog + schema, a bucket prefix or a directory of files, or a connector name the host knows. On a host that supports them, a fully qualified semantic view (`DB.SCHEMA.VIEW`) contributes exactly its base tables, which must share one schema (`semantic_view.md`). This is where tables are enumerated. Credentials are never requested in chat. | | **`prediction_entity`** | What receives one prediction — "each account", "each customer". A physical table/key is welcome but not required; preflight resolves it. | | **`prediction_target`** | The observable outcome (classification/regression) or the positive interaction (link prediction), **stated by the user**. For classification this includes the positive-label condition — "at least one fraud event"; "predict fraud" alone leaves it missing. For link prediction and ranking it includes the **item** being recommended and the **interaction** that counts as positive — "spare-part families the customer ordered"; "recommend to customers" alone leaves both missing. | | **`time_window`** | Tri-state. `specified` ("next 30 days") → pass. `not_critical` → the task is static (no temporal component: static link prediction, static regression/classification) → pass, do not ask. `missing` → the task is temporal and no window was given → ASK BACK. Decide static vs. temporal with the test below. | @@ -76,7 +76,7 @@ If the task is static, restate `Window: static` at the task gate and do not ask. may still be used as the graph's time column — that is structure, not a window. A request that names the dataset ("the e-commerce fraud dataset", a schema name) has supplied -`source_scope` if that name resolves to a database and schema, or to a semantic view. Do not ask the user to restate a +`source_scope` if that name resolves to a location the host accepts. Do not ask the user to restate a location they have already given in an earlier turn. ## `train` — inferred, then confirmed (never ask) @@ -103,9 +103,11 @@ confirmed **before** any PQL is written. If the user supplies an `output_location`, carry it verbatim into the task interpretation gate and the generated artifact — the notebook's predict call writes there. If none is supplied, -predictions stay local and the notebook shows how to add a destination. **The table is written with exactly the name passed** — no suffix, -no case change (`hosts/snowflake-coco/runtime.md` §7, `kumoai-reference/references/kumo_finetuning_sdk.md` -§9.1). Older SDK builds appended `_predictions`; the current one does not. +predictions stay local and the notebook shows how to add a destination. The destination must be +one the host can write (`hosts//adapter.md`, *Destinations*). **It is written with exactly +the name passed** — no suffix, no case change (the host's `runtime.md`, +`kumoai-reference/references/kumo_finetuning_sdk.md` §9.1). Older SDK builds appended +`_predictions`; the current one does not. --- @@ -148,7 +150,7 @@ What makes it good: > "Use my sales data to figure out which customers are risky." What is missing: -- `source_scope`: "sales data" does not resolve to a database and schema ✗ +- `source_scope`: "sales data" does not resolve to a location ✗ - `prediction_target`: "risky" is not an observable outcome ✗ - `time_window`: `missing` for what reads as a temporal question ✗ @@ -167,7 +169,7 @@ about columns or values for the fields that *are* present — those are confirme `[inferred]`. Example: > "To proceed I need: -> 1. **Where the data lives** — a Snowflake database and schema (or a connector name). +> 1. **Where the data lives** — for example a database and schema, or a folder of files. > 2. **The prediction window** — e.g. 'in the next 30 days'." Ask only for missing, ambiguous, or contradictory inputs, restate the resolved contract, and never diff --git a/agent-skills/skills/kumo-data-science/accept/preflight.md b/agent-skills/skills/kumo-data-science/accept/preflight.md index 7c3ed2e3c..5c6447227 100644 --- a/agent-skills/skills/kumo-data-science/accept/preflight.md +++ b/agent-skills/skills/kumo-data-science/accept/preflight.md @@ -1,11 +1,16 @@ # Read-only Preflight — inferring structure before the graph gate Once the required fields are present, the agent resolves **structure** itself. It does not ask -the user for column names, keys, or joins, and it does not call Kumo at this point — preflight is -catalog SQL against `INFORMATION_SCHEMA` plus explicitly targeted, read-only probes against +the user for column names, keys, or joins, and it builds no Kumo table or graph at this point — +preflight is a catalog read of the source plus explicitly targeted, read-only probes against source data, and it is read-only throughout. A probe exists only to produce evidence for the proposal; it is never written anywhere and never becomes a model input. +**What** preflight establishes is portable and written here. **How** it reads the source — which +tool, which catalog query — belongs to the active host, in the *Access* and *Preflight* sections +of `hosts//adapter.md`. The examples below are SQL; a host whose source has no SQL answers +the same questions with its own reader. + Preflight produces the **graph proposal** the user confirms at Node 1c. Every value carries a tag: | Tag | Meaning | @@ -21,6 +26,37 @@ If the harness or host has already run preflight and handed you a result (for ex --- +## 0. Access — can you read the source? + +Before §1, establish whether **you**, in this session, can read `source_scope` with a read-only +tool you already have. The host adapter's *Access* section says which tools count and the check +that proves each one reaches the data. + +- **You can read it** → run §1-§6 through that tool, and name the tool in the proposal's + `Source scope` line. +- **You cannot** — no suitable tool, or the tool reaches the system but not this scope (an + authorization error, an unknown schema or bucket) → stop before §1. In one message, tell the + user exactly what you checked and what happened, quoting any error, and offer both ways forward: + 1. **Access** — they make a read tool available, as the adapter describes, and tell you; you + then repeat the check. + 2. **Details** — they paste the structure: tables, columns with types, and any declared keys, + in whatever form they have (the adapter lists useful ones). You build the proposal from it. + + This is one ask-back. Do not also ask for anything §1-§8 would infer. +- **Never** ask for a credential's value or invite one into the chat, sign in, change a tool's + configuration, or set up a tool on the user's behalf. Never broaden the search beyond + `source_scope` because the scope itself was unreadable. + +A proposal built from the user's details follows §1-§8 unchanged, except that nothing that needs +row data can run: the §3 rule 5 probe is skipped, so its candidates stay `[inferred, unverified]`. +Say in the `Source scope` line that the structure came from the user, not from a read. + +This check concerns your session only. The generated artifact authenticates at run time through +its own connect step (the host's `runtime.md`), so your access — or its absence — changes +neither the generated code nor any gate. + +--- + ## 1. Resolve `source_scope` If `source_scope` is a semantic view (`DB.SCHEMA.VIEW`), read `accept/semantic_view.md` and @@ -32,7 +68,7 @@ SELECT TABLE_NAME, ROW_COUNT FROM .INFORMATION_SCHEMA.TABLES WHERE TABLE_SCH ``` - Not found, or zero tables → **ASK BACK**: the source cannot be resolved; ask for a valid - database and schema (or connector). Do not guess a similarly named schema. + location. Do not guess a similarly named schema, dataset, bucket or directory. - Found → enumerate tables **only inside this scope**. Never search other schemas, and never rank or curate tables by business relevance. @@ -47,7 +83,8 @@ ORDER BY TABLE_NAME, ORDINAL_POSITION; ## 3. Primary key per table In order of preference, first match wins: -1. A declared primary key (`SHOW PRIMARY KEYS IN SCHEMA .`). +1. A declared primary key, where the source declares any (the adapter's preflight shows how to + read them; files and object stores declare none). 2. `_ID`, where the stem is the table name without a `_DIM` / `_PROFILE` / `_EVENTS` suffix, singular or plural (`CUSTOMER_ID` in `CUSTOMERS`, `PAGE_VIEW_ID` in `PAGE_VIEW_EVENTS`). 3. A column named `ROW_ID`, `ID`, or `EVENT_ID`. @@ -122,7 +159,7 @@ exclusion the user can overturn at the gate, rather than asking the user for col A link is proposed when a column in table A has the **same name** as the primary key of table B (`FRAUD_EVENTS.CUSTOMER_ID → CUSTOMERS.CUSTOMER_ID`). Foreign-key constraints, if declared, are -taken as well (`SHOW IMPORTED KEYS IN SCHEMA ...`). Nothing else is inferred — no fuzzy matching, +taken as well, where the source declares any. Nothing else is inferred — no fuzzy matching, no value overlap checks, no joins executed. ## 7. Seeds and table mode @@ -146,7 +183,7 @@ no value overlap checks, no joins executed. Present exactly this shape, then **stop and wait** for confirmation: ``` -Source scope: USER_DATA.E_COMMERCE [supplied] — 7 tables found +Source scope: USER_DATA.E_COMMERCE [supplied] — 7 tables found, read via SQL Table mode: all-connected [default] — 7 of 7 tables reachable from the seeds Metadata @@ -183,6 +220,6 @@ format. result is reported and then discarded; it is never written anywhere and never reaches training. - Rank tables by relevance or drop tables the user did not ask to drop. - Proceed to PQL before the proposal is confirmed. -- Ask the user for columns, types, keys or joins. If preflight cannot run (no SQL access), say - that plainly and ask the user to confirm the proposal you can build from what they said — - still tagged, still confirmed, still no invented values. +- Ask the user for columns, types, keys or joins — with one exception: when §0 finds you cannot + read the source, offering to take its structure from the user is the §0 ask-back. Whatever the + proposal is built from, it is still tagged, still confirmed, and contains no invented values. diff --git a/agent-skills/skills/kumo-data-science/validate/examples/bad_request.md b/agent-skills/skills/kumo-data-science/validate/examples/bad_request.md index 9949af4cb..cfae6e7f5 100644 --- a/agent-skills/skills/kumo-data-science/validate/examples/bad_request.md +++ b/agent-skills/skills/kumo-data-science/validate/examples/bad_request.md @@ -16,7 +16,7 @@ Response to user: > "To proceed I need three things: > -> 1. **Where the data lives** — a Snowflake database and schema, or a connector name. +> 1. **Where the data lives** — for example a database and schema, or a folder of files. > 2. **What 'risky' means** — an observable outcome, e.g. 'a chargeback is filed' or 'no purchase'. > 3. **The prediction window** — e.g. 'in the next 30 days'." diff --git a/agent-skills/skills/kumo-model-explanations/SKILL.md b/agent-skills/skills/kumo-model-explanations/SKILL.md index 2745e3ece..35b9238fe 100644 --- a/agent-skills/skills/kumo-model-explanations/SKILL.md +++ b/agent-skills/skills/kumo-model-explanations/SKILL.md @@ -1,6 +1,6 @@ --- name: kumo-model-explanations -description: Inspect and interpret gradient attribution (including requests called radiant attribution), column analysis and sampled explanation subgraphs from a Kumo model trained in a Snowflake Python notebook. Locate the correct run's explanation artifacts, explain feature scores and cohorts, and identify missing or unsupported explanations without automatically retraining. +description: Inspect and interpret gradient attribution (including requests called radiant attribution), column analysis and sampled explanation subgraphs from a Kumo model trained in a Python notebook. Locate the correct run's explanation artifacts, explain feature scores and cohorts, and identify missing or unsupported explanations without automatically retraining. --- # Kumo model explanations diff --git a/agent-skills/skills/kumo-model-tuning/SKILL.md b/agent-skills/skills/kumo-model-tuning/SKILL.md index b6d637653..f60d4e5a5 100644 --- a/agent-skills/skills/kumo-model-tuning/SKILL.md +++ b/agent-skills/skills/kumo-model-tuning/SKILL.md @@ -1,6 +1,6 @@ --- name: kumo-model-tuning -description: Configure a Kumo model plan for a user-defined Snowflake task, including encoders, run mode, bounded experiments, sampling, optimization, architecture, metrics and refits. Preserve target and evaluation constraints and use existing diagnostics for comparisons. +description: Configure a Kumo model plan for a user-defined task, including encoders, run mode, bounded experiments, sampling, optimization, architecture, metrics and refits. Preserve target and evaluation constraints and use existing diagnostics for comparisons. --- # Configure a controlled experiment diff --git a/agent-skills/skills/kumo-training-diagnostics/SKILL.md b/agent-skills/skills/kumo-training-diagnostics/SKILL.md index 69b325bd7..be44dd0d2 100644 --- a/agent-skills/skills/kumo-training-diagnostics/SKILL.md +++ b/agent-skills/skills/kumo-training-diagnostics/SKILL.md @@ -1,6 +1,6 @@ --- name: kumo-training-diagnostics -description: Evaluate an existing Kumo training run in a Snowflake Python notebook using aggregate metrics, trial results, learning curves, refit evidence, and training progress. Use for model-quality questions or training diagnostics; inspect existing results without automatically tuning, rerunning, or deploying a model. +description: Evaluate an existing Kumo training run in a Python notebook using aggregate metrics, trial results, learning curves, refit evidence, and training progress. Use for model-quality questions or training diagnostics; inspect existing results without automatically tuning, rerunning, or deploying a model. --- # Kumo evaluation and training diagnostics diff --git a/agent-skills/skills/kumoai-reference/SKILL.md b/agent-skills/skills/kumoai-reference/SKILL.md index 30ef1caf3..5ef626ef6 100644 --- a/agent-skills/skills/kumoai-reference/SKILL.md +++ b/agent-skills/skills/kumoai-reference/SKILL.md @@ -45,9 +45,9 @@ after a rename. If a user's code contains `set_backend`, `LocalBackend`, `create `OutputConfig`, `kumo.Stype`, or edges written as dicts, it targets a **superseded** build: say so and rewrite it rather than copying its shape. See `kumoai-reference/references/kumo_finetuning_sdk.md` §11.1. -Connection, in this order: `SnowflakeConnector.from_spcs_env()` when `SPCS_TOKEN_PATH` exists, else -`.from_connection_toml()`, else a `SnowflakeIdentity` from Snowflake secrets attached to the -notebook (`kumoai-reference/references/kumo_finetuning_sdk.md` §3). None needs a key in code. +Connection is one step, written as the host's `runtime.md` says, from the connector facts in +`kumoai-reference/references/kumo_finetuning_sdk.md` §3 (Snowflake) and §3.4 (every other +connector). No credential literal belongs in code. ## Execution flow diff --git a/agent-skills/skills/kumoai-reference/references/graph_construction.md b/agent-skills/skills/kumoai-reference/references/graph_construction.md index f73971017..06a306357 100644 --- a/agent-skills/skills/kumoai-reference/references/graph_construction.md +++ b/agent-skills/skills/kumoai-reference/references/graph_construction.md @@ -49,11 +49,14 @@ keys: edges and PQL reference the **source table names** — `Edge(src_table='Ra --- -## Step 1: connect (Snowflake) +## Step 1: connect -**Setup and authentication are NOT documented here.** See `kumoai-reference/references/kumo_finetuning_sdk.md` §3. +**Setup and authentication are NOT documented here.** The host's `runtime.md` says how the +connector is built; the SDK facts are in `kumoai-reference/references/kumo_finetuning_sdk.md` §3 +(Snowflake) and §3.4 (`FileConnector`, `S3Connector`, `GCSConnector`, `BigQueryConnector`, +`DatabricksConnector`). There is no backend and no identity registry. -Build the connector directly from a factory — there is no backend and no identity registry: +For Snowflake, build the connector from a factory: ```python connector = kumo.SnowflakeConnector.from_spcs_env( @@ -70,8 +73,8 @@ connector reaches no tables. Take the Snowpark session once, before any connecto Never emit `kumo.init(...)` — hosted editions only, out of scope. Never construct `SnowflakeConnector` with `name=`, `account=` or `credentials=`; none is a parameter. -Only `SnowflakeConnector` is relevant here. `S3Connector`, `BigQueryConnector`, -`DatabricksConnector`, `GCSConnector` and `FileConnector` exist but are out of scope. +Everything after this step — tables, graph, query, training, prediction — is the same for every +connector. ### Accessing source tables @@ -142,8 +145,8 @@ Inspect resolved types via the table definition: it only exists because of an edge. Inference sees one table at a time, so it judges an FK column on its values alone and calls it `numerical`. It is a key; say so with `.with_column('', stype=Stype.ID)` for every FK column in every table. -2. **A time column must be a real date.** If the column is stored as `TEXT` in Snowflake the - catalog reports a string, and the table will not have a usable timeline. Declaring +2. **A time column must be a real date.** If the column is stored as text in the source (a + Snowflake `TEXT`, a CSV string) the catalog reports a string, and the table will not have a usable timeline. Declaring `.with_column('', dtype=Dtype.date)` is what makes staging cast it. Check the catalog dtype (section above) before assuming a `time_column=` argument is sufficient. @@ -349,7 +352,7 @@ import kumo.engine as kumo from kumo.engine import Edge from kumoapi.typing import Dtype, Stype -# `connector` is the SnowflakeConnector built per Step 1; the table names below are placeholders +# `connector` is the connector built per Step 1; the table names below are placeholders # for tables that exist behind it. account_profile = kumo.Table.from_source_table( diff --git a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md index d754e491f..fee4eafae 100644 --- a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md +++ b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md @@ -114,6 +114,10 @@ Source: shipped package (`kumo.engine.config`) ## 3. Connection +The connector is built in one step, as the host's `runtime.md` says; everything after it is the +same for every connector. §3-§3.3 cover `SnowflakeConnector`; §3.4 covers every other connector. +§3.2 (reading the catalog) applies to all of them. + `SnowflakeConnector` is built by a **factory**, which constructs the connector and its identity together from the environment. No password, key, or API credential literal belongs in a cell. Priority: the runtime token (`from_spcs_env`), then a `connections.toml` entry @@ -242,6 +246,55 @@ session = get_active_session() # first code cell, before any connector Source: shipped package (`kumo_snowflake.session`, which opens its session with `Session.builder.configs(...).create()`) +### 3.4 Every other connector + +Constructors and identities, all keyword-only. Source: shipped package +(`kumo.engine.connector.file`, `kumo.engine.connector.s3`, `kumo.engine.connector.gcs`, +`kumo.engine.connector.bigquery`, `kumo.engine.connector.databricks`, +`kumo.engine.connector.identity`): + +``` +FileConnector(*, root_dir) +S3Connector(*, root_dir=None, identity=None, region=None) +GCSConnector(*, root_dir=None, identity=None, requester_pays_project=None) +BigQueryConnector(*, project_id, dataset_id, identity=None, location='US', + billing_project_id=None, staging_uri=None, require_staged=False) +DatabricksConnector(*, catalog, schema_name, identity, warehouse_id=None, cluster_id=None, + staging_volume_path=None, require_staged=False) +DatabricksConnector.from_notebook(*, catalog, schema_name, warehouse_id=None, cluster_id=None, + staging_volume_path=None, require_staged=False) + +S3Identity(*, access_key_id=None, secret_access_key=None, session_token=None, role_arn=None, + partition='aws', connection=None) +GCSIdentity(*, service_account_key=None, workload_service_account_email=None) +BigQueryIdentity(*, service_account_key=None, workload_service_account_email=None) +DatabricksIdentity(*, host=None, client_id=None, client_secret=None) +``` + +**Prefer ambient credentials.** Each connector resolves its provider's own credential chain when +no secret is passed, so the connect step names no secret at all: + +| Connector | Ambient form | Credentials it resolves | Table names | `sql()` | +|---|---|---|---|---| +| `FileConnector` | `FileConnector(root_dir=...)` | none needed | relative paths below `root_dir`, optionally with `.csv` / `.parquet`, or a directory of parts in one format | no | +| `S3Connector` | `identity=None` | AWS chain: instance profile, `aws configure`, environment | relative paths below `root_dir`; with `root_dir=None`, full `s3://` URIs and no `table_names()` | no | +| `GCSConnector` | `identity=None` | Application Default Credentials | as S3, with `gs://` | no | +| `BigQueryConnector` | `identity=None` | Application Default Credentials | tables of `dataset_id` | yes | +| `DatabricksConnector` | `from_notebook(...)`, or `identity=DatabricksIdentity()` | the Databricks SDK's chain: a notebook's context, a CLI profile, environment | tables of `catalog.schema_name` | yes | + +- The explicit identity forms carry secrets (an access key, a service-account key dict, an OAuth + client secret). Use one only when the user asks for it, and read each secret from the + environment's secret store at run time — never a literal, never a value asked for in chat. +- `FileConnector.root_dir` is a filesystem directory, not a URI, and must not overlap the + scratch directory (§2.1). Absolute table paths, URIs, `..` and globs are rejected. A CSV time + column needs `.with_column(col, dtype=Dtype.time, timestamp_format=...)`. +- `region` on `S3Connector` is valid only with `root_dir`. +- `DatabricksConnector` takes exactly one of `warehouse_id` and `cluster_id`. +- `require_staged=True` needs `staging_uri` (BigQuery) or `staging_volume_path` (Databricks). +- ADLS has no connector in this package. + +Where each one writes is §9.2. + --- ## 4. Tables @@ -398,7 +451,8 @@ labeled rows and scores the unlabeled ones. If every row already has its target, `generate_prediction_table` raises `RuntimeError: prediction query produced no rows`. The scoring population confirmed at the task gate (`kumo-data-science/SKILL.md` Node 1d) decides -the path: +the path. The staging example below is Snowflake's; on any other connector the host's `runtime.md` +says how a population is staged, or that it is not covered there: - **Rows whose target is NULL** → `pquery.generate_prediction_table(snapshot)`, no `prediction_time`. @@ -514,7 +568,8 @@ PredictionResult.export(output_config) ``` - For Snowflake, always pass `artifact_type` and `table_name`. `output_paths` is for file and - object-store connectors only; with a `SnowflakeConnector` it raises `ValueError`. + object-store connectors only; with a `SnowflakeConnector` it raises `ValueError`. Every + connector's form is in §9.2. - **`snapshot=` is required.** `output_config` is optional — omit it to keep results local. - `predict` requires a trainer holding fitted state — from `fit` on that same `Trainer`, or from @@ -548,10 +603,28 @@ raises. Source: shipped package (`kumo.engine.trainer`) +### 9.2 Destinations by connector + +The output connector may differ from the source connector. Source: shipped package +(`kumo.engine.trainer.output_config`, `kumo.engine.connector.file`, `kumo_databricks.sink`): + +| Connector | `PredictionOutputConfig` form | Rules | +|---|---|---| +| `SnowflakeConnector` | `artifact_type=` + `table_name=` (bare name) | connector scoped to the destination database and schema; §9.1 | +| `FileConnector`, `S3Connector`, `GCSConnector` | `output_paths={'predictions': ..., 'embeddings': ...}` — the keys select the artifacts — or `artifact_type=` + `table_name=` | do not combine the two forms; a path ending in `/` is a directory and must be empty; otherwise one file, and `.parquet` is added when there is no suffix; paths are relative to a rooted connector, full URIs for a rootless one; `FileConnector` does not append | +| `BigQueryConnector` | `artifact_type=` + `table_name=` (unqualified table ID in its dataset) | predictions and embeddings only — not a model `export_config` and not a `Trainer.load_from_connector` source | +| `DatabricksConnector` | `artifact_type=` + `table_name='__kumo__
'` | the schema must be the connector's own; a bare table name fails; writes stage through the volume `.._kumo_staging`, created on first use | + +`output_paths` with a non-file connector raises `ValueError`. Read every destination back after +the write, as the host's `runtime.md` says (§9.1 for Snowflake). + --- ## 10. Complete verified example +On Snowflake. On any other connector, replace steps 1, 7 and 8 with the host `runtime.md`'s +connect step, the §9.2 destination and its read-back; nothing else changes. + Self-contained: `OUTPUT_DATABASE`, `OUTPUT_SCHEMA` and `OUTPUT_TABLE` are placeholders for the confirmed output destination (§12). @@ -700,11 +773,12 @@ targets a superseded build and needs rewriting against §10 — do not copy its | Code location | Source | |---|---| -| `TARGET = dict(warehouse=, database=, schema_name=)` | Data location from the intake contract; a lower-case name keeps its double quotes (§3) | -| `from_connection_toml(connection_name=)` | The user's `[connections.]` entry; ask | -| `SnowflakeIdentity(account=, username=)`, secret names | Account identifier, key-pair user, and the names of the attached key and passphrase secrets; ask only when neither factory applies, never for a secret's value | +| The connect step | The host's `runtime.md`, from the confirmed `source_scope` (§3, §3.4) | +| `TARGET = dict(warehouse=, database=, schema_name=)` (Snowflake) | Data location from the intake contract; a lower-case name keeps its double quotes (§3) | +| `from_connection_toml(connection_name=)` (Snowflake) | The user's `[connections.]` entry; ask | +| `SnowflakeIdentity(account=, username=)`, secret names (Snowflake) | Account identifier, key-pair user, and the names of the attached key and passphrase secrets; ask only when neither factory applies, never for a secret's value | | `configure(scratch_dir=)` | A local path; `/tmp/` is fine | -| `SnowflakeConnector(role=)` | Role from the intake contract | +| `SnowflakeConnector(role=)` (Snowflake) | Role from the intake contract | | `connector['TABLE']` names, column names | Exactly as `table_names()` / `.columns` return them (§3.2) | | Column selection, `stype` / `dtype` | Data structure from the intake contract | | `Edge(src_table=, fkey=, dst_table=)` | Relationships from the intake contract | @@ -713,7 +787,7 @@ targets a superseded build and needs rewriting against §10 — do not copy its | `num_classes_to_return=N` | Results per entity the user asked for, 1–500; defaults to `RANK TOP K` but may differ. Required for link prediction (§9) | | `binary_classification_threshold=` | Required for binary classification | | Population table (static task, rows that already have a target) | Confirmed at Node 1d (§7.1) | -| `PredictionOutputConfig(table_name=)` + output connector `database`/`schema_name` | Output destination; report back as `DB.SCHEMA.TABLE`, unchanged | +| `PredictionOutputConfig(...)` + output connector | Output destination in its connector's form (§9.2); report it back unchanged — `DB.SCHEMA.TABLE`, or the path | --- @@ -726,12 +800,14 @@ targets a superseded build and needs rewriting against §10 — do not copy its - [ ] `Stype` / `Dtype` imported from `kumoapi.typing`; `Column` / `Edge` from `kumo.engine`; `PredictionArtifactType` / `WriteMode` from `kumoapi.jobs`; `RunMode` from `kumoapi.model_plan`. -- [ ] Connection tries `from_spcs_env`, then `from_connection_toml`, then `SnowflakeIdentity` from - secrets (§3), with explicit `warehouse`, `database`, `schema_name`; `SPCS_TOKEN_PATH` - imported, never hardcoded. -- [ ] Lower-case or mixed-case `database` / `schema_name` passed with their double quotes (§3). +- [ ] The connect step is the host `runtime.md`'s, with no credential literal (§3, §3.4). +- [ ] Snowflake: connection tries `from_spcs_env`, then `from_connection_toml`, then + `SnowflakeIdentity` from secrets (§3), with explicit `warehouse`, `database`, + `schema_name`; `SPCS_TOKEN_PATH` imported, never hardcoded. +- [ ] Snowflake: lower-case or mixed-case `database` / `schema_name` passed with their double + quotes (§3). - [ ] Table and column names in the catalog's exact spelling (§3.2). -- [ ] `session = get_active_session()` once, in the first code cell; never again, no +- [ ] Snowflake: `session = get_active_session()` once, in the first code cell; never again, no `Session.builder` (§3.3). - [ ] `connector.table_names()` asserted before building tables. - [ ] Every foreign key corrected with `.with_column(col, stype=Stype.ID)`. @@ -747,8 +823,9 @@ targets a superseded build and needs rewriting against §10 — do not copy its - [ ] No code cell writes anywhere unless a destination was confirmed; an example of how to add one later is markdown text, not a code cell. - [ ] Smoke-test sizing called out as such. -- [ ] Output table reported with its exact name — no `_predictions` suffix. -- [ ] Written table read back and row count compared to `prediction_result.num_rows`. +- [ ] Destination in its connector's form (§9.2), reported with its exact name — no + `_predictions` suffix. +- [ ] Written destination read back and row count compared to `prediction_result.num_rows`. - [ ] Success judged by blocking calls returning without raising — not by a status value. --- @@ -770,8 +847,8 @@ targets a superseded build and needs rewriting against §10 — do not copy its - `PredictionResult.export` against Snowflake — in the package, exercised nowhere. - Whether a snapshot can be reused across graphs or persisted between processes. - The written column schema for a ranking task. -- Behaviour of `FileConnector`, `S3Connector`, `GCSConnector`, `BigQueryConnector`, - `DatabricksConnector`. Only Snowflake is covered here. +- `FileConnector`, `S3Connector`, `GCSConnector`, `BigQueryConnector`, `DatabricksConnector`: + §3.4 and §9.2 are read from the package source; none has been run end to end with this skill. **Known absent** — do not reach for these: `SourceTable.head(...)`, `graph.infer_links()`, `graph.get_edge_stats()`, `graph.get_table_stats()`, `table.validate()`, diff --git a/agent-skills/skills/kumoai-reference/references/outputs.md b/agent-skills/skills/kumoai-reference/references/outputs.md index ac8b690b9..2cca7a586 100644 --- a/agent-skills/skills/kumoai-reference/references/outputs.md +++ b/agent-skills/skills/kumoai-reference/references/outputs.md @@ -74,12 +74,12 @@ vectors; inspect TYPE and identifiers before calling them customer/product embed ## Serialization boundaries -The public prediction path exports selected Parquet through Snowflake's file sink, with schema -inference and COPY. This differs from the separate in-memory DataFrame writer, which rejects +On Snowflake, the public prediction path exports selected Parquet through Snowflake's file sink, +with schema inference and COPY. This differs from the separate in-memory DataFrame writer, which rejects list/dict columns. Do not apply that rejection indiscriminately to prediction export, and do not send nested cohort JSON through the scalar DataFrame path unchanged. -For vectors or other nested fields, inspect the destination's actual Snowflake type and a +For vectors or other nested fields, inspect the destination's actual column type and a bounded read-back value; inference is not a guarantee of the consumer's expected representation. Use [stage retention](../../kumo-model-persistence/references/retention.md) to preserve the original Parquet or JSON when a destination cannot represent the required shape. Conversion/flattening diff --git a/agent-skills/skills/kumoai-reference/references/task_journeys.md b/agent-skills/skills/kumoai-reference/references/task_journeys.md index 37d49530e..5d888d736 100644 --- a/agent-skills/skills/kumoai-reference/references/task_journeys.md +++ b/agent-skills/skills/kumoai-reference/references/task_journeys.md @@ -103,7 +103,7 @@ scores as if they established all horizons. Column analysis is not supported for ## Finish each journey -Use [outputs](outputs.md) for requested Snowflake write-back and a separate embedding call. +Use [outputs](outputs.md) for requested write-back and a separate embedding call. Inspect selected Parquet schemas before interpreting columns; do not hardcode optional fields. Use [diagnostics](../../kumo-training-diagnostics/SKILL.md) for actual metrics and trial histories, and [explanations](../../kumo-model-explanations/SKILL.md) for available attribution/cohorts. diff --git a/agent-skills/skills/kumoai-reference/references/troubleshooting_signals.md b/agent-skills/skills/kumoai-reference/references/troubleshooting_signals.md index 1d9fce6e6..b497ce5c9 100644 --- a/agent-skills/skills/kumoai-reference/references/troubleshooting_signals.md +++ b/agent-skills/skills/kumoai-reference/references/troubleshooting_signals.md @@ -37,7 +37,7 @@ Inspect a table through `.definition` (`.pkey`, `.time_col`, `.end_time_col`) an ### Symptom: `connector.table("X")` raises `ValueError` **Cause:** The table name does not exist in the connector. -**Fix:** List what is actually there before constructing tables: `connector.table_names()`, or guard with `connector.has_table(name)`. Snowflake object names are typically uppercase (`connector.table("ORDERS")`). +**Fix:** List what is actually there before constructing tables: `connector.table_names()`, or guard with `connector.has_table(name)`. Snowflake object names, for example, are typically uppercase (`connector.table("ORDERS")`). ### Symptom: a column's `stype` is unset, or the query fails because a column has no semantic type **Cause:** `Table.from_source_table()` and the explicit `Table(...)` constructor do **not** fully populate `dtype`/`stype`. A partially specified table is legal until it is used in a predictive query. diff --git a/agent-skills/tests/test_package_skill.py b/agent-skills/tests/test_package_skill.py index ef89482d1..0ccb12a81 100644 --- a/agent-skills/tests/test_package_skill.py +++ b/agent-skills/tests/test_package_skill.py @@ -54,3 +54,32 @@ def test_broken_references_are_reported(tmp_path: Path) -> None: 'SKILL.md: link (accept/missing.md) does not resolve', 'SKILL.md: `accept/nope.md` does not resolve from the package root', ] + + +def test_generic_ships_only_its_own_host(tmp_path: Path) -> None: + pkg = package_skill.build('generic', tmp_path) + assert pkg == tmp_path / 'generic' / 'skills' / 'kumo-data-science' + assert [p.name for p in (pkg / 'hosts').iterdir()] == ['generic'] + leaks = [str(p.relative_to(pkg)) for p in pkg.rglob('*.md') + if 'snowflake-coco' in p.read_text() or 'ml_job.md' in p.read_text()] + assert leaks == [] + + +def test_core_names_no_host() -> None: + """The portable core reaches its host as `hosts//`, never by a host's own name.""" + hosts = sorted(p.name for p in (package_skill.SKILLS / 'hosts').iterdir() if p.is_dir()) + named = re.compile(r'hosts/(?:' + '|'.join(map(re.escape, hosts)) + r')\b') + hits = [f'{p.relative_to(package_skill.SKILLS)}: {m.group(0)}' + for p in sorted(package_skill.SKILLS.rglob('*.md')) + if p.relative_to(package_skill.SKILLS).parts[0] != 'hosts' + for m in named.finditer(p.read_text())] + assert hits == [] + + +@pytest.mark.parametrize('host', sorted(package_skill.HOSTS)) +def test_adapter_covers_what_the_core_defers(host: str) -> None: + """Preflight §0, the intake contract and batch inference defer these to the adapter.""" + adapter = next(rel for rel in package_skill.HOSTS[host]['hosts'] if rel.endswith('adapter.md')) + text = (package_skill.SKILLS / 'hosts' / adapter).read_text() + for section in ['Source scope', 'Access', 'Destinations']: + assert section in text, f'{adapter} does not cover {section}' From 14a51ec228c722a91b53cecea56f4c065b1c7d76 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 16:48:43 -0600 Subject: [PATCH 02/20] Generic host: bare file table names; write path destinations exactly MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Found by the generic-host codegen eval (kumo-ai/kumo_skill_dev, eval/generic/): 32 of 38 files notebooks and 19 of 38 S3 notebooks named tables `x.parquet`, which carries the suffix into the graph and the PQL; 19 and 18 rooted the output connector at the confirmed directory and wrote `table_name='predictions'` one level below it. - adapter §3: name file and object-store tables without their suffix. - runtime §2: the same at the connect step. - runtime §3: root the output connector at the destination's parent and name the last segment in output_paths, so predictions land at exactly the confirmed path. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/adapter.md | 7 ++++-- agent-skills/skills/hosts/generic/runtime.md | 23 ++++++++++++++++++-- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/agent-skills/skills/hosts/generic/adapter.md b/agent-skills/skills/hosts/generic/adapter.md index c30b6f383..9ac292d28 100644 --- a/agent-skills/skills/hosts/generic/adapter.md +++ b/agent-skills/skills/hosts/generic/adapter.md @@ -96,8 +96,11 @@ column names exactly as that tool returns them; the artifact must use the same s (`kumoai-reference/references/kumo_finetuning_sdk.md` §3.2). Probe results are evidence for the proposal only: never written, never saved to a file, never fed to the artifact. -For files, a table is a file or a directory of parts in one format, named by its path relative to -the source directory — the name `FileConnector` uses. +For files and object stores, a table is a file or a directory of parts in one format, below the +source location. **Name it without its file suffix**: `accounts.parquet` is the table `accounts`. +The connector finds the file from the bare name, and the name you pass becomes the table's name +in the graph, the edges and the PQL, where a suffix's dot does not parse. Show the bare names in +the proposal. --- diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index 2f63ce2dd..70eca4e60 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -75,6 +75,9 @@ assert connector.table_names(), 'Connector reached no tables.' Emit only the line for the confirmed system. Never emit a rootless S3 or GCS connector for a source: it cannot list tables. +Name file and object-store tables without their suffix — `connector['accounts']` for +`accounts.parquet` — and use those names in the tables, edges and PQL (`adapter.md` §3). + **Credentials.** The ambient forms above name no secret. Use an explicit identity (§3.4) only when the user asks for one, and then read each secret at run time from an environment variable whose **name** is a config-cell constant — never a literal, never a value asked for in chat: @@ -103,6 +106,23 @@ that connector's form (`kumoai-reference/references/kumo_finetuning_sdk.md` §9. destination back and compare its row count to `prediction_result.num_rows` — the only proof the write worked. +**A path destination is written at exactly the confirmed path**, never inside it. Root the output +connector at the destination's **parent** and name the destination's last segment in +`output_paths`, keeping its trailing `/` for a directory: + +```python +# confirmed destination: s3://bucket/prediction_outputs/churn_scores/ +OUTPUT_PARENT, OUTPUT_LEAF = 's3://bucket/prediction_outputs/', 'churn_scores/' +output_connector = kumo.S3Connector(root_dir=OUTPUT_PARENT) # FileConnector / GCSConnector alike +output_config = kumo.PredictionOutputConfig( + connector=output_connector, output_paths={'predictions': OUTPUT_LEAF}) +OUTPUT_LOCATION = OUTPUT_PARENT + OUTPUT_LEAF # the confirmed path, for read-back +``` + +Rooting the connector at the destination itself and passing a `table_name` writes one level +below it, to a path the user never confirmed. A directory destination must be empty or absent; +the sink creates it. + ```python # Local files, S3, GCS — directory or single-file output import pyarrow.dataset as ds @@ -129,8 +149,7 @@ with snowflake.connector.connect(connection_name=SNOWFLAKE_CONNECTION) as conn: assert written == prediction_result.num_rows, (written, prediction_result.num_rows) ``` -- `OUTPUT_LOCATION` for files is the destination as written: a rooted connector's relative path - joined to its root. +- `OUTPUT_LOCATION` for files, S3 and GCS is the confirmed destination itself. - A Databricks destination is passed to Kumo as `'__kumo__
'` and read back as `..
`. With a cluster and no SQL warehouse, there is no read-back here: say so in the artifact and in your reply. From 8caea0e4ea92bd081871c911e467d1ad90688e2e Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 18:05:15 -0600 Subject: [PATCH 03/20] Generic host: stage a file population with TIMESTAMP, in a created directory MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit GPU runs of the generic files notebooks (kumo_skill_dev eval/generic/run_gpu.py) failed every static task with a staged population: pyarrow does not create the population directory, and kumo-engine 1.0.0 rejects a population without TIMESTAMP when the entity table is timestamped (kumo_finetuning_sdk.md §7.1 already says so; runtime §3.1 staged ENTITY1 only). Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 27 +++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index 70eca4e60..cacde9288 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -163,9 +163,30 @@ assert written == prediction_result.num_rows, (written, prediction_result.num_ro When Node 1d confirmed a population table (`kumoai-reference/references/kumo_finetuning_sdk.md` §7.1), stage it in the destination's system: -- **Files, S3, GCS:** read the entity key column from the source with `pyarrow.dataset`, rename it - `ENTITY1`, and write one Parquet file at the confirmed population path. Build the - `PredictionTable` from a connector rooted at that path's directory. +- **Files, S3, GCS:** read the entity key — and, when the entity table has a time column, that + column too — from the source with `pyarrow.dataset`; name them `ENTITY1` and `TIMESTAMP`; write + one Parquet file inside the confirmed population path, creating the directory first for local + files; and build the `PredictionTable` from a connector rooted there. Kumo rejects a + population without `TIMESTAMP` when the entity is timestamped, even for a static query (§7.1). + + ```python + import os + import pyarrow.dataset as ds + import pyarrow.parquet as papq + + keep = {ENTITY_KEY: 'ENTITY1', ENTITY_TIME: 'TIMESTAMP'} # drop ENTITY_TIME if there is none + population = ds.dataset(ENTITY_SOURCE, format='parquet').to_table(columns=list(keep)) + population = population.rename_columns([keep[c] for c in population.column_names]) + if '://' not in POPULATION_PATH: + os.makedirs(POPULATION_PATH, exist_ok=True) # object stores need no directory + papq.write_table(population, POPULATION_PATH + 'population.parquet') + population_connector = kumo.FileConnector(root_dir=POPULATION_PATH) # or S3Connector / GCSConnector + prediction_table = kumo.PredictionTable.from_source_table( + population_connector['population'], pq=pquery, snapshot=snapshot) + ``` + + `ENTITY_SOURCE` is the entity table's file or directory as preflight listed it; + `POPULATION_PATH` is the confirmed path, ending in `/`. - **Snowflake:** the §7.1 SQL, run through the read-back connection above. - **BigQuery, Databricks:** not covered on this host. Say so at Node 1d and offer the rows whose target is NULL instead. From 5a1508731c8e1411e13544aa995f8eb230810c6d Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 18:50:30 -0600 Subject: [PATCH 04/20] Generic host: a timestamped entity's population must carry TIMESTAMP The rerun's nov-sales-regression notebooks still staged ENTITY1 only: the example's 'drop if there is none' read as optional. Decide from the entity Table's time_column. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index cacde9288..a1c9f9214 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -166,15 +166,17 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin - **Files, S3, GCS:** read the entity key — and, when the entity table has a time column, that column too — from the source with `pyarrow.dataset`; name them `ENTITY1` and `TIMESTAMP`; write one Parquet file inside the confirmed population path, creating the directory first for local - files; and build the `PredictionTable` from a connector rooted there. Kumo rejects a - population without `TIMESTAMP` when the entity is timestamped, even for a static query (§7.1). + files; and build the `PredictionTable` from a connector rooted there. Decide from the entity's + `Table` as you built it: if it has a `time_column`, the population **must** carry it as + `TIMESTAMP`. Kumo rejects a population without it, even for a static query, whatever the + §7.1 Snowflake example shows (`ValueError: ... missing TIMESTAMP`). ```python import os import pyarrow.dataset as ds import pyarrow.parquet as papq - keep = {ENTITY_KEY: 'ENTITY1', ENTITY_TIME: 'TIMESTAMP'} # drop ENTITY_TIME if there is none + keep = {ENTITY_KEY: 'ENTITY1', ENTITY_TIME: 'TIMESTAMP'} # ENTITY_TIME: the entity Table's time_column population = ds.dataset(ENTITY_SOURCE, format='parquet').to_table(columns=list(keep)) population = population.rename_columns([keep[c] for c in population.column_names]) if '://' not in POPULATION_PATH: From 7ed8896d3e9b24f3fe6a0a37e12389b1da48e493 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 20:10:29 -0600 Subject: [PATCH 05/20] Generic host: stage a Databricks population through the statement API MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Databricks GPU runs of the generic notebooks staged the static population with a wait_timeout above Databricks' 50 s limit and without TIMESTAMP. §3.1 now covers Databricks: CREATE TABLE AS through the confirmed SQL warehouse, wait <= 50 s, poll to completion. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 27 +++++++++++++++++--- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index a1c9f9214..8acb1d244 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -133,7 +133,7 @@ from google.cloud import bigquery written = next(iter(bigquery.Client(project=PROJECT_ID).query( f'SELECT COUNT(*) FROM `{PROJECT_ID}.{OUTPUT_DATASET}.{OUTPUT_TABLE}`').result()))[0] -# Databricks — with the confirmed SQL warehouse +# Databricks — with the confirmed SQL warehouse; wait_timeout is at most '50s' from databricks.sdk import WorkspaceClient response = WorkspaceClient().statement_execution.execute_statement( warehouse_id=DATABRICKS_WAREHOUSE_ID, wait_timeout='30s', @@ -190,8 +190,29 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin `ENTITY_SOURCE` is the entity table's file or directory as preflight listed it; `POPULATION_PATH` is the confirmed path, ending in `/`. - **Snowflake:** the §7.1 SQL, run through the read-back connection above. -- **BigQuery, Databricks:** not covered on this host. Say so at Node 1d and offer the rows whose - target is NULL instead. +- **Databricks:** create the table through the confirmed SQL warehouse with the statement API, + wait for it, and build the `PredictionTable` from a `DatabricksConnector` scoped to the + population's schema. The same `TIMESTAMP` rule applies. `wait_timeout` is at most `'50s'` + (Databricks rejects more), so poll until the statement finishes: + + ```python + import time + from databricks.sdk import WorkspaceClient + from databricks.sdk.service.sql import StatementState + + ws = WorkspaceClient() + r = ws.statement_execution.execute_statement( + warehouse_id=DATABRICKS_WAREHOUSE_ID, wait_timeout='50s', + statement=(f'CREATE OR REPLACE TABLE {CATALOG}.{POP_SCHEMA}.{POP_TABLE} AS ' + f'SELECT {ENTITY_KEY} AS ENTITY1, {ENTITY_TIME} AS TIMESTAMP ' + f'FROM {CATALOG}.{SCHEMA}.{ENTITY_TABLE}')) + while r.status.state in (StatementState.PENDING, StatementState.RUNNING): + time.sleep(5) + r = ws.statement_execution.get_statement(r.statement_id) + assert r.status.state == StatementState.SUCCEEDED, r.status + ``` +- **BigQuery:** not covered on this host. Say so at Node 1d and offer the rows whose target is + NULL instead. --- From 76cd6f7f3325be25a4c226abc49b6fd1a209b3a5 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 21:53:42 -0600 Subject: [PATCH 06/20] Generic host: cast a population's TIMESTAMP to a timestamp GPU reruns staged TIMESTAMP from a DATE column (booked_date); kumo-engine 1.0.0 reads only a timestamp or int64 anchor and rejects date32. Cast it, in pyarrow and in the Databricks SQL. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index 8acb1d244..5df13f650 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -168,17 +168,24 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin one Parquet file inside the confirmed population path, creating the directory first for local files; and build the `PredictionTable` from a connector rooted there. Decide from the entity's `Table` as you built it: if it has a `time_column`, the population **must** carry it as - `TIMESTAMP`. Kumo rejects a population without it, even for a static query, whatever the - §7.1 Snowflake example shows (`ValueError: ... missing TIMESTAMP`). + `TIMESTAMP`, **cast to a timestamp**. Kumo rejects a population without it, even for a static + query, whatever the §7.1 Snowflake example shows (`ValueError: ... missing TIMESTAMP`), and + reads a `DATE` anchor as null (`Column 'TIMESTAMP' is date32[day]`): a date column is cast. ```python import os + import pyarrow as pa + import pyarrow.compute as pc import pyarrow.dataset as ds import pyarrow.parquet as papq keep = {ENTITY_KEY: 'ENTITY1', ENTITY_TIME: 'TIMESTAMP'} # ENTITY_TIME: the entity Table's time_column population = ds.dataset(ENTITY_SOURCE, format='parquet').to_table(columns=list(keep)) population = population.rename_columns([keep[c] for c in population.column_names]) + if 'TIMESTAMP' in population.column_names: # a DATE anchor reads as null + i = population.column_names.index('TIMESTAMP') + population = population.set_column(i, 'TIMESTAMP', + pc.cast(population['TIMESTAMP'], pa.timestamp('us'))) if '://' not in POPULATION_PATH: os.makedirs(POPULATION_PATH, exist_ok=True) # object stores need no directory papq.write_table(population, POPULATION_PATH + 'population.parquet') @@ -204,7 +211,7 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin r = ws.statement_execution.execute_statement( warehouse_id=DATABRICKS_WAREHOUSE_ID, wait_timeout='50s', statement=(f'CREATE OR REPLACE TABLE {CATALOG}.{POP_SCHEMA}.{POP_TABLE} AS ' - f'SELECT {ENTITY_KEY} AS ENTITY1, {ENTITY_TIME} AS TIMESTAMP ' + f'SELECT {ENTITY_KEY} AS ENTITY1, CAST({ENTITY_TIME} AS TIMESTAMP) AS TIMESTAMP ' f'FROM {CATALOG}.{SCHEMA}.{ENTITY_TABLE}')) while r.status.state in (StatementState.PENDING, StatementState.RUNNING): time.sleep(5) From feefa6da23a772322212eaa1444cdc8d4a3879f8 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Tue, 20 Oct 2026 23:36:37 -0600 Subject: [PATCH 07/20] Generic host: no region or profile placeholder for S3 and GCS SageMaker runs of the generic S3 notebooks: 5 of 38 added an AWS_REGION or AWS_PROFILE placeholder, which stops the run at the config guard; on a job role there is no profile to fill. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index 5df13f650..1e58df95d 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -31,7 +31,10 @@ The first code cell holds the imports and every location as a named constant: th source, the confirmed destination, and any population table — each written verbatim. Values only the runtime knows — a Snowflake warehouse or `connections.toml` entry, a Databricks -SQL warehouse or cluster — are constants here too. Write the user's value if they gave one; +SQL warehouse or cluster — are constants here too. S3 and GCS have none: the ambient chain +resolves credentials and region where the code runs (an instance or job role, the environment, +`aws configure`), so add no region, profile or project constant for them, and pass `region=` only +when the user named one. Write the user's value if they gave one; otherwise a placeholder of the form `''`, and list each placeholder in your reply as something to fill in before running. A placeholder never stands in for a confirmed value, and never for a secret. From ad5d7484bad17de77716a8102fa3232c37a1b1dd Mon Sep 17 00:00:00 2001 From: igutierrez Date: Wed, 21 Oct 2026 10:49:43 -0600 Subject: [PATCH 08/20] S3Connector: region is optional and normally omitted SageMaker runs: 3 of 38 generic S3 notebooks still invented an AWS_REGION placeholder after feefa6da, which stops them at the config guard. The SDK reference listed region= in the signature with no word that it is normally left out; say so there and at the connect step. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 2 +- .../skills/kumoai-reference/references/kumo_finetuning_sdk.md | 4 +++- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index 1e58df95d..adb0ce0d3 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -59,7 +59,7 @@ add arguments the user did not confirm. # Local files connector = kumo.FileConnector(root_dir=SOURCE_DIR) # S3 / GCS — rooted at the confirmed prefix, so table_names() works -connector = kumo.S3Connector(root_dir=SOURCE_URI) # 's3://bucket/prefix/' +connector = kumo.S3Connector(root_dir=SOURCE_URI) # 's3://bucket/prefix/'; no region= connector = kumo.GCSConnector(root_dir=SOURCE_URI) # 'gs://bucket/prefix/' # BigQuery connector = kumo.BigQueryConnector(project_id=PROJECT_ID, dataset_id=DATASET_ID) diff --git a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md index fee4eafae..579dcbeb1 100644 --- a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md +++ b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md @@ -288,7 +288,9 @@ no secret is passed, so the connect step names no secret at all: - `FileConnector.root_dir` is a filesystem directory, not a URI, and must not overlap the scratch directory (§2.1). Absolute table paths, URIs, `..` and globs are rejected. A CSV time column needs `.with_column(col, dtype=Dtype.time, timestamp_format=...)`. -- `region` on `S3Connector` is valid only with `root_dir`. +- `region` on `S3Connector` is optional and normally **omitted**: the ambient chain resolves the + bucket's region where the code runs. Pass it only when the user named one, and only with + `root_dir`. It is never a value to ask for, and never a placeholder. - `DatabricksConnector` takes exactly one of `warehouse_id` and `cluster_id`. - `require_staged=True` needs `staging_uri` (BigQuery) or `staging_volume_path` (Databricks). - ADLS has no connector in this package. From e6df3dbef35b6638ed50dcf6d5a2665e97bfe592 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Wed, 21 Oct 2026 11:21:03 -0600 Subject: [PATCH 09/20] Router: show the later-destination example as text, not a python block Codegen evals (CoCo and generic alike) lost 1-5 of 38 notebook-only cases to the 'add a destination later' example written as a python code block, which is a code cell that writes. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/kumo-data-science/SKILL.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/agent-skills/skills/kumo-data-science/SKILL.md b/agent-skills/skills/kumo-data-science/SKILL.md index 592c919e9..c2a221d58 100644 --- a/agent-skills/skills/kumo-data-science/SKILL.md +++ b/agent-skills/skills/kumo-data-science/SKILL.md @@ -266,7 +266,9 @@ After ACCEPT, load `kumoai-reference/SKILL.md` and hand off the confirmed payloa `PredictionOutputConfig` pointing at that destination, written **verbatim**. - no destination → predictions stay local; the notebook ends with a short markdown cell describing the one line to add a destination later. That example is markdown text, never a - code cell: a notebook with no confirmed destination contains no code that writes. + code cell: a notebook with no confirmed destination contains no code that writes. Inside + the markdown, show the code as text (a `text` fence), never as a `python` code block — + one that would run is a code cell that writes. The predict call is part of the artifact, not a separate job the agent manages. The agent still executes nothing (Step 3 is opt-in). From e35d691b34660d44768e9feaf1b9161481ddbfa2 Mon Sep 17 00:00:00 2001 From: igutierrez Date: Wed, 21 Oct 2026 16:42:46 -0600 Subject: [PATCH 10/20] SDK reference: threshold and top-k belong to predict, never to PredictionOutputConfig Generic Databricks notebooks put binary_classification_threshold on PredictionOutputConfig in 6 of 42 cases (S3 2 of 112; files and CoCo 0), a TypeError at predict time. The reference gave the signature but never said the config takes neither argument. Co-Authored-By: Claude Opus 5.5 --- .../kumoai-reference/references/kumo_finetuning_sdk.md | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md index 579dcbeb1..c5731c72d 100644 --- a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md +++ b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md @@ -583,6 +583,10 @@ PredictionResult.export(output_config) embeddings instead. Multiclass and multilabel accept it optionally and fall back to the query's `RANK TOP K` when it has one. Regression and binary classification reject it. - Binary classification requires an explicit `binary_classification_threshold`. +- `binary_classification_threshold` and `num_classes_to_return` are arguments of `predict`, and + only of `predict`. `PredictionOutputConfig` describes where to write, not how to predict: it + takes neither, and passing one is a `TypeError`. Whatever the destination, the config carries + only the fields in its signature above. - `PredictionResult` exposes `.num_rows`, `.files`, `.frame`, `.selection`, `.artifact_type`. Prefer `.num_rows` and `.files` — `.frame` collects every row into pandas. - `prediction_result.export(output_config)` writes an existing result to a destination. Passing @@ -786,8 +790,8 @@ targets a superseded build and needs rewriting against §10 — do not copy its | `Edge(src_table=, fkey=, dst_table=)` | Relationships from the intake contract | | PQL `query=` | Generated per `kumoai-reference/references/pql_writing.md` | | `prediction_time=` | A known in-range **naive** `datetime`; **cannot be computed** | -| `num_classes_to_return=N` | Results per entity the user asked for, 1–500; defaults to `RANK TOP K` but may differ. Required for link prediction (§9) | -| `binary_classification_threshold=` | Required for binary classification | +| `predict(num_classes_to_return=N)` | Results per entity the user asked for, 1–500; defaults to `RANK TOP K` but may differ. Required for link prediction (§9) | +| `predict(binary_classification_threshold=)` | Required for binary classification; never on `PredictionOutputConfig` | | Population table (static task, rows that already have a target) | Confirmed at Node 1d (§7.1) | | `PredictionOutputConfig(...)` + output connector | Output destination in its connector's form (§9.2); report it back unchanged — `DB.SCHEMA.TABLE`, or the path | @@ -819,6 +823,8 @@ targets a superseded build and needs rewriting against §10 — do not copy its - [ ] `graph.snapshot()` before generating any table. - [ ] **`trainer.fit(training_table, snapshot=snapshot)`** — `snapshot=` present. - [ ] **`trainer.predict(prediction_table, snapshot=snapshot, ...)`** — `snapshot=` present. +- [ ] `binary_classification_threshold` / `num_classes_to_return` on `predict`, never on + `PredictionOutputConfig` (§9). - [ ] `prediction_time` hardcoded and **naive** — no `tzinfo`, no `timezone.utc` — with the reason stated. - [ ] Static query: generated population only for rows whose target is NULL; otherwise the population table confirmed at Node 1d (§7.1). From b5d5245a5056c4e05dee340cb405bb065ed00cbb Mon Sep 17 00:00:00 2001 From: igutierrez Date: Fri, 23 Oct 2026 16:27:48 -0600 Subject: [PATCH 11/20] Address review on the generic host - Access: an S3/GCS listing proves access, not structure; keep looking for a column reader, else ask for the structure. - Path destinations end in '/', '.parquet' or '.csv'. File, S3 and GCS sinks all add '.parquet' to any other path (object_store_sink, connector.file), so the SDK reference's "FileConnector does not append" was wrong. Read back in the format the suffix names. - Population staging reads the entity source in its own format (CSV or Parquet) and still writes Parquet. - Binary threshold: 0.5 [default] at Node 1d unless the user gives one; predict requires it and has no default. - batch_inference: a connector named alone is a source only on a host that accepts connector names. - CoCo: a semantic view's connector scope is its base tables' schema. - graph_construction: connector-specific naming and sql(); non-ISO time strings take timestamp_format with Dtype.date (Dtype.time is time of day). Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/adapter.md | 12 +++++++++--- agent-skills/skills/hosts/generic/runtime.md | 14 ++++++++++---- .../skills/hosts/snowflake-coco/adapter.md | 3 ++- agent-skills/skills/kumo-data-science/SKILL.md | 1 + .../kumo-data-science/accept/batch_inference.md | 3 ++- .../kumo-data-science/accept/intake_contract.md | 7 ++++--- .../references/graph_construction.md | 13 +++++++++---- .../references/kumo_finetuning_sdk.md | 10 ++++++---- 8 files changed, 43 insertions(+), 20 deletions(-) diff --git a/agent-skills/skills/hosts/generic/adapter.md b/agent-skills/skills/hosts/generic/adapter.md index 9ac292d28..256bf2df8 100644 --- a/agent-skills/skills/hosts/generic/adapter.md +++ b/agent-skills/skills/hosts/generic/adapter.md @@ -47,7 +47,9 @@ ADLS has no Kumo connector. Data there is outside this host's reach: say so. ## 2. Access Implements `kumo-data-science/accept/preflight.md` §0. Check, in this order, and stop at the -first that reads `source_scope`: +first that reads the **columns and types** inside `source_scope`. A listing of object names +(`aws s3 ls`, `gcloud storage ls`) proves access, not structure: keep going down the list for a +reader, and if none is found, send the ask-back below with **Details** as the way forward. 1. **A tool in this session** that reads the system — a server or connector tool for that warehouse or storage. Use only its read operations. @@ -62,7 +64,7 @@ first that reads `source_scope`: | Source | Tool | Check, inside `source_scope` | Columns and types | Declared keys | §3 probe | |---|---|---|---|---|---| | Local files | your file tools; `duckdb`, or Python with `pyarrow` | list the directory | CSV header line; Parquet schema (`duckdb -c "DESCRIBE SELECT * FROM ''"`) | none | `duckdb`, or `pyarrow` | -| S3 | `aws` | `aws s3 ls s3:////` | file names only; for columns, a reader of `s3://` with the same credentials (`duckdb`), else §2 details | none | the same reader, else skip | +| S3 | `aws` | `aws s3 ls s3:////` | file names only — not enough on its own; for columns, a reader of `s3://` with the same credentials (`duckdb`, or check 3), else the ask-back | none | the same reader, else skip | | GCS | `gcloud` | `gcloud storage ls gs:////` | as S3, with `gs://` | none | as S3 | | BigQuery | `bq` | `bq ls :` | `bq show --schema --format=prettyjson :.
` | `INFORMATION_SCHEMA.TABLE_CONSTRAINTS` and `KEY_COLUMN_USAGE` of the dataset | `bq query --nouse_legacy_sql` | | Databricks | `databricks` | `databricks tables list ` | `databricks tables get ..
` | the same output's table constraints | only through a SQL warehouse the user named; else skip | @@ -77,7 +79,7 @@ profile or account, or edit a tool's configuration; never add or update software reader. Tool permission prompts are the user's to answer; a refused prompt means that path is unavailable. -**When nothing reads the source**, send the preflight §0 ask-back. State what you checked and +**When nothing reads the source's columns**, send the preflight §0 ask-back. State what you checked and what happened — "no BigQuery tool in this session; `bq ls my-proj:sales` returned `Access Denied`" — then offer: @@ -112,6 +114,10 @@ Databricks, or a path in a local directory, S3 or GCS. The destination may be in system from the source; it gets its own connector. Not destinations here: ADLS, and BigQuery for anything but predictions and embeddings. +A path destination ends in `/` (a directory) or in `.parquet` / `.csv` (one file). The sink adds +`.parquet` to any other path, so the file would not be where the user said. When the user gives +a path with neither, restate it with a trailing `/` at Node 1d and let them confirm or correct it. + --- ## 5. Runtime column constraints diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index adb0ce0d3..f56bda0e6 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -126,10 +126,15 @@ Rooting the connector at the destination itself and passing a `table_name` write below it, to a path the user never confirmed. A directory destination must be empty or absent; the sink creates it. +`OUTPUT_LEAF` ends in `/`, `.parquet` or `.csv` (`adapter.md` §4). File, S3 and GCS sinks alike +add `.parquet` to any other leaf, and the read-back would then miss the file. The suffix also +sets the format the sink writes, so read back in that format. + ```python # Local files, S3, GCS — directory or single-file output import pyarrow.dataset as ds -written = ds.dataset(OUTPUT_LOCATION, format='parquet').count_rows() # full path or URI +OUTPUT_FORMAT = 'csv' if OUTPUT_LOCATION.rstrip('/').lower().endswith('.csv') else 'parquet' +written = ds.dataset(OUTPUT_LOCATION, format=OUTPUT_FORMAT).count_rows() # full path or URI # BigQuery from google.cloud import bigquery @@ -183,7 +188,7 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin import pyarrow.parquet as papq keep = {ENTITY_KEY: 'ENTITY1', ENTITY_TIME: 'TIMESTAMP'} # ENTITY_TIME: the entity Table's time_column - population = ds.dataset(ENTITY_SOURCE, format='parquet').to_table(columns=list(keep)) + population = ds.dataset(ENTITY_SOURCE, format=ENTITY_FORMAT).to_table(columns=list(keep)) population = population.rename_columns([keep[c] for c in population.column_names]) if 'TIMESTAMP' in population.column_names: # a DATE anchor reads as null i = population.column_names.index('TIMESTAMP') @@ -197,8 +202,9 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin population_connector['population'], pq=pquery, snapshot=snapshot) ``` - `ENTITY_SOURCE` is the entity table's file or directory as preflight listed it; - `POPULATION_PATH` is the confirmed path, ending in `/`. + `ENTITY_SOURCE` is the entity table's file or directory as preflight listed it, and + `ENTITY_FORMAT` its format as preflight found it, `'csv'` or `'parquet'`. The population is + written as Parquet either way. `POPULATION_PATH` is the confirmed path, ending in `/`. - **Snowflake:** the §7.1 SQL, run through the read-back connection above. - **Databricks:** create the table through the confirmed SQL warehouse with the statement API, wait for it, and build the `PredictionTable` from a `DatabricksConnector` scoped to the diff --git a/agent-skills/skills/hosts/snowflake-coco/adapter.md b/agent-skills/skills/hosts/snowflake-coco/adapter.md index 2be48c5dc..e9c4349cb 100644 --- a/agent-skills/skills/hosts/snowflake-coco/adapter.md +++ b/agent-skills/skills/hosts/snowflake-coco/adapter.md @@ -94,7 +94,8 @@ Database identifiers written with a hyphen in chat (`synthetic-datasets`) are th their underscore form (`SYNTHETIC_DATASETS`) — resolve, do not ask. The `source_scope` the user confirmed at intake is also the `database` / `schema_name` passed to -the connector in `runtime.md` §3. Snowflake's `INFORMATION_SCHEMA` needs only `USAGE` on the database and +the connector in `runtime.md` §3. For a semantic view, pass the database and schema its base +tables share (`kumo-data-science/accept/semantic_view.md`), never the view's own `DB.SCHEMA.VIEW`. Snowflake's `INFORMATION_SCHEMA` needs only `USAGE` on the database and schema plus `SELECT` (or `REFERENCES`) on the tables to be listed. diff --git a/agent-skills/skills/kumo-data-science/SKILL.md b/agent-skills/skills/kumo-data-science/SKILL.md index c2a221d58..b40b8629c 100644 --- a/agent-skills/skills/kumo-data-science/SKILL.md +++ b/agent-skills/skills/kumo-data-science/SKILL.md @@ -212,6 +212,7 @@ generated before the task is confirmed: - Task type: - Output: predictions written to in the same notebook | notebook only, no predictions written - Scoring population (static tasks): rows whose is NULL | all rows, staged at +- Threshold (binary classification, predictions written): - Embeddings: ``` diff --git a/agent-skills/skills/kumo-data-science/accept/batch_inference.md b/agent-skills/skills/kumo-data-science/accept/batch_inference.md index 215162615..67993eb28 100644 --- a/agent-skills/skills/kumo-data-science/accept/batch_inference.md +++ b/agent-skills/skills/kumo-data-science/accept/batch_inference.md @@ -42,7 +42,7 @@ preds in the scores table", after a turn that named the model and the table, sup | Field | How it is obtained | Shown as | |---|---|---| | **task** — entity, outcome, window, task type | Reused from the model unchanged. Never reopened. | restated at 1d | -| **`source_scope`** | In order: a source the user named → the connector the user named for this request (one connector reads the source and writes the scores unless the user separates them) → the source scope recorded with the model at publication. Ask only if none of the three exists. | `[supplied]` / `[inferred]` at 1c | +| **`source_scope`** | In order: a source the user named → the connector the user named for this request, on a host whose *Source scope* accepts a connector name (one connector reads the source and writes the scores unless the user separates them) → the source scope recorded with the model at publication. Ask only if none of the three exists. On a host that does not accept a connector name, a connector named alone is not a source: ask for the system and location as that host's adapter says. | `[supplied]` / `[inferred]` at 1c | | **graph** | Rebuilt from the model's recorded graph definition inside `source_scope`, then confirmed | `[inferred]` at 1c | | **`scoring_population`** | Default: every entity of the trained entity table, anchored at the latest timestamp in the rebuilt snapshot ("current predictions"). A static task has no anchor. If the user names a subset or an anchor date, use theirs. | `[default]` / `[supplied]` at 1d | | **model version** | The version the user named. If the address resolves to more than one version and none was named, ask which one. Never pick the latest yourself. | `[supplied]` at 1d | @@ -102,6 +102,7 @@ For `batch_inference`, restate in this shape and stop for confirmation: - Source: [supplied | inferred] - Population: as of latest snapshot timestamp [default] | the user's subset/date [supplied]> - Output: predictions written to +- Threshold (binary classification): - Embeddings: ``` diff --git a/agent-skills/skills/kumo-data-science/accept/intake_contract.md b/agent-skills/skills/kumo-data-science/accept/intake_contract.md index f18406721..a2ffe2f7e 100644 --- a/agent-skills/skills/kumo-data-science/accept/intake_contract.md +++ b/agent-skills/skills/kumo-data-science/accept/intake_contract.md @@ -99,14 +99,15 @@ confirmed **before** any PQL is written. | **`output_location`** | none — training yields a notebook; when given, the same notebook also predicts into it (SKILL.md Step 2) | | **artifact format** | one notebook (user may ask for a script) | | **embeddings** | not included; add only when the user explicitly asks (see below) | +| **binary classification threshold** | `0.5`, shown `[default]` at Node 1d whenever a binary classification writes predictions. `predict` requires one and has no default of its own (`kumoai-reference/references/kumo_finetuning_sdk.md` §9) | | **metrics** | by task type, see `SKILL.md` | If the user supplies an `output_location`, carry it verbatim into the task interpretation gate and the generated artifact — the notebook's predict call writes there. If none is supplied, predictions stay local and the notebook shows how to add a destination. The destination must be -one the host can write (`hosts//adapter.md`, *Destinations*). **It is written with exactly -the name passed** — no suffix, no case change (the host's `runtime.md`, -`kumoai-reference/references/kumo_finetuning_sdk.md` §9.1). Older SDK builds appended +one the host can write (`hosts//adapter.md`, *Destinations*). **A table is written with +exactly the name passed** — no suffix, no case change (the host's `runtime.md`, +`kumoai-reference/references/kumo_finetuning_sdk.md` §9.1). A file path follows §9.2. Older SDK builds appended `_predictions`; the current one does not. --- diff --git a/agent-skills/skills/kumoai-reference/references/graph_construction.md b/agent-skills/skills/kumoai-reference/references/graph_construction.md index 06a306357..8daf017b9 100644 --- a/agent-skills/skills/kumoai-reference/references/graph_construction.md +++ b/agent-skills/skills/kumoai-reference/references/graph_construction.md @@ -73,8 +73,10 @@ connector reaches no tables. Take the Snowpark session once, before any connecto Never emit `kumo.init(...)` — hosted editions only, out of scope. Never construct `SnowflakeConnector` with `name=`, `account=` or `credentials=`; none is a parameter. -Everything after this step — tables, graph, query, training, prediction — is the same for every -connector. +Everything after this step — tables, graph, query, training, prediction — follows the same +workflow for every connector. Two things still differ by connector: how a table is named +(an identifier, or a path below a file connector's root without its suffix) and `connector.sql()`, +which only Snowflake, Databricks and BigQuery run (`kumo_finetuning_sdk.md` §3.4). ### Accessing source tables @@ -147,8 +149,11 @@ Inspect resolved types via the table definition: `.with_column('', stype=Stype.ID)` for every FK column in every table. 2. **A time column must be a real date.** If the column is stored as text in the source (a Snowflake `TEXT`, a CSV string) the catalog reports a string, and the table will not have a usable timeline. Declaring - `.with_column('', dtype=Dtype.date)` is what makes staging cast it. Check the - catalog dtype (section above) before assuming a `time_column=` argument is sufficient. + `.with_column('', dtype=Dtype.date)` is what makes staging cast it. When the strings + are not ISO-8601 (`03/15/2024`), also pass their format: + `.with_column('', dtype=Dtype.date, timestamp_format='%m/%d/%Y')`. `Dtype.date` is + Kumo's date-and-time type; `Dtype.time` is a time of day and is wrong here. Check the catalog + dtype (section above) before assuming a `time_column=` argument is sufficient. --- diff --git a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md index c5731c72d..78800ccd2 100644 --- a/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md +++ b/agent-skills/skills/kumoai-reference/references/kumo_finetuning_sdk.md @@ -582,7 +582,8 @@ PredictionResult.export(output_config) (`kumo_core.prediction_options.MAX_LINKS_TO_PREDICT`); for more than 500 per entity, export embeddings instead. Multiclass and multilabel accept it optionally and fall back to the query's `RANK TOP K` when it has one. Regression and binary classification reject it. -- Binary classification requires an explicit `binary_classification_threshold`. +- Binary classification requires an explicit `binary_classification_threshold`; there is no SDK + default. Use the one confirmed at Node 1d (§12). - `binary_classification_threshold` and `num_classes_to_return` are arguments of `predict`, and only of `predict`. `PredictionOutputConfig` describes where to write, not how to predict: it takes neither, and passing one is a `TypeError`. Whatever the destination, the config carries @@ -612,12 +613,13 @@ Source: shipped package (`kumo.engine.trainer`) ### 9.2 Destinations by connector The output connector may differ from the source connector. Source: shipped package -(`kumo.engine.trainer.output_config`, `kumo.engine.connector.file`, `kumo_databricks.sink`): +(`kumo.engine.trainer.output_config`, `kumo.engine.connector.file`, `kumo_core.object_store_sink`, +`kumo_databricks.sink`): | Connector | `PredictionOutputConfig` form | Rules | |---|---|---| | `SnowflakeConnector` | `artifact_type=` + `table_name=` (bare name) | connector scoped to the destination database and schema; §9.1 | -| `FileConnector`, `S3Connector`, `GCSConnector` | `output_paths={'predictions': ..., 'embeddings': ...}` — the keys select the artifacts — or `artifact_type=` + `table_name=` | do not combine the two forms; a path ending in `/` is a directory and must be empty; otherwise one file, and `.parquet` is added when there is no suffix; paths are relative to a rooted connector, full URIs for a rootless one; `FileConnector` does not append | +| `FileConnector`, `S3Connector`, `GCSConnector` | `output_paths={'predictions': ..., 'embeddings': ...}` — the keys select the artifacts — or `artifact_type=` + `table_name=` | do not combine the two forms; a path ending in `/` is a directory and must be empty; otherwise one file, written in the format its `.parquet` or `.csv` suffix names, and `.parquet` is added when it has neither — by all three connectors; paths are relative to a rooted connector, full URIs for a rootless one | | `BigQueryConnector` | `artifact_type=` + `table_name=` (unqualified table ID in its dataset) | predictions and embeddings only — not a model `export_config` and not a `Trainer.load_from_connector` source | | `DatabricksConnector` | `artifact_type=` + `table_name='__kumo__
'` | the schema must be the connector's own; a bare table name fails; writes stage through the volume `.._kumo_staging`, created on first use | @@ -791,7 +793,7 @@ targets a superseded build and needs rewriting against §10 — do not copy its | PQL `query=` | Generated per `kumoai-reference/references/pql_writing.md` | | `prediction_time=` | A known in-range **naive** `datetime`; **cannot be computed** | | `predict(num_classes_to_return=N)` | Results per entity the user asked for, 1–500; defaults to `RANK TOP K` but may differ. Required for link prediction (§9) | -| `predict(binary_classification_threshold=)` | Required for binary classification; never on `PredictionOutputConfig` | +| `predict(binary_classification_threshold=)` | Required for binary classification; never on `PredictionOutputConfig`. The threshold restated at Node 1d: the user's, else `0.5 [default]` (`kumo-data-science/accept/intake_contract.md`) | | Population table (static task, rows that already have a target) | Confirmed at Node 1d (§7.1) | | `PredictionOutputConfig(...)` + output connector | Output destination in its connector's form (§9.2); report it back unchanged — `DB.SCHEMA.TABLE`, or the path | From f1638449e984b37855dce247bbeafe177ea9904a Mon Sep 17 00:00:00 2001 From: igutierrez Date: Fri, 23 Oct 2026 20:32:44 -0600 Subject: [PATCH 12/20] agent-skills: address remaining quick review comments MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - generic runtime §3.1: stage the population's TIMESTAMP only when the entity table has a time column, in both the pyarrow and Databricks paths. - batch_inference: the field summary applies the same host condition on connector names as the table below it. Co-Authored-By: Claude Opus 5.5 --- agent-skills/skills/hosts/generic/runtime.md | 12 ++++++++---- .../kumo-data-science/accept/batch_inference.md | 5 +++-- 2 files changed, 11 insertions(+), 6 deletions(-) diff --git a/agent-skills/skills/hosts/generic/runtime.md b/agent-skills/skills/hosts/generic/runtime.md index f56bda0e6..009967da3 100644 --- a/agent-skills/skills/hosts/generic/runtime.md +++ b/agent-skills/skills/hosts/generic/runtime.md @@ -187,7 +187,9 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin import pyarrow.dataset as ds import pyarrow.parquet as papq - keep = {ENTITY_KEY: 'ENTITY1', ENTITY_TIME: 'TIMESTAMP'} # ENTITY_TIME: the entity Table's time_column + keep = {ENTITY_KEY: 'ENTITY1'} + if ENTITY_TIME: # the entity Table's time_column, else None + keep[ENTITY_TIME] = 'TIMESTAMP' population = ds.dataset(ENTITY_SOURCE, format=ENTITY_FORMAT).to_table(columns=list(keep)) population = population.rename_columns([keep[c] for c in population.column_names]) if 'TIMESTAMP' in population.column_names: # a DATE anchor reads as null @@ -208,8 +210,9 @@ When Node 1d confirmed a population table (`kumoai-reference/references/kumo_fin - **Snowflake:** the §7.1 SQL, run through the read-back connection above. - **Databricks:** create the table through the confirmed SQL warehouse with the statement API, wait for it, and build the `PredictionTable` from a `DatabricksConnector` scoped to the - population's schema. The same `TIMESTAMP` rule applies. `wait_timeout` is at most `'50s'` - (Databricks rejects more), so poll until the statement finishes: + population's schema. The same `TIMESTAMP` rule applies: project `CAST(