Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,29 @@
# Changelog

## [v0.29.0] - 2026-09-27

### Highlights

**A caught error reaches the caller with its reason and its place.** A run of an invalid bundle is refused before any pipe runs with the same located validation items `/validate` gives, a failed run names the pipe that failed and its root fault, and a run the caller's own method refuses keeps its explanation under STRICT disclosure.

### Added

- **Validation items carry more locators**: a TOML syntax error's item carries the 1-based `line` and `column` the parser stopped at, an `unresolved_concept` item carries `declared_concepts`, and the new `unknown_model` item carries `model_reference` (the reference as the bundle wrote it), `model_type` and `suggestions` (the model deck's close matches of that kind), with an `unsafe` rename fix when there is exactly one. The OpenAPI artifact publishes the new fields and the new `unknown_model` value of `PipeValidationErrorType`.
- **A run graph marks a list-valued stuff**: every io item of a `graph_spec` carries `multiplicity`, `true` when the stuff is a list, a fixed-count one included, and `null` otherwise, so a renderer can show a `Document[]` input or a `Record[]` output as a list. The schema also admits a positive integer, which the runtime does not emit.

### Changed

- **Pinned `pipelex` 0.67.0 (Breaking)**: up from `==0.66.1`, exactly, the release that carries the located error reporting the entries below describe. The `.pipelex/` config shipped here already sits at the current schema, so no migration is required. The next step of an invalid-bundle verdict, its `user_action.detail`, now reads "Edit the bundle as each validation error says: apply its suggested fix where it has one, after confirming an unsafe one".
- **The run routes refuse an invalid bundle with its validation verdict (Breaking)**: `POST /v1/execute` and `POST /v1/start` answer every refusal of the bundle while it loads, before any pipe runs, with the `422` `ValidateBundleError` problem document carrying the same located `validation_errors` `POST /v1/validate` gives, which STRICT disclosure keeps. A misspelled concept or a wiring mismatch used to answer a `500`, a TOML fault a `422` without items, and an unknown model the raw model-choice error. An entry pipe the bundle does not declare still answers its own `PipeNotFoundError`.
- **A failed run reports its root fault, located at the failing pipe (Breaking)**: a run failure's problem document carries the `error_type`, `title`, `type`, `error_domain` and status of the innermost Pipelex error rather than the run-level wrapper's, and its `detail` opens with `Pipe '<pipe>' failed (<entry pipe> → … → <pipe>):`, so a consumer branching on `error_type == "PipelineExecutionError"` branches on the root fault's type. A run the caller's own method refuses, such as a `PipeCondition` whose outcome is `fail`, a `PipeParallel` branch whose multiplicity does not fit its field, a step missing a required input or an inline model the deck does not define, answers a `422` with a `detail` STRICT disclosure keeps, where it answered `500` with `An internal error occurred.`. The completion webhook's `error` carries the same report.
- **Every load-time refusal is an item of the validation verdict (Breaking)**: `POST /v1/validate`, `POST /v1/resolve`, `POST /v1/codegen` and `POST /v1/build/*` answer any refusal of the submitted bundle, an unknown model and a pipe factory's refusal included, as a `200` `is_valid: false` verdict with a located item, where some of them escaped as a no-verdict problem document.
- **A failing dry run is one located `dry_run` item per failing pipe (Breaking)**: an invalid verdict's `validation_errors` carries one `dry_run` item per pipe whose dry run failed, with the `pipe_code`, `domain_code` and `source` of the innermost pipe that failed, where it carried a single message-only item for the whole sweep. Its message is the failure's own when that is caller-facing and its title otherwise, so a configuration fault met during a dry run no longer reaches the caller through the verdict.
- **`POST /v1/codegen` stamps `engine_version` `0.67.0`**: the stamp is the pinned `pipelex` version, so a `codegen.lock` committed against `0.66.1` no longer matches until it is regenerated. `POST /v1/build/runner` carries the same stamp.

### Fixed

- **A verdict names no path on the server**: an item located inside a package a method depends on by address names the bundle by the package's address and its path inside it, and an item located in one of the server's own library directories carries no `source` or `field_path` naming its file, so STRICT disclosure no longer hands a caller a path on the host.

## [v0.28.1] - 2026-09-27

### Changed
Expand Down
9 changes: 5 additions & 4 deletions api/routes/pipelex/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -334,8 +334,8 @@ async def validate_mthds(request: Request, request_data: ValidateRequest) -> JSO
separate questions — a caller building a fill-in form wants the inputs, a caller rendering a
result or registering a tool signature with a return type wants the output.
- **Invalid verdict (200, `is_valid: false`):** the `InvalidReport` arm — `validation_errors[]`
(the structured per-error diagnostics, built by pipelex's one shared builder, incl. the
`dry_run` residual item) + `message`, with the structural artifacts absent. The runner
(the structured per-error diagnostics, built by pipelex's one shared builder, incl. one
located `dry_run` item per pipe whose dry run failed) + `message`, with the structural artifacts absent. The runner
returns this as a value (`ErrorReport` with `validation_errors`) regardless of backend — the
in-process arm from the bundle's `ValidateBundleError`, the dispatched arm recovered from the
worker — so the route maps it to a 200 by matching validation diagnostics, never by catching an
Expand Down Expand Up @@ -449,8 +449,9 @@ def _invalid_report_response(error_report: ErrorReport, *, requested_formats: se

The `validation_errors[]` come straight from pipelex's one shared builder via
`ValidateBundleError.to_error_report()`, so the hosted invalid arm carries the same typed
items the agent CLI emits (including the `dry_run` residual item — the structured-info
invariant guarantees this list is non-empty on every invalid verdict that reaches the wire,
items the agent CLI emits (including one located `dry_run` item per pipe whose dry run failed —
the structured-info invariant, which the parse-level `blueprint_validation` residual makes total,
guarantees this list is non-empty on every invalid verdict that reaches the wire,
since the empty-`mthds_contents` edge case is a request-shape 422 via `min_length=1`).
`message` is the caller-facing summary the error report already carries.
"""
Expand Down
46 changes: 39 additions & 7 deletions docs/error-responses.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,15 +46,19 @@ When a bundle fails validation, the `ValidateBundleError` carries a `validation_
| `category` | The failure family — one of `blueprint_validation`, `pipe_factory`, `pipe_validation`, `dry_run`. |
| `message` | Human-readable description of this specific error. |
| `error_type` | Finer error subtype within the category, when the source error provides one. |
| `source` | The owning file of the error — present on `pipe_validation` and `blueprint_validation` items that the runtime could attribute to a file. On the in-memory submit path it is the matching `mthds_sources[i]` (see [Sourcing submitted files](pipe-validate.md)); `null` when the caller sent no sources. Absent for `pipe_factory` and `dry_run` errors (the latter is graph-level), and absent on the parse-level `blueprint_validation` residual (a raw TOML-syntax error, an empty blueprint, or an elaborator failure), which carries the failure message but no file attribution — see the note below. |
| `pipe_code`, `concept_code`, `domain_code` | The pipe / concept / domain the error is about, when applicable. |
| `source` | The owning file of the error, when the runtime could attribute it to one: on `blueprint_validation` and `pipe_validation` items, and on a `dry_run` item, where it is the file of the pipe whose dry run failed. On the in-memory submit path it is the matching `mthds_sources[i]` (see [Sourcing submitted files](pipe-validate.md)), so it is absent when the caller sent no sources. Absent on `pipe_factory` items and on the parse-level residual described below. Beside a server's own library directories, an item never names one of their files. |
| `pipe_code`, `concept_code`, `domain_code` | The pipe / concept / domain the error is about, when applicable. On a `dry_run` item they name the innermost pipe that failed, not the controller the failure passed through. |
| `field_path`, `field_name` | The offending field within the bundle, when the error localizes to one. |
| `variable_names`, `missing_concept_code`, `missing_pipe_code`, `declared_concepts` | Extra context for specific failure shapes (undefined variables, an unresolved concept or pipe reference, the set of concepts that were declared). |
| `line`, `column` | The 1-based position where the TOML parser stopped, on a TOML syntax error. |
| `model_reference`, `model_type`, `suggestions` | On an `unknown_model` item — a pipe naming a model the deployment's model deck does not define — the reference as the bundle wrote it, the kind of model the pipe needs (`llm`, `extract`, …), and the deck's close matches of that kind. With exactly one suggestion the item also carries an `unsafe` `suggested_fix` renaming the model. |
| `suggested_fix` | A structured, deterministic fix for this error — present only when the fix planner derived one. See [Suggested fixes](#suggested-fixes). |

Items carry only the fields that apply to their category — absent fields are omitted, not null. `validation_errors` is **retained under STRICT disclosure** (it describes the caller's own submitted bundle, not server internals). It is present only on `ValidateBundleError`; other error types omit it.

Every invalid verdict carries a **non-empty** `validation_errors` array — the structured-info invariant is total. A parse-level failure the runtime cannot attribute to a known pipe/concept/field — a raw TOML-syntax error, an empty blueprint, or an elaborator failure — still becomes one `blueprint_validation` residual item carrying the failure message (no `source`, no `error_type` at this layer), so the array is never empty on an invalid verdict. The richer, locator-bearing items appear only when the runtime could attribute the failure; the human-readable summary (the `detail` on a run-route 422, the `message` on a diagnostic-route 200 invalid verdict) stays available alongside, but a consumer can always read at least one structured item.
Every invalid verdict carries a **non-empty** `validation_errors` array — the structured-info invariant is total. A parse-level failure the runtime cannot attribute to a known pipe/concept/field — an empty blueprint, an elaborator failure — still becomes one `blueprint_validation` residual item carrying the failure message (no `source`, no `error_type` at this layer), so the array is never empty on an invalid verdict. A TOML syntax error is an item of the same category without an `error_type`, which carries the `line` and `column` the parser stopped at and the `source` when the caller sent one. A failing dry run is one `dry_run` item per failing pipe, located at the innermost pipe that failed. The richer, locator-bearing items appear only when the runtime could attribute the failure; the human-readable summary (the `detail` on a run-route 422, the `message` on a diagnostic-route 200 invalid verdict) stays available alongside, but a consumer can always read at least one structured item.

On the run routes, **every refusal of the bundle while it loads is this verdict**, and the load happens before any pipe runs: a misspelled concept, a wiring mismatch, an unknown model, a TOML fault or a refusal raised while a pipe is built all answer the same **422** carrying the same items validating the bundle gives. None of them reaches a run as a `500`. An entry pipe the bundle does not declare keeps its own `PipeNotFoundError`.

## Suggested fixes

Expand Down Expand Up @@ -88,7 +92,7 @@ A validation error item may carry a `suggested_fix`: a deterministic repair the

- `fix_code` — the kebab-case rule id that produced the fix (`match-sequence-output`, `sync-controller-inputs`, `strip-native-concept-redecl`, `strip-namespace`, …). Stable; use it to allow-list or suppress rules.
- `description` — human-readable summary of what the fix does.
- `safety` — `safe` or `unsafe`. Only apply an `unsafe` fix behind an explicit opt-in: it resolves an ambiguity the runtime could not resolve on the caller's behalf.
- `safety` — `safe` or `unsafe`. Only apply an `unsafe` fix behind an explicit opt-in: it is a likely correction, such as the one close match for an unknown model, that a person or an agent must confirm, and `pipelex fix bundle` never applies one on its own.
- `source` — the file the ops target, when known. **An applier must only apply ops to the file they target** — in a multi-file library the ops are meaningless against any other file.
- `ops` — the semantic TOML patch operations, in order.

Expand All @@ -110,6 +114,34 @@ For `ensure_table` and `delete_table`, `table_path` addresses the table itself r

**The ops are the machine contract; any rendered diff is presentation.** Apply them with a style-preserving TOML editor rather than reconstructing the file from a diff: that is what keeps the caller's formatting, comments, and key order intact.

## Run failures: the root fault, located at the failing pipe

When a run fails, the problem document describes the **root fault**, the innermost Pipelex error on the cause chain, never the run-level wrapper around it: `error_type`, `title` and `type` are the root fault's, and so are `error_domain`, the HTTP status and whether STRICT disclosure keeps the `detail`. The `detail` names the pipe that failed and its path from the entry pipe, `Pipe '<failing pipe>' failed (<entry pipe> → … → <failing pipe>): <the fault's own message>`, and `user_action` names the next step for that pipe. A consumer that branched on `error_type == "PipelineExecutionError"` branches on the root fault's type instead.

So a run the caller's own method refuses reads its reason even under STRICT: a `PipeCondition` whose outcome is `fail`, a `PipeParallel` branch whose multiplicity does not match its output field, a step started without a required input, or a model named inline that the deck does not define all answer a **422** in the `input` domain, with a `detail` that says what to change. A model the deck names but does not serve stays a redacted `config` failure, since the deployment, not the caller, has to fix it.

```http
HTTP/1.1 422 Unprocessable Entity
Content-Type: application/problem+json

{
"type": "https://docs.pipelex.com/latest/errors/stuff-factory-error/",
"title": "Stuff factory",
"status": 422,
"detail": "Pipe 'analyze_topic' failed (review_topic → analyze_topic): PipeParallel 'analyze_topic' cannot combine its branch results into its output 'TopicReview'. Branch 'draft_idea' gives result 'ideas' as a single 'Idea', but field 'ideas' of 'TopicReview' holds a list. Declare the field as a single concept in the structure of 'TopicReview', with type 'concept' and concept_ref 'Idea', or make branch 'draft_idea' output 'Idea[]'.",
"instance": "/v1/execute",
"error_type": "StuffFactoryError",
"error_domain": "input",
"user_action": {
"kind": "change_input",
"detail": "Branch 'draft_idea' gives result 'ideas' as a single 'Idea', but field 'ideas' of 'TopicReview' holds a list. Declare the field as a single concept in the structure of 'TopicReview', with type 'concept' and concept_ref 'Idea', or make branch 'draft_idea' output 'Idea[]'."
},
"request_id": "9f2c1ab3-…"
}
```

The failure of a `/start` run reaches its completion webhook as the same report, under the payload's `error` key (see [Async callbacks](#async-callbacks-webhook-payload)).

## Status codes

The HTTP status follows pipelex's `error_domain_to_http_status`:
Expand Down Expand Up @@ -146,7 +178,7 @@ The `ERROR_DISCLOSURE` env var controls how much of the originating error makes

- `verbose` (default) — renders the full `ErrorReport`. Use in dev, staging, and any deployment where the caller is trusted.
- `strict` — redacts `detail` and provider fields for errors that do not author caller-facing messages. Specifically:
- `detail` is preserved only for error classes flagged as authoring caller-facing messages (today: `MthdsParserError`, `ValidateBundleError`). Everything else has `detail` replaced with a generic title-derived string.
- `detail` is preserved only when the error authored a caller-facing message: the bundle's validation verdict (`ValidateBundleError`), a parse error (`MthdsParserError`), and a failure the runtime classifies as the caller's own, such as a pipe refused while it is built or a run that the submitted method itself refuses (see [Run failures](#run-failures-the-root-fault-located-at-the-failing-pipe)). Everything else has `detail` replaced with a generic title-derived string.
- `model`, `provider`, `provider_metadata` are always stripped — they have no business on a caller-facing surface.
- The redaction is keyed on the **provenance of the message** (`_authors_caller_facing_message` ClassVar), not on `error_domain`. A `RuntimeError` raised `from` an `INPUT`-domain cause does not leak the wrapper's internal message.

Expand Down Expand Up @@ -194,13 +226,13 @@ X-Request-ID: 9f2c1ab3-…
"type": "https://docs.pipelex.com/latest/errors/validate-bundle-error/",
"title": "Validate bundle",
"status": 422,
"detail": "Validation error(s):\n\nValue errors: 'main_pipe': Value error, Invalid main pipe syntax 'Not A Valid Pipe Code!'. Must be in snake_case.",
"detail": "Value error, Invalid main pipe syntax 'Not A Valid Pipe Code!'. Must be in snake_case.",
"instance": "/v1/execute",
"error_type": "ValidateBundleError",
"error_domain": "input",
"user_action": {
"kind": "change_input",
"detail": "Check the validation_errors array for specific issues"
"detail": "Edit the bundle as each validation error says: apply its suggested fix where it has one, after confirming an unsafe one"
},
"request_id": "9f2c1ab3-…",
"validation_errors": [
Expand Down
Loading
Loading