Skip to content

feat: structured errors and exit codes - #78

Merged
edornd merged 12 commits into
v2from
66-structured-errors
Oct 31, 2026
Merged

edornd merged 12 commits into
v2from
66-structured-errors

Conversation

@edornd

@edornd edornd commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Summary

A failed run now prints one JSON error object on stdout instead of a traceback or argparse usage text. The exit code tells the cases apart: 0 ok, 1 the command failed, 2 the input was rejected.

App.__call__(argv)
  _run(argv)
    parse_known_args          argparse error   -> VALIDATION_ERROR, argparse message, exit 2
    leftover tokens           -> unknown_flag / unexpected_argument field errors
    CliApp.run                ValidationError  -> one field error per pydantic error
                              SettingsError    -> VALIDATION_ERROR, its message, exit 2
    any field errors          -> VALIDATION_ERROR, exit 2
    command(**inputs)         CommandError     -> its own code and exit code (default 1)
                              other Exception  -> UNKNOWN, exit 1, traceback on stderr if ARGD_DEBUG

Error object: {code, message, retryable, hint?, fieldErrors?}. Field error: {path, code, message, expected?, received?, allowed?, source?}.

  • path is the dotted input path (optim.lr), not the flag spelling.
  • allowed holds Literal and enum choices, or for unknown_flag the flags the command accepts.
  • received is left out for SecretStr and SecretBytes fields and for missing.
  • message keeps only its first line. CommandError raises ValueError if its code is not SCREAMING_CASE, or if its exit code is 0 or 2, which mean success and rejected input.
  • Malformed JSON in a list, dict or model flag (--sizes '[1,') raises pydantic-settings' SettingsError. It gives VALIDATION_ERROR and exit 2. pydantic-settings names the field only in its message text, so this report has no fieldErrors.
  • ARGD_DEBUG uses pydantic's bool parsing. A value it cannot parse counts as off.
  • Leaf parsers set allow_abbrev=False, so --epo is an unknown flag instead of a silent --epochs.
  • A word right after an unknown flag counts as that flag's value. Any other stray word gets the code unexpected_argument with path: "". So do -, -- and negative numbers such as -1 and -.5.
  • argdantic.errors._string_errors = True prints pydantic's and argparse's own text instead of the object. It exists for the benchmark's structured-vs-string ablation and is not exported.

New module src/argdantic/errors.py. Only CommandError is exported from argdantic. app.py uses four names from it: EXIT_OK, InputRejected, fail and rejected_input.

Evidence

  • Before: add-numbers --first-value x --secnd 3 exited 2 with argparse usage text on stderr, and --first 1 quietly ran as --first-value 1.
    After:
    {"code": "VALIDATION_ERROR", "message": "invalid input", "retryable": false, "fieldErrors": [
      {"path": "secnd", "code": "unknown_flag", "message": "unknown flag --secnd", "received": "--secnd",
       "allowed": ["--first-value", "--second-value"], "source": "flag"},
      {"path": "first_value", "code": "int_parsing", "message": "Input should be a valid integer, unable to parse string as an integer",
       "received": "x", "source": "flag"}]}
  • tests/test_errors.py has one or more tests per acceptance criterion. 60 tests pass, and ruff and ty are clean.

Merge Danger

Door: two-way. The output shape is new on an unreleased branch, and nothing outside this repo consumes it yet.

Blast Radius: contained. Groups (#67) and output formats (#70) build on this output and its exit codes.

edornd added 12 commits October 29, 2026 22:18
A rejected input now prints a VALIDATION_ERROR object on stdout and exits
with code 2. It has one field error per pydantic error, with the dotted
input path and pydantic's error type as the code. Literal and enum fields
list their choices in `allowed`. pydantic validates Literal choices, so
argparse never rejects them first.

Refs #66
argv is now parsed with parse_known_args. Each leftover flag becomes an
unknown_flag field error, and its `allowed` list holds the flags the
command accepts. A word right after an unknown flag counts as that flag's
value. Any other stray word becomes an unexpected_argument error.
Unknown flags come first, then pydantic's errors, all in one
VALIDATION_ERROR.

Errors that argparse finds itself, such as a flag without its value,
also give VALIDATION_ERROR and exit code 2, with argparse's message
as-is. Prefix matching is off, so --epo no longer means --epochs.

Refs #66
A command raises CommandError(code, message, *, retryable=False,
hint=None, exit_code=1). argdantic prints it with the command's own code
and exits with its exit code. The constructor raises ValueError if the
code is not SCREAMING_CASE.

Every error object now always has `retryable`. `hint` appears only when
it is set. `message` keeps only its first line.

Refs #66
An exception the command does not expect now prints an UNKNOWN error
and exits with code 1. The message is the exception text, or the
exception's type name when the text is empty. This also covers a result
that fails to serialise. KeyboardInterrupt and other BaseExceptions
pass through.

The traceback goes to stderr only when ARGD_DEBUG is on. pydantic's
bool parsing reads the value, and a value it cannot parse counts as
off.

Refs #66
A field error on a SecretStr or SecretBytes field has no `received`
key, so a rejected secret never reaches stdout. The check also covers
secrets inside Optional, Annotated and nested models.

Refs #66
The benchmark compares structured errors against plain text errors.
Setting argdantic._errors._string_errors = True makes a failed run print
the text pydantic and argparse produce on their own. For CommandError
and uncaught exceptions it prints the exception text. Exit codes and the
output stream stay the same. The switch is not exported from argdantic.

Refs #66
A field error now has `expected` when pydantic gives one, for example
"'parquet' or 'webdataset'" for a Literal or enum field. That completes
the structured error object.

Closes #66
The package's other internal module, docstrings.py, has no underscore, and
CommandError is public, so its repr should not point at a private module.
The bench switch keeps its underscore: argdantic.errors._string_errors.

Refs #66
app.py imported eleven names from the errors module and dispatched on
exception type itself. It now imports four: EXIT_OK, InputRejected,
fail and rejected_input.

- InputRejected and CommandError each carry their own report, so fail()
  is the one place that picks a report, prints it and exits. The
  ARGD_DEBUG traceback moves there too, out of report construction.
- rejected_input() builds the whole rejection from leftover tokens and a
  ValidationError, including the text the string-errors ablation prints.
- FieldError and ErrorReport are pydantic models, so model_dump replaces
  the two hand-written to_dict methods. ErrorReport keeps only the first
  line of its message.

The JSON output is unchanged.

Refs #66
pydantic-settings raises SettingsError, not ValidationError, when a list,
dict or model flag holds JSON it cannot parse. That fell through to
UNKNOWN with exit 1, as if the command had failed. It is now
VALIDATION_ERROR with exit 2. The field name appears only in
pydantic-settings' message text, so the report carries that message and
no fieldErrors.

Refs #66
CommandError(exit_code=0) printed an error object and exited 0, and
exit_code=2 looked like rejected input. A caller could no longer tell
success, a failed command and rejected input apart from the exit code.

Refs #66
A leftover "--" became unknown_flag with an empty path and swallowed the
next word as its value. "-.5" became unknown_flag with path ".5". Both
are now unexpected_argument, like "-1" and "-" already were.

Refs #66
@edornd
edornd added this pull request to stack #81 October 31, 2026 19:48
@edornd
edornd merged commit c5738fa into v2 Oct 31, 2026
5 checks passed
@edornd
edornd deleted the 66-structured-errors branch October 31, 2026 19:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant