Skip to content

secrets dump: error handling, --skip-errors, --start-from, --simple - #10

Open
thegreenflag wants to merge 6 commits into
wlatanowicz:masterfrom
thegreenflag:feat/secrets-dump-skip-errors
Open

thegreenflag wants to merge 6 commits into
wlatanowicz:masterfrom
thegreenflag:feat/secrets-dump-skip-errors

Conversation

@thegreenflag

Copy link
Copy Markdown

Summary

Improves ecsctrl secrets dump so it can handle large or partly inaccessible parameter sets and produce simpler output. The new options are optional, and the default behavior is the same as before, apart from error reporting.

Changes

Error handling

  • A failed get_parameter (for example AccessDeniedException on a KMS key) now ends the command with a clean message and exit code 1, instead of a traceback:
    Error: /path/name: An error occurred (AccessDeniedException) ...
  • A failed describe_parameters (listing) is reported the same way. It never touches an existing output file.
  • Only ClientError and BotoCoreError are handled. Other exceptions still propagate.

New options

  • --skip-errors: a secret that fails to be fetched is reported on stderr (⚠️ Skipping <name>: <error>) and the dump continues.
  • --start-from VALUE: only secrets whose name is alphabetically equal to or after VALUE are fetched, which is useful for resuming a dump. The comparison is a plain, inclusive, case-sensitive string comparison. It doesn't rely on listing order, and skipped names cost no get_parameter calls. It can be combined with --filter.
  • --simple: writes every secret as name: value, omitting Type and Description. By default, only SecureString parameters without a description are written that way.

Output file

  • Each secret is appended and flushed as soon as it's fetched, and its 🔑 Dumped secret ... line prints at that point. An interrupted or failed dump keeps what was already written, and the error message says so.
  • The output file is created when the first secret arrives, so a failure while listing leaves an existing file untouched. If nothing matches, an empty file is still created, as before.

Notes

  • --simple is lossy. Storing such a file with secrets store creates every parameter as a SecureString. This is documented in the README.
  • The default, non---simple output is unchanged.

Testing

  • 36 tests, all passing: CLI tests in tests/cli/secrets/test_dump.py and unit tests for dump_secrets in tests/test_dump_secrets.py.
  • Covered:
    • fail-fast and --skip-errors
    • listing failures
    • --start-from (boundary, unsorted listing, pagination, combined with --filter and --skip-errors)
    • --simple (including loading its output back through the secrets parser)
    • live writing and interruption
  • black --check ecsctrl is clean.
  • Everything runs against mocked boto3, with no AWS calls.

@wlatanowicz
wlatanowicz requested review from wlatanowicz and a balanced review from Copilot October 29, 2026 12:29

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

A later-page parameter-listing failure can leave an existing output file truncated and partially overwritten, violating the promised listing-failure protection.

2 open issues
What changed in this PR

Improves ecsctrl secrets dump for large or partly inaccessible SSM parameter sets, with configurable error handling and simpler output.

Changes:

  • Adds --skip-errors, inclusive name filtering with --start-from, and lossy --simple output.
  • Writes and flushes secrets incrementally and reports AWS errors through clean CLI messages.
  • Documents the options and adds mocked tests for filtering, errors, output formats, and interruptions.
File Description
tests/​test_dump_secrets.py Tests name filtering and fetch-error handling.
tests/​cli/​secrets/​test_dump.py Tests CLI options, live writes, failures, and interruptions.
tests/​cli/​secrets/​__init__.py Adds the secrets CLI test package marker.
README.md Documents new options and output behavior.
ecsctrl/​dump/​secrets.py Implements filtering, fetch-error handling, incremental writes, and simple rendering.
ecsctrl/​cli.py Exposes new options and formats AWS errors for CLI users.

Review effort: Balanced

Comment thread ecsctrl/dump/secrets.py
def dump_secrets(
ssm, filter=None, on_error=None, start_from=None
) -> Generator[Parameter, None, None]:
for parameter in list_secrets(ssm):
Comment thread README.md
Comment on lines +209 to +212
Use `--start-from` to dump only parameters whose names are alphabetically equal to or after the given value (e.g. to resume after a failure). It can be combined with `--filter`.

```bash
ecsctrl secrets dump -e production.env --start-from "/prod/db" secrets.yaml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants