Repository navigation
secrets dump: error handling, --skip-errors, --start-from, --simple - #10
Open
thegreenflag wants to merge 6 commits into
Open
thegreenflag wants to merge 6 commits into
thegreenflag wants to merge 6 commits into
Conversation
wlatanowicz
requested review from
wlatanowicz
and
a balanced review from Copilot
October 29, 2026 12:29
There was a problem hiding this comment.
🟡 Changes recommended
A later-page parameter-listing failure can leave an existing output file truncated and partially overwritten, violating the promised listing-failure protection.
2 open issues
What changed in this PR
Improves ecsctrl secrets dump for large or partly inaccessible SSM parameter sets, with configurable error handling and simpler output.
Changes:
- Adds
--skip-errors, inclusive name filtering with--start-from, and lossy--simpleoutput. - Writes and flushes secrets incrementally and reports AWS errors through clean CLI messages.
- Documents the options and adds mocked tests for filtering, errors, output formats, and interruptions.
| File | Description |
|---|---|
tests/test_dump_secrets.py |
Tests name filtering and fetch-error handling. |
tests/cli/secrets/test_dump.py |
Tests CLI options, live writes, failures, and interruptions. |
tests/cli/secrets/__init__.py |
Adds the secrets CLI test package marker. |
README.md |
Documents new options and output behavior. |
ecsctrl/dump/secrets.py |
Implements filtering, fetch-error handling, incremental writes, and simple rendering. |
ecsctrl/cli.py |
Exposes new options and formats AWS errors for CLI users. |
Review effort: Balanced
| def dump_secrets( | ||
| ssm, filter=None, on_error=None, start_from=None | ||
| ) -> Generator[Parameter, None, None]: | ||
| for parameter in list_secrets(ssm): |
Comment on lines
+209
to
+212
| Use `--start-from` to dump only parameters whose names are alphabetically equal to or after the given value (e.g. to resume after a failure). It can be combined with `--filter`. | ||
|
|
||
| ```bash | ||
| ecsctrl secrets dump -e production.env --start-from "/prod/db" secrets.yaml |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


Summary
Improves
ecsctrl secrets dumpso it can handle large or partly inaccessible parameter sets and produce simpler output. The new options are optional, and the default behavior is the same as before, apart from error reporting.Changes
Error handling
get_parameter(for exampleAccessDeniedExceptionon a KMS key) now ends the command with a clean message and exit code 1, instead of a traceback:Error: /path/name: An error occurred (AccessDeniedException) ...describe_parameters(listing) is reported the same way. It never touches an existing output file.ClientErrorandBotoCoreErrorare handled. Other exceptions still propagate.New options
--skip-errors: a secret that fails to be fetched is reported on stderr (⚠️ Skipping <name>: <error>) and the dump continues.--start-from VALUE: only secrets whose name is alphabetically equal to or afterVALUEare fetched, which is useful for resuming a dump. The comparison is a plain, inclusive, case-sensitive string comparison. It doesn't rely on listing order, and skipped names cost noget_parametercalls. It can be combined with--filter.--simple: writes every secret asname: value, omittingTypeandDescription. By default, onlySecureStringparameters without a description are written that way.Output file
🔑 Dumped secret ...line prints at that point. An interrupted or failed dump keeps what was already written, and the error message says so.Notes
--simpleis lossy. Storing such a file withsecrets storecreates every parameter as aSecureString. This is documented in the README.--simpleoutput is unchanged.Testing
tests/cli/secrets/test_dump.pyand unit tests fordump_secretsintests/test_dump_secrets.py.--skip-errors--start-from(boundary, unsorted listing, pagination, combined with--filterand--skip-errors)--simple(including loading its output back through the secrets parser)black --check ecsctrlis clean.