mailsec: bulk remediation across a caller-supplied selection

[maximelb]

Binds the three bulk-remediation routes the API gateway serves as SDK methods and two CLI verbs. This is what turns any search result — the message index, a hunt, a shell pipeline — into provider-side action, instead of a loop of single-message calls.

The routes

Route	Permission	Shape
POST /mailsec/{oid}/actions/bulk/preview	mailsec.get	Reads the index, reports per-message state, mints a selection-bound confirm. Changes nothing and creates no job. A POST only because it carries up to 500 ids.
POST /mailsec/{oid}/actions/bulk/execute	mailsec.act	Accepts a confirmed action and returns a bulk_id immediately. The provider work runs on the collector that holds the org's lease.
GET /mailsec/{oid}/actions/bulk/{bulk_id}	mailsec.get	The durable job record: state, running counts, per-message items with each one's action_id, and stalled.

Vocabulary: quarantine_message, trash_message, move_to_spam, restore_message, banner_message, unbanner_message. Note it is not the per-message list — it adds move_to_spam and drops submit_to_triage, which moves no mail.

The property this is built around

The confirm token is derived from the exact selection, not issued as a nonce: it hashes (oid, action, attempt, normalized member list). Any change to any of those invalidates it. So a client that re-ran its search between the preview a human read and the execute it sent is refused rather than acting on messages nobody approved.

The CLI therefore normalizes one list — trim, drop blanks, dedupe, sort, mirroring the server's own normalization — and never rebuilds it. Because the token binds the attempt and the action too, the help text names all three (--action, --msg-uuids, --attempt) rather than only the selection: previewing with --attempt inc-9 and executing without it is a stale-token error pointing at a list the caller never changed.

CLI

limacharlie mailsec message bulk-action --action ACTION
    [--msg-uuids a,b,c]... [--input-file uuids.txt|-]
    [--attempt TOKEN] [--banner HTML] [--confirm TOKEN]
    [--wait/--no-wait] [--timeout 300] [--poll-interval 3]

limacharlie mailsec message bulk-status BULK_ID

The grammar is the siblings' grammar. campaign action and hunt remediate, in this same file, spell the two-step one way: omit --confirm to preview, pass --confirm TOKEN to execute. bulk-action now does the same.

no --confirm	Runs the preview. The blast-radius report goes to stderr for a human; the preview document, confirm token included, goes to stdout for a script. Exit 0. Nothing is changed.
--confirm TOKEN	Skips the preview entirely — re-previewing would mint a fresh token and discard the one that was reviewed — and executes the normalized selection.

There is no --yes, no --preview-only, no prompt and no mutual-exclusion guard. Note that within the mailsec group --confirm takes a value, unlike the boolean --confirm used by 19 other command modules; a bare --confirm now fails with click's "requires an argument", which is the safe direction.

Selection input: --msg-uuids (repeatable, comma-separated), --input-file (a JSON/YAML list or one id per line, - for stdin), or — following hive.py's idiom — piped stdin when neither is named.

• Every member is listed, including the ones no longer indexed. The point of the preview is the blast radius, and a truncated one under-reports exactly the thing being approved. mailbox_count is shown beside the message count because "38 messages" and "38 people's inboxes" feel very different.
• The wait is bounded and always terminates with exactly one line, whichever of complete, interrupted, stalled or the deadline stopped it.
• stdout carries exactly one document. Progress narration, the job handle and the terminal line go to stderr, so --output json keeps its promise.

The exit code carries the outcome

A runbook chains with && and reads nothing else, so the outcome has to be in the exit code:

Outcome	Exit
Preview produced a token	0
--no-wait, accepted: true	0
complete, at least one message acted on	0 (per-item failures are the caller's data, in counts)
complete with ok == 0 and failures	non-zero — nothing was remediated
interrupted / stalled / still running at the timeout	non-zero
accepted != true, no bulk_id, or a poll that errored	non-zero

The stall message states the gateway's own documented repair: re-send the same execute request with the same confirmation token to finish it — every message already acted on collapses onto its existing action row rather than being acted on twice.

--quiet silences both streams, as it does everywhere in this CLI; with it, the exit code above and bulk-status are how the result is read. The explain text says so.

Two deliberate deviations from the obvious shape

No --reason. The gateway's execute allow-list forwards only action, msg_uuids, confirm, attempt and the banner. A reason would be accepted by this client and dropped in transit — a justification that silently never reaches the audit trail is worse than one the caller knows it cannot give. Documented in the explain text, which points at mailsec message action --reason for the per-message case.

No client-side 500 cap. The server refuses an oversized selection rather than truncating it, and names the cap in the refusal (error_code: bulk_too_large). A client that trimmed 900 to 500 would report success over a batch whose tail was silently left in inboxes nobody will look at.

Honest outcomes, preserved

Expired ids and already-done messages are reported, not dropped and not errors — acting on a search result taken minutes ago legitimately includes both. They stay in the confirmed set and the provider re-checks each one, because already_in_target_state is read off the index and is advisory. Partial failure is reported beside what worked and is never a rollback: provider actions are not transactional, and "restoring" a quarantined message is a second visible move in somebody's mailbox rather than an undo.

Review pass

Beyond the grammar and the exit codes, six things the review found and this now fixes:

• The bulk_id could be orphaned. Measured: a 502 on the first poll exited 1 with an empty stdout and the id nowhere, leaving a running job with no handle. It is now announced on stderr before anything that can fail, and a failed poll prints the resume command and puts the acceptance document (which carries the id) on stdout.
• --poll-interval 0 issued 30,580 requests in two seconds, and a negative one crashed after the provider writes were already ordered. Both bounds are now click.IntRange, enforced before the execute fires.
• A pasted CSV export silently corrupted the consent surface. Its header row and second column became members, and the preview reports a phantom member as "NOT IN INDEX" — the same answer a legitimately expired id gets — so member_count and mailbox_count were wrong in a way nobody could see. Ids are now shape-checked against the msg_uuid format (the backend derives them as v5-shaped UUIDs) and a bad one is refused by name.
• The preview rendered itself away in a table. Measured: format_table renders messages as [2 items] and summary as {7 keys}, and those two are the preview. Now that the preview is this verb's default output, it falls back to JSON when the format would be a table. (The token itself was never truncated — the gateway's is 32 hex characters and the renderer's cell width never drops below 40 — but it is also printed on stderr, so a copy-paste does not depend on that arithmetic staying true.)
• The wait loop moved into the SDK as Mailsec.wait_for_bulk, mirroring Jobs.wait: three terminal conditions, the last status document returned either way, no timeout exception. The CLI half is now narration only. It uses the existing status route, so the route inventory stays at 28.
• Reuse convergence: --input-file goes through _input_helpers' canonical loader (whose docstring mandates it for new code) and accepts -; _note becomes a shared note() beside command_output in _output_helpers. The local id splitter deliberately stays local — a cross-module comma-split helper would widen this PR past its subject — but is now comma-only, matching the twelve existing comma-splitters.

One thing the review flagged that turned out to be correct as written, checked against the gateway source: the SDK sends banner, and mailsecBulkExecuteArgs translates it to the collector's banner_html. The pin stays. Its test comment did not: it claimed the allow-list would drop a literal banner_html, which is false (it forwards both spellings), and a pin that encodes a wrong belief about the gateway is worse than no pin. The comment now says what the allow-list actually does and why the documented public name is still the one to pin.

Tests

tests/unit/test_cli_mailsec_bulk.py (54 tests) and the TestBulkSelectionNormalization / TestBulkRemediation / TestWaitForBulk trio in tests/unit/test_sdk_mailsec.py (64 tests in the file). The CLI tests wire wait_for_bulk to the real SDK implementation and stub only the HTTP-facing status call, so the loop whose terminal semantics decide the exit code is the shipped one.

Covered: preview-as-default and the token on stdout; the two-step sending a byte-identical selection across two invocations; every exit code in the table above; the first-poll exception leaving the bulk_id in both streams; --poll-interval and --timeout rejections happening before the execute; stdin via - and via a bare pipe; JSON-array and YAML-list input files; the CSV-paste refusal naming its offending token; the table-format preview keeping its substance; and wait_for_bulk's three terminal conditions. The route-coverage inventory stays exact at 28.

The old prompt-path tests are not carried forward: they were inert, because the sys.stdin.isatty patch never took effect under CliRunner.

Full suite: 4083 passed, 5 skipped (up from 4048; the 5 are pre-existing platform/docstring skips).