feat: support Platform API 1.101 features (catch-up backlog)

[cristianrcv]

Summary

Exposes API features that were already in Platform API 1.101.0 or earlier (1.101.0 is the oldest spec in tower-sdk-gencode's history), which tw never surfaced.

This is one PR in a stack that catches tw up with Platform API 1.230.0, one PR per API version that introduced the features, then one PR per topic for agents and actions. Each PR adds the items whose API elements all exist in that version.

• Stack: master → 1.101 (feat: support Platform API 1.101 features (catch-up backlog) #685) → 1.113 (feat: support Platform API 1.113 features (SSH keys) #686) → 1.145 (feat: support Platform API 1.145 features (roles, folder mounts, linked datasets) #697) → 1.167 (feat: support Platform API 1.167 features (audit logs, IdP groups, SCIM, Intelligent Compute) #698) → 1.190 (feat: add 'tw info versions' and studio stars #699) → 1.200 (feat: add the lineage commands and studio logs #700) → 1.230 (fix(launch): stop launch --wait duplicate runs and support API 1.230 options [COMP-2645] #701) → agents (feat: add the agents and service-accounts command groups #702) → actions (feat(actions): add bucket, cron and pipeline-status sources and trigger history #703). This PR is based on master; the next one is feat: support Platform API 1.113 features (SSH keys) #686.
• SDK: unchanged. tw stays on tower-java-sdk 1.200.0, which already contains every API element used here. The last PR in the stack bumps it to 1.230.0.

Changes

• feat(compute-envs): add CloudWatch log group and DRAGEN options to AWS Batch (5642525)
• feat(compute-envs): add --propagate-head-job-options to HPC platforms (ab7c9dc)
• feat(compute-envs): add --data-retention-policy to Seqera Compute (4ee5cd4)
• feat(compute-envs): add log group and project ID cloud options (2cc169a)
• feat(compute-envs): add enable and disable commands (be7c471)
• feat(launch): add --tower-config option (e0c5267)
• feat(runs): add 'tw runs star' and 'tw runs unstar' (2a19581)
• feat(runs): delete several runs in one 'tw runs delete' (b331c35)
• feat(runs): add 'tw runs view log' to print a run or task execution log (b21111e)
• feat(runs): page through run logs with --next (45a4efa)
• fix(pipelines): keep labels and icon on 'tw pipelines update' (b32f4f3)
• feat(pipelines): add 'tw pipelines schema' to read a pipeline's parameter schema (4fa3264)
• feat(studios): add extend and rename-checkpoint (c968b07)
• feat(studios): show progress warnings while waiting (8b87123)
• feat(data-links): add delete-content and download-script (98bbddb)
• fix(data-links): stop printing a Jersey warning on delete-content (7adcc8f)
• feat(datasets): disable dataset versions (23b5cc2)
• feat(teams): add view, update and workspaces commands (52edfd3)
• fix(teams): find teams by name beyond the first page (e6b6978)
• feat(organizations): add quotas command (7bcb946)
• feat(access-tokens): add access-tokens list, add and delete commands (e9377d3)
• feat(managed-identities): add managed identities and their SSH credentials (03a046c)
• fix(managed-identities): list credentials for every member (a8b1ecc)
• test: capture the native binary's output verbatim (65ce579)
• docs: document the commands and options added for API 1.101.0 (f9f5838)
• build(native): register reflection metadata for the new commands (49b33e3)
• build(native): register every field of the new response classes (72a4526)

Notes for review

• Changes to existing behaviour:
  • pipelines update now sends back the existing labels and icon. Platform's update replaces both, so they were silently dropped before.
  • Looking up a team by name now finds teams beyond the first page of 50, in both teams and participants.
• Jersey warning: data-links delete-content is a DELETE with a body. Jersey logs Entity must be null for http method DELETE to stderr through JUL, so that logger is raised to SEVERE.
• Test harness: BaseCmdTest.execBinary now copies the native binary's output verbatim. Re-joining lines with println turned line feeds inside printed content into CRLF on Windows.
• conf/reflect-config.json:
  • backfills entries that master was already missing for 17 existing classes (e.g. computeenvs.ValidateCmd, RunView);
  • registers allDeclaredFields for the new responses, so -o json doesn't lose fields in the native binary.
• Adaptations for SDK 1.200.0: the SDK types LogPage.entries as Object (the spec says Iterator_String_). Managed-identity 409 bodies are parsed as DeleteCredentialsConflictResponse.

Verification

• ./gradlew test: 819 tests, 0 failures. Every new command and option has MockServer tests that assert the request (method, path, query, body) and the output.
• Each item was checked against the OpenAPI spec of this version and the previous one (tower-sdk-gencode history) to confirm where it first appears.
• The native binary built from this branch passed the full test suite (TOWER_CLI=build/native/nativeCompile/tw ./gradlew cleanTest test).

🤖 Generated with Claude Code

[cristianrcv]

@educalleja @t0randr @swampie @jonmarti Rob Syme reported a bug on CLI that some missing features were returning a 400. Within that context I reviewed all the SDK bumps against the features supported by the CLI and opened a bunch of pull requests. While the other ones can be targgeted per SDK bump, this targets all the old features that the CLI does not support.

Before doing the effort of splitting this into PRs, do you know if supporting these features is needed in the CLI? Are the users on your use cases missing or complaining about these missing features?

[swampie]

Staging test report — full stack, cloud.stage-seqera.io (Platform API 1.231.0)

Every PR in the stack was tested against its own native binary, built from its own branch head, against live staging in org seqerakit (user is a root user). No mock servers — all results below are real API calls.

Note on numbering. The stack was re-created at 16:18 UTC while testing was in progress, so the PRs I tested as #687–#692 are now closed and replaced, with agents and actions split out into their own PRs. I verified that every defective code path below is present verbatim in the replacement branches, so the findings carry over unchanged. Mapping used throughout:

tested as	now	notes
#685	#685	unchanged
#686	#686	unchanged
#687	#697	bucket actions moved to #703
#688	#703	cron actions folded into the actions PR
#689	#698
#690	#699 + #702	agents extracted to #702
#691	#700 + #702	agents extracted to #702
#692	#701 + #702 + #703

---

PR health at a glance

PR	Area	Verdict
#685	API 1.101 catch-up	⚠️ had a blocker — now fixed (B1, merged as ed484abe)
#686	1.113 — SSH keys	✅ healthy — no defects found
#697	1.145 — roles, folder mounts, linked datasets	✅ healthy (1 cosmetic nit, m4)
#698	1.167 — audit logs, IdP/SCIM, Intelligent Compute	🛑 blocker — B2
#699	1.190 — studio stars, Nextflow versions	🛑 blocker — B4 (fixed by ed484abe, not yet in this line)
#700	1.200 — lineage, studio logs, IC limits	🛑 blocker — B3
#701	1.230 — SDK bump, service accounts, federation	⚠️ major — M2
#702	agents command group	✅ healthy — and it resolves M1
#703	actions: bucket/cron/pipeline-status + triggers	⚠️ major — M3 (server-side), plus m1

Healthy: #686, #697, #702. Everything else has at least one open issue.

---

⚠️ Read first: the auth fix is not in the rest of the stack

ed484abe (the B1/B4 fix, merged into feat/NOTASK-api-1.101-catch-up) is not an ancestor of any other branch in the stack, including #686's head — they all fork from 72a45266, which predates the merge.

ed484abe present in: feat/NOTASK-api-1.101-catch-up  only
            absent in: #686, #697, #698, #699, #700, #701, #702, #703

So tw runs star, tw runs unstar and tw info versions are still broken everywhere downstream until #686 is rebased onto the updated #685 and the rebase propagates. Nothing is wrong with the fix — this is purely propagation.

---

🛑 Blocking defects

B2 — tw audit-logs list / view cannot parse a successful response — #698

The request succeeds (HTTP 200, valid JSON). The CLI then fails to deserialize it:

JsonMappingException: AuditLogTargetStateNewState cannot be null
  ListAuditLogV2Response["auditLogs"]→[0]→["target"]→["state"]→AuditLogTargetState["previousState"]

The server returns "state":{"previousState":null,"newState":null,…}. AuditLogTargetStateNewState is a generated oneOf wrapper (34 variants) whose deserializer has:

public AuditLogTargetStateNewState getNullValue(DeserializationContext ctx) throws JsonMappingException {
    throw new JsonMappingException(ctx.getParser(), "AuditLogTargetStateNewState cannot be null");
}

— unconditional, the generator's pattern for a oneOf not marked nullable: true.

• Frequency: 41 of 41 sampled logs have previousState: null. This is the normal case, not an edge case.
• Not fixed by the SDK bump — reproduced identically against SDK 1.230.0 via a direct Jackson probe.
• audit-logs export is unaffected (CSV is rendered server-side) — usable workaround today.
• Not a permissions issue: the export CSV shows actorIsRoot: true.
• User-facing symptom is misleading: it reports Connection error. Check the connection using 'tw info' when the network is fine.

Fix options: mark those two fields nullable: true in the spec (fixes all SDK consumers); or override the null handling CLI-side — there is precedent in AbstractApiCmd, which already registers TaskExitMixin for a structurally identical SDK mismatch.

B3 — tw lineage view returns 404 for every record — #700

Against the same encoded lid, same workspace, same token:

endpoint	status
/lineage/{lid} ← used by the default view	404
/lineage/{lid}/summary	200
/lineage/{lid}/upstream	200
/lineage/{lid}/downstream	200

Reproduced for FileOutput, TaskRun and WorkflowRun. The record is demonstrably present — three sibling routes resolve the identical identifier.

• Workaround: tw lineage view --summary works.
• Ruled out — path encoding. The CLI emits a double-encoded segment (%253A, %252F), which looked like the obvious culprit. It isn't: single-encoded, double-encoded and literal-slash forms all 404, and the siblings succeed with the identical double-encoded segment.
• Ruled out — response shape as a diagnostic. Every 404 on this API is empty-bodied, including known-good routes with bogus ids, so it can't distinguish a missing route from a missing record.
• Undetermined from the client: whether /lineage/{lid} is undeployed at 1.231.0, or deployed but unable to materialise the record. The --summary help text hints at a mechanism — summary comes from the lineage index, while the full record's recordUri is an S3 object (s3://seqera-lineage-…/.data.json). Needs someone with server/bucket visibility.
• Caveat: all 1000 lineage records on staging live in one workspace (64430228002560), so a workspace-specific condition can't be excluded.

B4 — tw info versions sends no Authorization header — #699

Same root cause as B1: NextflowApi.nextflowVersions passes null authNames, and ApiClient.invokeAPI skips authentication entirely when that array is null rather than falling back.

Confirmed by curl: GET /platform/versions → 401 without auth, 200 with auth.

Already fixed by ed484abe — but see the propagation note above. This was the third command hit by this root cause, which is why that fix defaults auth generally instead of special-casing the star endpoints.

---

⚠️ Major

M2 — the documented service-account workflow does not work — #701

#701's own USAGE.md says: "Add one to a workspace with tw participants add -t MEMBER -n <name>." That command fails:

ERROR: Service accounts cannot be added as workspace collaborators

AddCmd's MEMBER branch calls findOrganizationMemberByName, falling back to userNameOrEmail on MemberNotFoundException. Service accounts are not returned by /orgs/{id}/members (?search=… → {"members":[],"totalSize":0}), so the lookup always throws and the fallback sends a shape the server rejects for service accounts.

The API supports it — PUT …/participants/add with {"memberId":227302255131700} → 200, userType: SERVICE_ACCOUNT. That member id is already surfaced by tw service-accounts view ("Member ID").

This blocks the service-account → agent flow end to end; I had to add the participant via raw API to test agents at all. participants update -t MEMBER is unaffected (it resolves against existing participants).

M3 — tw actions add pipeline-status fails server-side — #703

HTTP 400 {"message":"Unexpected error while processing request - Error ID: …"}

Reproduced via raw curl, identically, for three different watched-pipeline ids including a bogus one — so the CLI's request is well-formed and this is a platform failure, not a CLI defect. The body sent looks correct:

{"launch":{…},"name":"…","pipelineStatus":{"pipelineId":196256443154544,"runStatus":"FAILED"},"source":"pipeline_status"}

Error IDs for the platform team: 3iars46Gfj1Y1MD4Px6wvt, 7TirUNr5EOFtlC7mpzENm9, 1w0X4z3ZTcx9VCrNQG9ZWp, 2EYZrnbdzxBCg6m7VKskJk.

Consequence: actions add pipeline-status is unverified, and actions triggers list|view could only be exercised against an action with no triggers.

M1 — agents add was unusable before the agents split — resolved in #702

The server requires serviceAccountId; SDK 1.200.0's CreateAgentRequest has no such field, so it was unsettable in the old #690/#691. SDK 1.230.0 adds it and #702 adds the matching --service-account-id. Verified working: agent created, agents view shows Service account | cli-test-sa. Recorded only because it meant the old 1.190/1.200 PRs could not be validated standalone — the split into #702 resolves that.

---

Minor

id	issue	where
m1	AbstractAddCmd.java:107 does catch (Exception e) and throws "Unable to add action for workspace '<ws>'", discarding the server body — e is never used. For a bad cron the server actually says {"message":"Cron expression contains 3 parts but we expect one of [5]"}. Pre-existing on master; cron makes it far more visible.	#703
m2	Empty-body errors render as ERROR: error (seen on 403s from seqera-compute add, rename-checkpoint, bulk run delete; 404s from lineage). The response carries X-Request-Id, which could be surfaced.	pre-existing
m3	actions list -w <ws> always prints Actions for <user> user: regardless of -w (ActionsList.java:54).	pre-existing
m4	tw roles view -n nope → Role nope doesn't exist (double space).	#697
m5	teams delete accepts only -i, while #685 added -n to teams view/update/workspaces.	#685
m6	Latent: SDK AuditEventType lacks action_triggered, which the server emits (12 of 100 sampled logs). Currently masked by B2 — likely to surface as the next failure once B2 is fixed. Worth folding into the same spec pass.	#698

---

What passed

Substantial parts of the stack are solid. Highlights verified end-to-end against staging:

• feat: support Platform API 1.113 features (SSH keys) #686 — SSH keys: full lifecycle, both ref forms, both output formats, 4 distinct error paths, and keys round-trip byte-identical through -k <file> and -k -.
• feat: support Platform API 1.145 features (roles, folder mounts, linked datasets) #697 — roles: full custom-role lifecycle; -p correctly replaces permissions; deletion refused while assigned ("assigned to 0 participants and 1 teams"); participants update --role accepts a custom role. Linked datasets (--url) and studios folder mounts (mountDataV2: {dataLinkId, path}) both correct, including trailing-slash normalisation.
• feat(actions): add bucket, cron and pipeline-status sources and trigger history #703 — cron: next-execution arithmetic correct in both directions (02:00 Europe/London → 01:00 GMT; Mon 06:30 America/New_York → 10:30 GMT), UTC defaulting, and the --marker-file/--events source guard.
• feat: support Platform API 1.167 features (audit logs, IdP groups, SCIM, Intelligent Compute) #698 — Intelligent Compute: all options map correctly on AWS/Google/Azure (pool.{enabled,desiredWarm,scaleToZeroSecs} etc.), and every out-of-range value is rejected with no CE created.
• fix(launch): stop launch --wait duplicate runs and support API 1.230 options [COMP-2645] #701 — PATCH on Java 25 works in the native image (service-accounts update → PATCH → 200, change applied). This was the riskiest item in that PR. Service accounts, federation-setup (both providers, correctly rejecting azure), --fusion-version, --billing-export-table, --key-vault-url, and lineage resolve-by-run all pass.
• feat: support Platform API 1.101 features (catch-up backlog) #685 fixes: pipelines update now preserves labels (verified: aws,gcp survive a description-only update), and team-lookup-by-name resolves exact matches over prefix siblings.

Behaviour-changing fixes in #685 were regression-tested specifically, since those carry the most risk for existing users.

---

Not covered — stated explicitly rather than implied

• IdP groups / SCIM (feat: support Platform API 1.167 features (audit logs, IdP groups, SCIM, Intelligent Compute) #698) — IdP claims mapping is not enabled on the org; skipped by agreement.
• agents launch happy path (feat: add the agents and service-accounts command groups #702) — org is out of Seqera credits. The command reaches the server and errors cleanly.
• studios logs with real content — aws-cloud CE creation began returning HTTP 400 {"message":"Malformed JSON"} for any body partway through the session. A previously-built binary fails identically, so this is staging-side, not a PR regression. Only the empty-log path was verified.
• launch --wait through failed polls (fix(launch): stop launch --wait duplicate runs and support API 1.230 options [COMP-2645] #701) — needs a transient API failure to exercise; code-reviewed only (the PR adds 80 lines of unit tests for it).
• Compute-env options requiring real cloud resources (DRAGEN, log groups, project ids, warm pools) — verified at the wire level by asserting the serialized request body; the server-side effect is unverified.

Housekeeping: all test artifacts were removed and the engineering team was restored to its original admin role. One exception — an azure-cloud CE cli-kv (7g8TjHbBFHSZe3R4neH2rY) in seqerakit/megatest13 is stuck in CREATING and refuses deletion (Unable to delete a compute environment in CREATING state); it needs manual cleanup.

---

Suggested order of attack

1. Propagate ed484abe — rebase feat: support Platform API 1.113 features (SSH keys) #686 so B1/B4 clear across the whole stack. Cheapest, unblocks three commands.
2. B2 — spec nullable fix, or the CLI-side mixin workaround if you'd rather not wait.
3. B3 and M3 — both need Platform-side investigation; Error IDs are above.
4. M2 — resolve service accounts by member id in participants add.

🤖 Generated with Claude Code

[swampie]

@cristianrcv the commit claude references above is the small correction I did to the tip of the stack but the actual fix would be the one you did in the platform api. So feel free to ignore

[cristianrcv]

@swampie I have rebased this PR and the next 2 to master so that they can be merged independently. Major issues are addressed and I will address the minor issues as a follow-up once all is merged (it is easier to track and fix).