Repository navigation
feat(actions): add bucket, cron and pipeline-status sources and trigger history - #703
Open
cristianrcv wants to merge 80 commits into
Open
cristianrcv wants to merge 80 commits into
cristianrcv wants to merge 80 commits into
Conversation
…S Batch The Platform API (at least since 1.101.0) accepts AwsBatchConfig.logGroup and the ForgeConfig DRAGEN fields (dragenEnabled, dragenAmiId, dragenInstanceType), and the web UI exposes them, but tw never set them. - --log-group for 'aws-batch forge' and 'aws-batch manual' - --dragen, --dragen-ami-id, --dragen-instance-type for 'aws-batch forge' DRAGEN settings are checked client-side because the AMI and instance type are only validated during asynchronous forge creation, so a missing value would otherwise surface later as an errored compute environment. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Slurm, LSF, Univa Grid Engine, Altair PBS and Moab configs have accepted propagateHeadJobOptions since at least API 1.101.0, and the web UI shows it as 'Apply head job submit options to compute jobs'. tw never set it, so CLI-created HPC environments could not pass the head job submit options to the Nextflow compute jobs (process.clusterOptions). The option requires --head-job-options: Platform reads the head job options unguarded when propagation is on, so the combination is rejected up front. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
SeqeraComputeConfig.defaultDataRetentionPolicy (API 1.101.0 or earlier) asks Seqera Compute to put a lifecycle policy on the bucket it creates, deleting intermediate files after 28 days. The web UI shows it as 'Automatic Data Retention Policy' and turns it on by default; tw never sent it, so CLI-created environments always got a bucket without the policy. The flag is opt-in, which keeps the current tw behaviour when it is not given. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two cloud config fields accepted by the Platform API (at least since 1.101.0) were never set by tw: - aws-cloud --log-group (AwsCloudConfig.logGroup): CloudWatch Logs group for the CE; Platform defaults it to /seqera/platform. - google-cloud --project-id (GoogleCloudConfig.projectId): written as google.project and used to qualify short network/subnetwork names. Platform does not fill it from the credentials for Google Cloud. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The API has long exposed POST /compute-envs/{id}/enable and /disable
(EnableComputeEnv, DisableComputeEnv), but tw offered no way to call
them, so a disabled compute environment could only be re-enabled from
the UI.
Both commands take the usual -i/-n and -w references, like
compute-envs delete. The endpoints return 204, so the output only
confirms the action. Platform rejects invalid transitions (e.g.
enabling an AVAILABLE CE) with a 400 whose message tw prints as is;
disabling a primary CE also unsets it as primary.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
WorkflowLaunchRequest has carried towerConfig since at least API 1.101.0, but it could not be set from the CLI. It is the per-run Seqera configuration in tower.yml format, overriding the pipeline's tower.yml (the "Seqera Cloud config file" field of the launch form; Platform caps it at 3500 characters). --tower-config is a plain string, so it goes through the same coalesce chains as --config in launch, pipelines add|update, actions add|update and runs relaunch. Actions are wired too, because LaunchOptions exposes the option there and leaving it unapplied would silently drop it. The stored value is inherited by pipelines update and runs relaunch through ModelHelper's JSON copy; the tests now assert that. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
POST/DELETE /workflow/{workflowId}/star have been in the API since at
least 1.101.0, but runs could only be starred from the web UI. Starring
is per user, so a starred run surfaces in that user's run list in the UI.
Unstarring a run that is not starred is a 404 on Platform; it is
reported like any other API error rather than reinterpreted.
The starred state is not added to 'tw runs view': it would cost an extra
request, and the SDK types GET /workflow/{id}/star as
CreateWorkflowStarResponse (workflowId only) although Platform returns
DescribeWorkflowStarResponse with a 'starred' flag, so the flag cannot be
read through the SDK.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
POST /workflow/delete (deleteWorkflowMany, in the API since at least
1.101.0) deletes a list of runs and reports the ones it could not delete,
but tw only took one id, so cleaning up runs meant one invocation per
run.
'-i' now accepts several ids, repeated or comma-separated. A single id
keeps calling DELETE /workflow/{id}, so its output and errors are
unchanged. Several ids go through the bulk endpoint with --force passed
on; each run is reported as deleted or not, and the command exits 1 when
any could not be deleted so scripts notice partial failures. Platform
caps the list size and rejects longer ones with a 400.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
'tw runs view download' and 'tw runs dump' fetch log files through
/workflow/{id}/download, which serves the files the run leaves in its
work directory. GET /workflow/{id}/log and /log/{taskId} (in the API
since at least 1.101.0) are what the UI's Execution log tab uses: Platform
reads the compute platform's live log stream when the provider has one
and only falls back to the output file otherwise, so they also show the
output of a run that is still going. tw had no way to reach them.
'tw runs view -i <id> log' prints the Nextflow head job output and
'-t <taskId>' the task output. Platform's own notice (for example why the
log could not be read) is shown after the entries, and a truncated page
says so and points to 'download' for the complete file.
Only the first page is fetched. Following the cursor would turn the
command into a 'tail -f', and how the cursor behaves differs per compute
platform, so that is left for when someone needs it.
Up to SDK 1.200.0 the spec types LogPage.entries as an Iterator object,
so the SDK exposes it as Object; Platform sends a JSON array of strings,
which LogCmd reads back as a list.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Live log streams such as CloudWatch return one page per call (up to 10k events or 1 MB) without
flagging truncation. So 'runs view log' printed the first page of a long log as if it were all of
it. Print the forward cursor and accept it with --next (the 'next' query parameter of
GET /workflow/{id}/log, in the API since at least 1.101.0). An empty page ends the hint, because
CloudWatch keeps returning a cursor at the end of the stream.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The update sent no labelIds and no icon. Platform's update replaces the
pipeline's labels with labelIds; the API docs for both PUT
/pipelines/{id} and POST /pipelines/{id}/versions/{versionId} say a null
or omitted labelIds removes existing labels. It also sets the icon to
the request value and, when that is null, recomputes it from the
repository, so an uploaded icon was replaced. The web form sends back
the current labels and icon; tw now does the same, reading the labels
through describePipeline's 'labels' attribute.
Found while checking whether moving to updatePipeline (PUT
/pipelines/{id}) would gain anything. It would not: both endpoints call
the same PipelineService.updatePipeline, and PUT is the versionId=null
case, which only adds auto-promoting a new draft version to default.
tw already handles that, with --allow-draft on top.
The label loss is inferred from the Platform source and API docs, not
reproduced against a live Platform: LabelRefServiceImpl.applyLabelsToEntities
evaluates labelIds.contains(...) for each existing label.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…eter schema
GET /pipelines/{pipelineId}/schema (describePipelineSchema, in the API
since at least 1.101.0) returns the schema Platform uses to render a
pipeline's launch form: the custom schema attached to the launch
configuration if there is one, otherwise the repository's
nextflow_schema.json. tw could attach a custom schema
('tw pipeline-schemas add', 'tw pipelines add --pipeline-schema-id') but
never read one back, and had no way to see which parameters a pipeline
takes before writing a --params-file.
'tw pipelines schema -n <name>' prints the schema; '--params' prints the
parameter values the pipeline launches with instead (pipeline defaults
merged with the stored launch parameters). --version-id/--version-name
select a version, as in 'tw pipelines view'. Platform answers 204 when it
cannot resolve a schema; that is reported as an error rather than empty
output.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Platform API (at least since 1.101.0) exposes studio operations that tw
never wired up: lifespan extension (POST /studios/{sessionId}/lifespan)
and checkpoint renaming (PUT /studios/{sessionId}/checkpoints/{checkpointId}).
This adds 'tw studios extend|rename-checkpoint' so CLI users can do what
the UI already does.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DataStudioProgressStep.warnings (in the API since at least 1.101.0; e.g. cross-region data, failed checkpoint mounts) were never shown. They are now printed once each while waiting on a studio with '--wait'. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Two data-explorer operations available since API 1.101.0 had no tw
command:
- 'tw data-links delete-content' calls DELETE /data-links/{id}/content
to delete files (--file) and folders with their contents (--dir)
while keeping the data link. The platform supports it for Seqera
Compute data links only, and answers a partial deletion with 500 plus
the failed items, which tw reports per item instead of a bare error.
- 'tw data-links download-script' calls GET
/data-links/{id}/script/download and prints the cloud CLI commands
(aws s3 cp, gcloud storage cp, ...) the UI offers for downloading.
GET /data-links/{id}/download/{filePath} is not added: it streams the
same file through the platform, with a size limit, that
'tw data-links download' already fetches through a signed URL.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
DELETE /data-links/{id}/content takes a request body. The SDK disables Jersey's HTTP compliance check,
so the body is sent, but Jersey still logs 'Entity must be null for http method DELETE' to stderr
through JUL on every call. In-process JVM tests don't capture System.err, so only the native binary
tests caught it. Raise that logger to SEVERE.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
POST /datasets/{id}/versions/{version}/disable (API 1.101.0 or earlier)
had no tw command, so a dataset version could only be disabled from the
UI.
- 'tw datasets disable-version --dataset-version <n>' disables it; the
version can no longer be used for runs.
- 'datasets view ... versions' gains a Disabled column
(DatasetVersionDto.disabled), so the result is visible.
DatasetsCmdTest now resets MockServer before each test, as the other
suites do, so request verifications are not affected by earlier tests.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The API has long had describeOrganizationTeam, updateOrganizationTeam and listWorkspacesByTeam (at least since 1.101.0), but tw could only list, add and delete teams. Add 'teams view', 'teams update' and 'teams workspaces', addressing a team by --id or --name. The update endpoint replaces name, description and avatar, so 'teams update' reads the team first and keeps unset fields. The team response only carries the avatar download URL, so the avatar ID is taken from its last path segment to avoid clearing the avatar. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
listOrganizationTeams pages by 50 by default, so looking a team up by name missed any team after the first page. That affects the new teams view/update/workspaces commands and the existing participants commands. Pass the name as the search filter and keep the exact-name match. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
describeOrganizationQuotas (/orgs/{orgId}/quotas, in the API since at
least 1.101.0) had no tw command.
Add 'tw organizations quotas' to show the organization limits.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The /tokens endpoints (API 1.101) let users manage their personal access tokens, but tw could only consume one. 'tw access-tokens' now lists, creates and deletes them by id or name. 'add' prints the token value once, since Platform never returns it again. 'list' drops the deprecated basicAuth field, which is a usable credential for legacy tokens. DELETE /tokens/delete-all is not exposed: tw has no confirmation pattern for bulk destructive operations, and it would also revoke the token tw itself is running with. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…tials Managed identities (/identities, API 1.101) are organization-level HPC cluster connections (Altair PBS, LSF, Moab, Slurm, UGE): an owner registers the login host and port once, and each member attaches their own SSH key and Linux username as managed credentials. tw could not manage either. 'tw managed-identities' adds list/add/view/update/delete, and 'tw managed-identities credentials' lists members' credentials and adds, replaces or deletes them, for yourself or (as owner) for another member. Deletions honour Platform's running-jobs check unless --force. Two SDK gaps are worked around: the describe endpoint is modelled with the wrong response shape, so identities are looked up in the list; and the managed credentials metadata model lacks the required SSH userName, so tw sends it through a small subclass. SDK 1.200.0 generates a separate conflict type for each 409 response, so the managed credentials 409 body is parsed as the credentials one (same shape) to reuse CredentialsInUseException. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The endpoint returns one row per organization member, 50 by default. The list silently stopped at 50 members. Page through all rows, 100 at a time, in the helper that the list and the member lookups share. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
execBinary read the binary's stdout line by line and re-printed each line with println. On Windows that turned every line feed inside printed content into CRLF. The new commands print server content verbatim (e.g. data-link download scripts), so their binary tests failed only on the Windows native job even though tw's output was right. Copy the raw streams instead, which matches how in-process tests capture output. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sections for teams and organization quotas, access tokens and managed identities, plus a summary of the new options on existing commands. Examples are plain commands in bash blocks (markdownlint MD014). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native image only sees picocli options and Jackson-serialized response fields through reflection. Register the classes added in this branch (and the existing ones the config lacked), copied from the entries the GraalVM tracing agent recorded for the full catch-up to API 1.230.0. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The traced entries for some response classes listed only the fields the tests happened to read, while existing responses register allDeclaredFields. Jackson serializes the whole class for -o json, so a field missing from the list (e.g. RunLog.next) silently disappears from the native binary's JSON output. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Platform API 1.113 added user-scoped SSH public keys (/ssh-keys), used for SSH access to Studios, but tw had no way to manage them. The new 'tw ssh-keys' group lists, adds (from a .pub file or stdin), views and deletes the authenticated user's keys, looking them up by id or name. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Adds a Personal access section with the SSH keys commands, so users can find the new 'tw ssh-keys' group without reading the CLI help. The example uses a bash block (markdownlint MD014 rejects $-prompted blocks without output) and parses against the built CLI. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The native binary resolves picocli options and serializes responses by reflection, so the new ssh-keys command, option and response classes need entries. They are copied from the GraalVM 25 tracing-agent run of the full API catch-up. The SDK models for the API 1.113 /ssh-keys endpoints were already registered. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Platform API 1.230.0 (PLAT-6639) added the pipeline_status action source: an action that launches its pipeline when a run of a watched Launchpad pipeline ends in a chosen state. tw had no way to create one. - 'actions add pipeline-status --watch-pipeline-id <id> --run-status <state>' sends CreateActionRequest.pipelineStatus. - 'actions update --watch-pipeline-id/--run-status' sends a partial UpdateActionRequest.pipelineStatus; the API keeps omitted fields. - Only SUCCEEDED, FAILED and CANCELLED can fire the action, as in the UI and the API's terminal-state check, so tw refuses other states before calling the API instead of hiding the 400 behind its generic error. - 'actions view' shows the watched pipeline and run state. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Platform API 1.230.0 records every time an action fires, with how it
ended (launched, launch failed, or suppressed by the cycle or rate guard)
and the event that caused it. The UI shows it as the action's trigger
history; tw had no way to see why an action did or did not launch.
- 'actions triggers list -n|-i <action> [--outcome ...] [--max/--page/
--offset]' calls GET /actions/{id}/triggers, newest first.
- 'actions triggers view -n|-i <action> --trigger-id <id>' calls
GET /actions/{id}/triggers/{triggerId} and prints the event payload.
The event summary, outcome detail and payload come from outside
Platform (a GitHub commit message, an S3 notification), and the API
marks them as untrusted. The console output strips their control
characters through FormatHelper.formatUntrusted so they cannot inject
terminal escape sequences. JSON output keeps them as sent, escaped by
the JSON encoder.
The firedAfterOrEqual/firedBeforeOrEqual filters are not exposed yet.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Bucket, cron and pipeline-status sources for actions add and update, plus actions triggers list|view. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Entries come from a full test run under the GraalVM tracing agent. Response classes register allDeclaredFields, so -o json keeps every field in the native binary. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Oct 5, 2026
Open
Merged
cristianrcv
added this pull request to stack #704
October 5, 2026 16:12
This was referenced Oct 5, 2026
Closed
cristianrcv
removed this pull request from stack #704
October 5, 2026 16:20
cristianrcv
added this pull request to stack #705
October 5, 2026 16:20
jonmarti
approved these changes
Oct 5, 2026
|
this one is on me, hold from merging until strand applies the necessary changes. |
cristianrcv
removed this pull request from stack #705
October 7, 2026 13:26
cristianrcv
force-pushed
the
feat/NOTASK-tw-agents
branch
from
October 7, 2026 13:50
72f3deb to
8e8d05e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the new action sources and trigger history, grouped in one PR so it reviews as a single feature. It spans three API versions:
actions triggers list|view.This is one PR in a stack that catches tw up with Platform API 1.230.0: one PR per API version that introduced the features, then one PR per topic for agents and actions, which are grouped across versions so they're easier to review.
tower-java-sdk1.230.0, from the 1.230 PR.Changes
Notes for review
actions listno longer throws an NPE for actions without an endpoint: cron and pipeline-status actions, and bucket actions until provisioned.actions viewis gone.actions updaterejects options for a different source before calling the API. Otherwise the API's 400 is hidden behind tw's generic "Unable to update action" error.--tower-config/--fusion-versionwiring inactions add|updatestays with those options, in the 1.101 and 1.230 PRs.Verification
./gradlew test: 989 tests, 0 failures. Every new command and option has MockServer tests that assert the request (method, path, query, body) and the output.🤖 Generated with Claude Code