Skip to content

feat: add a workflow status view and honor limit on GET /workflows - #238

Merged
kaiitunnz merged 9 commits into
mainfrom
kaiitunnz/confident-dirac-gb2uzj
Oct 17, 2026
Merged

kaiitunnz merged 9 commits into
mainfrom
kaiitunnz/confident-dirac-gb2uzj

Conversation

@kaiitunnz

@kaiitunnz kaiitunnz commented Oct 17, 2026 •

Copy link
Copy Markdown
Collaborator

Purpose

Relates to #225; follow-up to #227. Workflow pollers and listers have the same two problems #227 fixed for tasks:

  • Every poll returns the whole workflow. GET /api/v1/workflows/{id} returns task_ids plus the dispatched, completed, failed and cancelled task-ID lists, so the payload grows with the size of the graph. flowmesh workflow watch and workflows.wait() poll it on every tick and read only status.
  • GET /api/v1/workflows ignored limit. It was treated as an unknown filter key, so every workflow came back, ordered by random uuid4 id.

With this PR, a poller can ask for a workflow's status view, which holds per-state task counts instead of the ID lists. limit keeps the newest matching workflows, the SDK and CLI pollers use the status view, and both CLI list commands can bound their output.

Changes

  • Status view. ?view=status on GET /workflows/{id} and GET /workflows returns a WorkflowStatusView (src/server/schemas/workflow.py): workflow_id, status, submitted_at and updated_at, which match their Workflow fields, plus task_count, dispatched_count, completed_count, failed_count and cancelled_count. The default view stays full.
  • Listing order and limit. GET /workflows lists workflows oldest submission first, with ties broken by id; a submitted_at that can't be parsed sorts oldest. limit (1–10000) keeps the newest matching workflows, applied after filtering and permissions. Neither view nor limit is treated as a field filter.
  • SDK.
    • workflows.status(id) and workflows.list_statuses(...) return status views, sync and async.
    • workflows.list(limit=...).
    • workflows.wait_status() polls until the workflow is DONE, FAILED or CANCELLED and returns its status view. workflows.wait() polls the same way, then fetches the full workflow once; its return type is unchanged.
  • CLI.
    • flowmesh workflow watch polls the status view and still prints the full workflow at the end.
    • flowmesh workflow list and flowmesh task list take --limit N (1–10000), which keeps the newest N matches.
  • Documentation. docs/API.md, docs/SDK.md and docs/CLI.md cover view, limit, --limit and the new SDK calls. The GET /workflows row now lists its real filters (workflow_id, status, task_ids); owner was listed but isn't a Workflow field, so it was silently ignored.
  • Tests. A new router suite, plus SDK request-building, SDK model, schema-parity and CLI tests.

Design

The view mirrors #227's opt-in design, so released SDKs keep working. The counts are the only fields with no Workflow counterpart. They keep the payload a fixed size while still showing progress; a poller that needs the failed task IDs can call tasks.list_statuses(workflow_id=..., status="FAILED").

Because the counts aren't a subset of Workflow, the SDK's WorkflowStatusView derives any missing count from the matching ID list. An older server ignores view and returns the full workflow, and that still validates, so the new SDK calls work against any server.

Workflows have no submitted_ts, so the order key parses submitted_at, shaped like task_order.

Left for later:

  • Cursor pages for /workflows.
  • Server-side Redis cost. limit and view bound the response, not the reads: GET /workflows still reads every accessible workflow before it filters and sorts, as GET /tasks does.

Test Plan

uv sync --all-packages --group ci --frozen
uv run pre-commit run --all-files
uvx bandit==1.9.4 -c pyproject.toml -r src/
uv run pytest tests/ --ignore=tests/worker/test_mp_executor_cleanup_gpu.py

Test Result

  • uv run pre-commit run --all-files: passed (gitleaks, isort, black, ruff, codespell, mypy, and requirements sync).
  • bandit: no issues.
  • uv run pytest tests/ --ignore=tests/worker/test_mp_executor_cleanup_gpu.py: 3091 passed, 54 skipped.
  • New tests cover:
    • the status view returning exactly its declared fields, with the shared fields equal to the full view's and each of the five counts equal to its list's length;
    • each shared field matching the type and description of its Workflow counterpart;
    • a bad view being a 422 on both routes, and limit=0 and limit=10001 being 422s;
    • the list ordering by submission, an unparsable submitted_at sorting oldest, limit keeping the newest, and limit applying after a status filter and after permissions;
    • status(), list_statuses() and wait() requesting view=status, sync and async, with wait() then fetching the full workflow once;
    • list(limit=) sending limit without view;
    • the SDK status view parsing both a status-view payload and a full workflow;
    • flowmesh workflow watch polling the status view and printing the full workflow once;
    • flowmesh workflow list --limit and flowmesh task list --limit forwarding the limit, defaulting to none, and rejecting out-of-range values before calling the server.
  • End to end, on a local stack with two CPU workers, all checks passed. echo_local.yaml and echo_three_node_graph.yaml workflows ran to DONE and a graph with a failing python node ended FAILED; for each, the status view held exactly its fields, matched the full workflow on the shared ones, and had each count equal to its list's length. GET /workflows listed them in submission order, limit kept the newest after status, workflow_id and task_ids filters, and bad view and limit values got 422s. The sync and async SDK calls returned the right types and values, and the server's access log showed wait() polling ?view=status and fetching the full workflow once. flowmesh workflow list --limit and flowmesh task list --limit printed the newest matches and rejected out-of-range values. flowmesh workflow watch on a running ssh_debug_shell.yaml workflow followed it to CANCELLED after flowmesh workflow cancel, polling the status view and fetching the full workflow exactly once.

Pre-submission Checklist
  • I have read the contribution guidelines.
  • I have run pre-commit run --all-files and fixed any issues.
  • I have added or updated tests covering my changes (if applicable).
  • I have verified that uv run pytest tests/ passes locally.
  • If I changed shared schemas or proto definitions, I have checked downstream compatibility across Server and Worker.
  • If I changed the SDK or CLI, I have verified the affected packages work (uv sync --all-packages --group ci --frozen).
  • If this is a breaking change, I have prefixed the PR title with [BREAKING] and described migration steps above.
  • I have updated documentation or config examples if user-facing behavior changed.

Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
@kaiitunnz
kaiitunnz force-pushed the kaiitunnz/confident-dirac-gb2uzj branch from 8eaae1b to e931aa7 Compare October 17, 2026 08:31
Signed-off-by: Noppanat Wadlom <noppanat.wad@gmail.com>
@kaiitunnz
kaiitunnz merged commit 3ae8ab1 into main Oct 17, 2026
11 checks passed
@kaiitunnz
kaiitunnz deleted the kaiitunnz/confident-dirac-gb2uzj branch October 17, 2026 10:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant