feat(build)!: replace the legacy build and distribution surface - #802
feat(build)!: replace the legacy build and distribution surface#802guill wants to merge 3 commits into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
548c7c0 to
1f6bfb7
Compare
1f6bfb7 to
4f8054f
Compare
Swaps the whole public `comfy build` tree in one change, because the old and new verb sets cannot coexist: cmdline registration, the discovery map, the schema files, the emitted command labels and the error registry all name the same group, and the guardrail tests resolve every mention and schema key against the live Typer tree. New surface: init, push, pull, status, ls, show, validate, update, delete, plus the release, refs and blob subgroups. Gone: the `comfy distribution` alias and the scan / create / version / artifact download / from-snapshot / from-workflow / blob upload commands it fronted. Both importers survive as `init` and `update` flags. `--from-workflow` reads a ComfyUI workflow, in the editing dialect or the API export, into the local spec. It rides POST /v1/workflows/resolve rather than the builder's one-call POST /v1/builds/from-workflow: the spec `init` writes is a local file and `update` amends a build that already exists, so neither has anywhere to put a build row the server minted on its own. That read looks every distinct node class up in the registry, so it gets its own 90s timeout; the shared 30s default truncates a large graph. `--from-snapshot` and `--from-workflow` refuse each other, and both refuse the scan-only flags. `--base-image <id>` restores the base-image choice the removed `from-snapshot` verb carried, as a flag on `init` and `update`. It writes `baseImage` into the definition, which `push` already sends verbatim, so the choice reaches the build itself rather than sitting in the spec: the builder validates the id on every write and the baker resolves it when the image is baked. No new client method, and no local id check — only the builder knows the catalog, and `comfy build refs base-images` is how a caller reads it. An absent key selects the default, so the flag never writes a null. It is applied AFTER `merge_definition`, because a scan reports no base image and the merge carries the stored value forward, which would otherwise overwrite an incoming one; a plain `update` therefore keeps the base image it was not asked to change. Ports #779's advisory rendering with it. The renderer is shared with `--from-snapshot`, so this hardens a path that already shipped: server-supplied class names and filenames are scrubbed before they reach a terminal line (a crafted workflow could otherwise forge output that reads as the CLI's own), a long list says how many names it held back, and a key that arrives in a shape the renderer cannot read says so instead of vanishing — a dropped key is how a partial import comes to look clean. Canonical keys only, no aliases: `buildId`, `releaseId`, `builds`, `releases`. `build ls` emitted `distributions`, `build delete` and `_report_builder_error` emitted `distributionId`, and `build push` emitted `versionId`; all four now say what the builder says, and the four schemas follow. `build_release_logs` drops `versionId` outright rather than mirroring `releaseId` into it — the builder's own BuildLogContent requires `releaseId`, so the CLI was declaring a key its source does not serve. distribution_api.py, the deprecation shim, goes with the surface it shimmed. `create_release` now refuses a missing or empty target list, locally, before a request is issued: an implicit target spends build minutes nobody asked for. It could not land with the client in the previous change, because the `comfy build release create` of that moment took no `--target` and relied on the server's default; the guard arrives with `comfy build push`, which computes its own targets. Error registry: adds the seven codes deferred to the commands that raise them (build_spec_exists, build_missing_input, build_update_needs_confirm, build_spec_stale, build_pull_needs_confirm, build_id_unknown, build_release_not_found) and retires build_upload_unavailable, build_definition_invalid, build_missing_comfy_version and build_output_write_error with the paths that raised them. The deferred-code guard is added here, now that all seven have call sites. build_workflow_invalid keeps its registration and moves its call site onto the flag. Telemetry: adds `from_snapshot` to the redaction set. The retired `--from` was redacted because it is a local path naming the user's home directory and install layout; `--from-snapshot` carries exactly the same value, and the rename would otherwise have shipped it verbatim. `_scrub_value` cannot cover this — it only strips credentials out of URLs and returns a bare path untouched by design — so the key has to be named. Regression tests live with the other redaction cases in test_tracking.py. The bundled comfy-build skill is rewritten against the new surface. It had to be: `test_no_mentions_of_nonexistent_commands` resolves every backticked mention under comfy_cli/**/*.md against the live Typer tree, so the moment `scan` and `create` are gone the shipped skill fails the build. Its needle list in test_installer.py moves to init / push / release for the same reason. The judgement in that document — the pin discipline, the conflict prediction, the disclose-before-you-spend rules, the failure triage — is retargeted, not rewritten. Restores the four hints and docstrings the two preceding changes had to word around `comfy build init` / `push` / `pull`, which exist as of this change.
4f8054f to
303e31d
Compare
skishore23
left a comment
There was a problem hiding this comment.
The surface swap itself checks out — verified the new tree against the builder (POST /v1/workflows/resolve with {workflow} → {definition, report, format}, canonical buildId/releaseId/builds/releases, BuildLogContent.releaseId required, Definition.baseImage), the 90 s timeout reaches urlopen, the schemas match the emit sites for push/pull/status/init/update/release-*, the registry adds 7 / retires 4, push partial-failure leaves the spec consistent, and README/docs carry no references to the removed verbs. Two defects, both one-line fixes, are holding approval:
1. (high) tracking.py:111 — the redaction set adds from_snapshot and workflow but not from_workflow. Typer kwargs are keyed by parameter name, so comfy build init --from-workflow ~/ComfyUI/private/wf.json ships the path verbatim to PostHog/Mixpanel. Confirmed: filter_command_kwargs({"from_workflow": "/Users/x/ComfyUI/private/wf.json", "from_snapshot": ...}) returns the workflow path unredacted and <redacted> for the snapshot. The PR body's redaction rationale applies to both flags identically; test_tracking.py only covers from_snapshot.
2. (medium-high) update_cmd (build.py:~1649) via merge_definition (build_diff.py:241) — update --from-snapshot silently discards the base image the importer picked. The builder's snapshot resolver always writes baseImage (snapshot/resolve.go:95), but merge_definition carries every stored key outside _SCANNED_DEFINITION_KEYS forward over the incoming value and baseImage isn't scanned. Confirmed: stored cuda12-py311 + incoming cuda12-py312 → merged cuda12-py311, and the diff reports baseImage: unchanged. Net effect: re-importing a 3.12 snapshot keeps building on py311 while the advisory says pythonSatisfied: true for an image that isn't written. The "applied after the merge" placement only covers the explicit --base-image flag.
Non-blocking:
_builder_call(build.py:~2297) letssocket.timeout/TimeoutErrorandhttp.client.RemoteDisconnectedescape as tracebacks — urllib only wrapsrequest()inURLError, not the wait for response headers. Inherited, but this PR advertises the slow path (large--from-workflowgraphs hitting the 90 s budget) where it bites. Same pattern recurs in #803 (deploy_api.py) and #804 (all three data-plane clients) — worth one shared except tuple.- 17 new
CliRunner(mix_stderr=False)call sites are unguarded;main's existing tests use atry/except TypeErrorfallback because typer ≥ 0.27 / click ≥ 8.2 drop the kwarg.pyprojectpins onlytyper>=0.12.5, so a fresh resolve turns 177 tests red. _unrenderablesays "read it with --json", but the JSON payload carries only renderedadvisories, never the rawreport.
Tests on the branch with the mix_stderr shim: 549 passed, 1 skipped; ruff clean.
# Conflicts: # tests/comfy_cli/command/test_build.py
Two defects from the #802 review, and a third of the same shape found while fixing them. `--from-workflow` shipped the user's workflow path to analytics verbatim. Redaction is keyed on the Typer parameter name and `_scrub_value` returns a bare local path unchanged by design, so a path is redacted by being named in `_SENSITIVE_EXACT` or not at all. `from_snapshot` was named; its interchangeable sibling was not. `update --from-snapshot` discarded the base image the importer resolved. `merge_definition` carried every stored key outside the scanned set forward *over* the incoming value, so a snapshot resolving `cuda12-py312` kept building on the stored `cuda12-py311` -- and the diff reported no change at all, so nothing said so, while the advisory reported that image's Python as satisfied. The carry now tests the incoming definition rather than the key set alone: a local scan states no `baseImage`, so the stored one still survives a plain `update`, but an importer that resolves one now wins and the change appears in the diff the user confirms. `--base-image` is applied after the merge and still outranks both. `_merge_entry` carried entries the same way, and a snapshot resolves each pack's `commit`, which no scan reports. The stored SHA outranked the imported one and the entry then read as unchanged, so an import could never refresh a pinned commit. Same fix, same property: the scan path is byte-identical because a scan states only scanner-owned keys. Not addressed here, all called non-blocking in review: connection-level failures escaping the error mapping (one shared except tuple spanning this PR, #803 and #804 -- better as one focused change than three partial ones); the unguarded `CliRunner(mix_stderr=False)` call sites, which CI's locked resolve pins away from; and `_unrenderable` pointing at a `--json` payload that carries only rendered advisories. The CodeQL chmod alerts on these tests are false positives: every flagged line is a `finally` restoring owner-only bits, not a world-readable mask.
Stack of 7 — review bottom-up (this is 802)
feat(build): packaging, sync and Builder client primitives — basemainfeat(build)!: replace the legacy build and distribution surface (breaking) ← this PRfeat(deploy): the deployment control planefeat(deploy): the asset and job data-plane clientscomfy deploy run#805 —feat(deploy):comfy deploy runfix(build): a pull no longer deletes definition data silentlyfix(cli): unbreak the interactive pickers and Go timestamp parsing on 3.10Each PR is based on the branch below it, so its own diff is only its commit. Merge in order.
Rebased onto
main@c1fa1f4. The original bottom PR (#800, raising the Python floor to 3.11) was closed — see that thread for why; the stack is 3.10-compatible instead.TL;DR: Swaps the whole public
comfy buildtree in one change. The old and new verb sets cannot coexist — cmdline registration, the discovery map, the schema files, the emitted command labels and the error registry all name the same group, and the guardrail tests resolve every mention and schema key against the live Typer tree. This is the breaking change in the stack.Stacked on #801. Review from the bottom up; this is the largest diff in the series (73 files).
What changed
init,push,pull,status,ls,show,validate,update,delete, plus therelease,refsandblobsubgroups.comfy distributionalias and the scan / create / version / artifact download / from-snapshot / from-workflow / blob upload commands it fronted. Both importers survive asinitandupdateflags.distribution_api.py, the deprecation shim, goes with the surface it shimmed.--from-workflowreads a ComfyUI workflow — editing dialect or API export — into the local spec. It ridesPOST /v1/workflows/resolverather than the builder's one-callPOST /v1/builds/from-workflow: the specinitwrites is a local file andupdateamends a build that already exists, so neither has anywhere to put a build row the server minted on its own. That read looks every distinct node class up in the registry, so it gets its own 90s timeout — the shared 30s default truncates a large graph.--from-snapshotand--from-workflowrefuse each other, and both refuse the scan-only flags.--base-image <id>restores the base-image choice the removedfrom-snapshotverb carried, as a flag oninitandupdate. It writesbaseImageinto the definition, whichpushalready sends verbatim. No local id check — only the builder knows the catalog, andcomfy build refs base-imagesis how a caller reads it. Applied aftermerge_definition, because a scan reports no base image and the merge carries the stored value forward, which would otherwise overwrite an incoming one; a plainupdatetherefore keeps the base image it was not asked to change.--from-snapshot, so this hardens a path that already shipped: server-supplied class names and filenames are scrubbed before they reach a terminal line (a crafted workflow could otherwise forge output that reads as the CLI's own), a long list says how many names it held back, and a key arriving in an unexpected shape no longer makes a partial import look clean.buildId,releaseId,builds,releases.build lsemitteddistributions,build deleteand_report_builder_erroremitteddistributionId,build pushemittedversionId; all four now say what the builder says, and the four schemas follow.build_release_logsdropsversionIdoutright rather than mirroring — the builder's ownBuildLogContentrequiresreleaseId, so the CLI was declaring a key its source does not serve.create_releasenow refuses a missing or empty target list locally, before a request is issued: an implicit target spends build minutes nobody asked for. It could not land one PR earlier, when the verb took no--target.from_snapshotto the redaction set. The retired--fromwas redacted because it names the user's home directory and install layout;--from-snapshotcarries exactly the same value, and the rename would otherwise have shipped it verbatim._scrub_valuecannot cover this — it only strips credentials out of URLs and returns a bare path untouched by design — so the key has to be named.test_no_mentions_of_nonexistent_commandsresolves every backticked mention undercomfy_cli/**/*.mdagainst the live Typer tree. The judgement in that document — pin discipline, conflict prediction, disclose-before-you-spend, failure triage — is retargeted, not rewritten.Python 3.10 compatibility:
test_build_auth_matrix.pyimportsassert_neverfromtyping_extensions. The declared floor stays>=3.10.Validation
buildjob, Python 3.10) at this commit: 6419 passed, 38 skipped, 0 failed. The environmental failure noted in feat(build): add packaging, sync and Builder client primitives #801 is gone here — this change replaces the surface that test covered.ruff check ./ruff format --diff .clean at CI's pin (0.15.15).Contradicts: the previous
comfy build/comfy distributionverb set, deliberately and in full. This is the!in the commit subject.