Skip to content

Introduce the unified resource service and move each kind onto it #9489

Open

Description

@bobbai00

Task Summary

Today each kind is served by its own code in a different service: workflows and warehouses by the Dashboard Service, datasets and models by the File Service, computing units by the Computing Unit Managing Service, and environments by the computing unit itself. This task adds one resource service that keeps the records of every kind, built on the tables of #9484, the API of #9485 and the interfaces of #9486.

  1. Reads first. The service lists, reads and searches every kind from the new tables, and answers access checks. Nothing else changes yet.
  2. Then writes, one kind at a time: warehouses and environments, then datasets and models, then computing units, and workflows last. For each kind, its writes move to the resource service, its triggers are dropped, its old tables become read-only views, and its old routes call the new code, so the frontend does not notice. A resource created after the switch still gets an old id, so the current frontend and old links keep working.
  3. Lineage. Saving a version records the versions it uses, and an execution or a clone records what its result was derived from.
  4. Kind-specific work stays where it is. The File Service keeps uploading and downloading files, and the Computing Unit Managing Service keeps starting and stopping pods. Both use the shared code for versions and access, so the resource service holds no storage or cluster credentials.

Discussion points

  1. Creating storage and pods: the File Service creates a dataset's storage on its first upload and the Computing Unit Managing Service starts a pod when a new unit appears, or the resource service calls them when a resource is created.
  2. The Dashboard Service: after Upgrade amber/web from Dropwizard 1.3.23 to 4.0.7 #5423, merge its remaining parts, such as sign-in and users, into the resource service, or keep them apart.

Done when all six kinds are created, read, versioned and shared through the resource service, and their old routes only call it.

Task Type

  • Refactor / Cleanup
  • DevOps / Deployment / CI
  • Testing / QA
  • Documentation
  • Performance
  • Other

Activity

  1. added theissue type on Nov 16, 2026
  2. changed the title [-]Serve every kind's reads through the unified resource endpoints[/-] [+]Introduce the unified resource service and move each kind onto it[/+] on Nov 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions