Skip to content

feat(ui): Bulk Export workspace — scoped kick-off, filters, active exports - #538

Open
angela-helios wants to merge 4 commits into
mainfrom
feat/537-bulk-export-ui
Open

feat(ui): Bulk Export workspace — scoped kick-off, filters, active exports#538
angela-helios wants to merge 4 commits into
mainfrom
feat/537-bulk-export-ui

Conversation

@angela-helios

Copy link
Copy Markdown
Contributor

Closes #537

The pull-based companion to the Bulk Import workspace: a web UI over the server's own, already-shipped $export operation, replacing the "Export" coming-soon placeholder in the Batch & Data nav. First functional round of the new Export design.

/ui/bulk-export

  • Scope ("What are you exporting?"): Everything / Patients / Group (with a Group id field), matching the design's copy.
  • Resource types: checkbox grid sourced from the tenant's live resource-type list → _type. Nothing checked = every type.
  • Narrow it down: _elements, _typeFilter, and _since as presets (last day / 7 days / 4 weeks / all time) or a custom RFC 3339 instant.
  • Optional job name ("Diabetes registry 2024") and an Active-exports shortcut with the running count.

Self-calls that work under auth

Kick-offs, status polls, and cancels target this same server via the request's Host header, forwarding the caller's own Authorization and tenant — the export runs with the user's credentials, no service token needed (unlike the conformance self-fetch, which still needs HFS_OUTBOUND_BEARER_TOKEN). Relative Content-Location poll URLs resolve against the same base.

/ui/bulk-export/active

One card per job, newest first, with status chips (in progress / complete / failed / cancelled):

  • In progress: the poll's X-Progress verbatim + Cancel (DELETEs the job server-side). Cards htmx-refresh every 5s; each fetch performs exactly one server-side poll, and the completed card simply lacks the trigger — the same no-JS polling pattern as the import workspace.
  • Complete: file count, elapsed time, and the completion manifest's output files as download links.
  • Failed: the kick-off/poll error verbatim + Retry (same parameters, fresh kick-off).

Job state persists in the per-user settings document under the reserved byTenant subtree, like the import workspace.

Tests

The suite mounts the UI over a mock FHIR export backend on a real socket, so the Host-header self-calls loop back into the mock: kick-off query shape asserted at the API (_type/_elements/_since), Patient and Group paths, the 202-progress → 200-manifest lifecycle (polling starts and stops where it should), rejected kick-off → failed card → retry re-runs it, and cancel arriving as a DELETE. Full helios-ui suite green (incl. i18n parity ×3), clippy clean, --features ui builds.

Note: one implementation detail worth a look — the export form is parsed by hand via form_urlencoded because the repeated types checkboxes exceed what serde_urlencoded accepts.

…ports

The pull-based companion to the Bulk Import workspace (#537), driving
the server's own $export operation from the web UI and replacing the
Export placeholder in the Batch & Data nav.

- /ui/bulk-export: scope choice (everything / patients / group with a
  Group id), a resource-type checkbox grid sourced from the tenant's
  live resource types (becomes _type), narrowing filters (_elements,
  _typeFilter, _since as day/week/4-weeks presets or a custom RFC 3339
  instant), and an optional job name.
- Kick-offs and status polls are self-calls addressed via the request's
  Host header, with the caller's own Authorization and tenant forwarded
  -- the export runs with the user's credentials, so the workspace works
  under auth without a service account. Relative Content-Location poll
  URLs resolve against the same base.
- /ui/bulk-export/active: one card per job (newest first) with status
  chips; in-progress cards htmx-refresh every 5s, each fetch performing
  exactly one server-side poll (202 records X-Progress, 200 stores the
  completion manifest's output files and stops the polling trigger).
  Complete cards list the files as download links with elapsed time;
  failed cards show the error verbatim with Retry (same parameters,
  fresh kick-off); Cancel DELETEs the job server-side.
- Job state lives in the per-user settings document under the reserved
  byTenant subtree, like the import workspace.
- The export form is parsed by hand (form_urlencoded): the repeated
  checkbox fields are more than serde_urlencoded accepts.

Tests mount the UI over a mock FHIR export backend on a real socket so
the Host-header self-calls loop back into the mock: kick-off query
shape (_type/_elements/_since), patient and group paths, the
202-progress -> 200-manifest polling lifecycle, rejected kick-off ->
failed card -> retry, and cancel reaching the API as a DELETE.
@angela-helios

Copy link
Copy Markdown
Contributor Author

Screenshots of the workspace working end-to-end against a live server built from this branch (sqlite, 73 seeded resources) — real $export kick-offs, not mocks.

Export builder — named job, three resource types checked, Since = last 7 days:

form

Active Exports, happy path — the job completed in 6s; the card stopped polling on its own and lists the completion manifest's file as a download link (only Patient had data among the checked types):

complete

Failure path — a Group-scope export against a group that doesn't exist: the card lands as Failed with the server's OperationOutcome verbatim and a Retry that re-runs the same parameters:

failed

Side observation while producing these, not this PR's concern: GET /Group/{missing}/$export answers 500 with a generic internal-error OperationOutcome rather than a 404 — worth a small consumer-side issue.

…lint

CI's newer toolchain flags the must_use attribute async-trait 0.1.89
emits in its expansion (clippy::double_must_use, promoted by
-D warnings) on every #[async_trait] trait, first surfacing in
helios-observability's DashboardProvider. 0.1.92's expansion is clean.
No source changes; the lock adds syn v3 alongside v2 as its dependency.
Same adjustment Import needed in #528: the Bulk Export workspace made
the sidebar's Export item a real link, so the chrome spec expects a
navLink instead of a soon-item. SQL-on-FHIR remains the section's only
placeholder.
The Export and Import page headers stacked their action buttons under
the title; page-head--row (the existing flex variant the resources
workspace uses) puts them top-right, matching the design. The giant
gap reported under the Export header turned out to be a Playwright
full-page stitching artifact with the fixed nav rail, not a layout
bug -- the screenshots are retaken at a tall viewport instead.
@angela-helios

Copy link
Copy Markdown
Contributor Author

Follow-up on a layout question from review: the large empty band under the header in the earlier screenshots was a Playwright full-page-capture stitching artifact (the fixed nav rail confuses the scroll-and-stitch), not real layout — live DOM shows the standard 20px. What was real: the header action buttons stacked under the title. Fixed by using the existing page-head--row flex variant on the four bulk workspace pages, matching the design(actions top-right). Screenshots above re-captured artifact-free at a tall viewport.

@codecov

codecov Bot commented Aug 13, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 92.42820% with 29 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
crates/ui/src/bulk_export.rs 92.20% 29 Missing ⚠️

📢 Thoughts on this report? Let us know!

@smunini

smunini commented Aug 14, 2026

Copy link
Copy Markdown
Contributor

Resource-type picker isn't actually FHIR-version aware

Reviewing where the export form's resource-type checkboxes come from, I hit a discrepancy worth deciding on. Note this is pre-existing behavior this PR inherited, not something introduced here — but the Bulk Export page is a new consumer of it, so it seems like the right moment to flag it.

The chain today

crates/ui/src/bulk_export.rs:265

let resource_types = state
    .compartments
    .resource_type_names(&rt.id, helios_fhir::FhirVersion::default())
    .await;

CompartmentCatalog::resource_type_names (crates/ui/src/compartments.rs:131) takes the first cached CompartmentDefinition (sorted by code, so Device) and maps resource[].code. Each definition enumerates the full type set for its version, so any one works as a registry.
CompartmentCatalog::definitions fetches GET /CompartmentDefinition from the server's own API over loopback, cached per (tenant, version).
→ Storage is seeded at startup from the version-specific bundles data/compartment-definitions-{r4,r4b,r5,r6}.json (seed_spec_compartment_definitions, crates/persistence/src/search/seeder.rs:274).

The discrepancy

The underlying data is version-specific, but two things flatten it before it reaches the page:

  1. The page passes FhirVersion::default() (R4) rather than the request version rv.0 — matching the three existing callers in lib.rs:973,995,1020 (Search, Queries, Resources). So the UI's version selector doesn't change the type list.
  2. HttpConformanceSource::fetch ignores its version argument entirely (_version, crates/ui/src/conformance.rs:63) and just issues GET {base}/CompartmentDefinition. The FhirVersion argument therefore functions only as a cache key.

Net effect at runtime: the list reflects whatever the server seeded — config.default_fhir_version (crates/hfs/src/main.rs:1012, i.e. HFS_FHIR_VERSION) — regardless of what the user picks in the version selector.

The bundles do differ meaningfully:

Bundle Resource types Compartments
R4 145 5
R4B 140 5
R5 157 5
R6 126 6 (adds Group)

Why it isn't visible in tests

StaticConformanceSource::fetch is keyed by (resource_type, version) (crates/ui/src/conformance.rs:203), so the test double honors the version the production HTTP source discards. A test can't currently surface the mismatch.

Options

  • Leave as-is — correct for a single-version deployment, which is the common case. If so, worth a comment at resource_type_names recording that the version argument is a cache key only, so the next reader doesn't assume it filters.
  • Make it real — pass rv.0 at the call sites and have HttpConformanceSource::fetch propagate the version to the self-call. That's a cross-cutting change touching all four callers, so probably its own issue rather than scope creep here.

No objection to merging on this — just want the behavior to be a decision rather than an accident.

@smunini smunini assigned angela-helios and unassigned smunini Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(ui): Bulk Export workspace — scoped kick-off, filters, active exports with progress

2 participants