Skip to content

Variant level CSV download - #706

Merged
bencap merged 4 commits into
release-2026.2.4.1from
feature/bencap/variant-level-csv-download
Aug 12, 2026
Merged

Variant level CSV download#706
bencap merged 4 commits into
release-2026.2.4.1from
feature/bencap/variant-level-csv-download

Conversation

@bencap

@bencap bencap commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Adds a variant-level CSV download and replaces the score-set custom-download dialog with a shared column picker driven by the API's namespace-discovery endpoints. Requires the API-side GET /score-sets/{urn}/csv-namespaces, GET /variants/{urn}/csv-namespaces, and GET /variants/{urn}/csv. See VariantEffect/mavedb-api#804.

CSV column picker

  • New use-csv-namespaces.ts composable and MvCsvColumnDialog.vue fetch a score set's or variant's available column namespaces and let the user choose which to include.
  • Sections are grouped and labeled by the server (Measurements, Annotations, Clinical interpretation, Provenance) rather than maintained client-side, and are split by owning score set when a variant's calibrations span more than one.
  • Initial selection follows the API's selectedByDefault flag, so a research-use-only or rangeless calibration is offered but must be opted into explicitly.
  • Replaces the old hand-picked checkbox list (scores / counts / mappedHgvs / etc.), whose query params the API never actually read. Used in MvVariantPreview.vue's "Custom Data" dialog, ScoreSetDownloads.vue, and VariantScreen.vue.

Variant-level download

  • VariantScreen.vue gains a "Download variant CSV" control backed by GET /variants/{urn}/csv, kept separate from the existing VA-Spec annotation downloads.
  • use-variant-lookup.ts gains downloadVariantCsvFile and a shared downloadInProgressLabel.

Annotation-stream memory fix

  • streamAnnotationsInto accumulated NDJSON chunks as decoded strings, then joined them, then built a Blob from the result — holding roughly five copies of the payload at once, enough to run the tab out of memory on a large pathogenicity-statement download. It now accumulates raw Uint8Array chunks, builds the Blob directly from them, and counts newline bytes for progress instead of decoding.
  • A stream that delivers fewer lines than X-Total-Count now throws rather than silently saving a partial file.

Shared download indicator

  • Per-feature annotatedDownloadInProgress / Progress state is replaced by a single fileDownloadLabel / fileDownloadProgress pair covering every download the composable offers (scores, counts, mapped variants, custom data, VA-Spec streams), via a withIndicator() wrapper that also blocks concurrent downloads.
  • ScoreSetDownloads.vue renders one progress bar under the button row instead of one hung off the annotated-variants split button, and disables every download button while any download is in flight. Progress is indeterminate except for the VA-Spec streams, which count records against X-Total-Count.
  • reportingFailure() surfaces a rejected download as a toast instead of an unhandled promise rejection.

Supporting changes

  • getErrorResponse moves out of api/mavedb/index.ts into a new src/lib/errors.ts, joined by describeRequestError, which replaces the ad hoc extractErrorDetail in ScoreSetCalibrationsView.vue. All call sites now import from @/lib/errors.
  • score-sets.ts and variants.ts switch their CSV requests from drop_na_columns and include_post_mapped_hgvs to drop_unused_hgvs_columns plus the namespace list, matching the API's parameter names.
  • openapi.d.ts regenerated for the AvailableCsvNamespace and CsvNamespaceGroup schemas and the new endpoints.

Tests

Unit tests added for use-csv-namespaces, use-score-set-downloads, and lib/errors.

@coveralls

coveralls commented Aug 6, 2026

Copy link
Copy Markdown

Coverage Status

coverage: 9.654% (+3.0%) from 6.656% — feature/bencap/variant-level-csv-download into release-2026.2.4.1

@bencap
bencap changed the base branch from release-2026.2.5 to release-2026.2.4.1 August 10, 2026 19:21
@bencap bencap changed the title Feature/bencap/variant level csv download Variant level CSV download Aug 10, 2026
@bencap
bencap marked this pull request as ready for review August 10, 2026 19:27
@bencap
bencap force-pushed the feature/bencap/variant-level-csv-download branch from 51884a9 to 2c364df Compare August 10, 2026 19:29
bencap added 3 commits August 11, 2026 09:27
- Extract `getErrorResponse` out of `api/mavedb/index.ts` and add
  `describeRequestError` alongside it in the new `src/lib/errors.ts`,
  replacing the ad hoc `extractErrorDetail` in ScoreSetCalibrationsView
- Update all call sites to import from `@/lib/errors` instead of
  `@/api/mavedb`
- Add unit tests covering both exported functions
Add use-csv-namespaces.ts and MvCsvColumnDialog.vue: a composable and
dialog that fetch a score set's or variant's available CSV column
namespaces from the new discovery endpoints and let a user pick which
to include, replacing the score-set custom-download dialog's
hand-picked checkbox list (scores/counts/mappedHgvs/etc.) that mapped
to query params the API never actually read.

Sections are grouped and labeled by the server (Measurements,
Annotations, Clinical interpretation, Provenance) rather than
hand-maintained client-side, and split by owning score set when a
variant's calibrations span more than one. Defaults come from the
API's selectedByDefault flag, so a research-use-only or rangeless
calibration is offered but opts in explicitly.

Wire it up in two places:
- MvVariantPreview.vue's "Custom Data" dialog now uses it for score-set
  downloads instead of the old checkbox dialog
- VariantScreen.vue gets a new "Download variant CSV" control backed by
  the variant-level CSV endpoint, split from the existing VA-Spec
  annotation downloads since they're a different kind of artifact (flat
  table vs. nested standard objects); use-variant-lookup.ts gains
  downloadVariantCsvFile and a shared downloadInProgressLabel

Update openapi.d.ts for the new AvailableCsvNamespace /
CsvNamespaceGroup schemas and endpoints, and switch score-sets.ts /
variants.ts's CSV requests from drop_na_columns and
include_post_mapped_hgvs to drop_unused_hgvs_columns and the namespace
list, matching the API's new parameter names.
Fix streamAnnotationsInto (use-score-set-downloads.ts) accumulating
NDJSON chunks as decoded strings: decoding to UTF-16, then joining, then
building a Blob from the result meant a large pathogenicity-statement
download held roughly five copies of the payload in memory at once and
could run the tab out of it. Accumulate raw Uint8Array chunks instead
and build the Blob directly from them, and count newline bytes rather
than decoding to track progress. Also make a truncated stream (fewer
lines than X-Total-Count) throw instead of silently saving a partial
file, since the response has already started by the time that could be
detected.

Replace the per-feature annotatedDownloadInProgress/Progress state with
a single fileDownloadLabel/fileDownloadProgress pair shared by every
download this composable offers (scores, counts, mapped variants,
custom data, and the VA-Spec streams), via a withIndicator() wrapper
that also prevents two downloads running concurrently. Progress is
indeterminate except for the VA-Spec streams, which can count records
against X-Total-Count.

Wire the shared indicator into ScoreSetDownloads.vue as one progress
bar under the button row instead of one hung off the annotated-variants
split button, disable every download button while any download is in
flight, and add reportingFailure() so a rejected download surfaces a
toast instead of an unhandled promise rejection in the console. Switch
the custom-data dialog to MvCsvColumnDialog, matching MvVariantPreview
and VariantScreen.
@bencap
bencap force-pushed the feature/bencap/variant-level-csv-download branch from 2c364df to 9277f0f Compare August 11, 2026 16:27
@bencap
bencap merged commit 6d80c90 into release-2026.2.4.1 Aug 12, 2026
1 check passed
@bencap bencap mentioned this pull request Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

refactor: extract score set CSV generation into a dedicated, tested module

2 participants