Skip to content

Share the NPZ object serialization of the Web API between DataLab and DataLab-Kernel #90

Closed

Description

@PierreRaybaut

Problem

The NPZ format of the DataLab Web API (in-memory archive of NumPy arrays and JSON metadata, used to transfer signals and images) is implemented twice, by copies that diverge:

  • DataLab, datalab/webapi/serialization.py (server side);
  • DataLab-Kernel, datalab_kernel/serialization_npz.py (client side).

Each fix must be done twice: non-uniform image coordinates were added to DataLab (DataLab-Platform/DataLab@34ad31c) and only later to the Kernel (DataLab-Platform/DataLab-Kernel#4); the Kernel still cannot transfer object metadata holding NumPy arrays nested in lists (e.g. ROI-related results), which DataLab handles.

Proposal

Both depend on Sigima, which defines the objects: provide the implementation once, in sigima.io (serialize_object_to_npz, deserialize_object_from_npz), taken from DataLab (the most complete): labels, units, uncertainties, uniform or non-uniform image coordinates, object metadata (NumPy arrays and scalars converted, geometry coords restored), checks of invalid archives, compatibility with archives written by earlier versions, no pickling. The format is documented next to the code.

DataLab and DataLab-Kernel then use it instead of their own copies.

Activity

  1. PierreRaybaut commented on Oct 25, 2026

    @PierreRaybaut
    ContributorAuthor

    Implemented by #91 (merged into develop). Adopted by DataLab-Platform/DataLab#430; DataLab-Platform/DataLab-Kernel#6 waits for Sigima 1.4.0 on PyPI.

  2. added this to the v1.4.0 milestone on Oct 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions