[PierreRaybaut]

Import of multidimensional data, legacy NeXus scans (SOLEIL) and compressed detector data in the HDF5 layer and browser.

Multidimensional data

• Datasets and NeXus signals of three dimensions or more are imported through a view, one signal or image per frame (sigimax.h5.views, NDView, default_view).
• The HDF5 browser shows a view selector for such items (displayed axes, frames along an axis with first/last/step, fixed indices). The first frame is previewed, and the frames are imported once "Import these frames" is checked. A confirmation is asked beyond 100 frames.
• The default view follows the NeXus interpretation attribute (spectrum, rgba-image), read on the signal dataset or on its group.
• Frames of NeXus signals keep the axes of their displayed dimensions. Their title and experimental_source give the frame indices (view) and positions (positions).
• expand_views replaces a node by its frame nodes; H5TreeWidget.get_nodes applies it, so DataLab's import loop is unchanged.

NeXus conventions

• Legacy axes: without @axes, the axis attribute (dimension number, from 1) of axis datasets gives the axis of each dimension, preferring the lowest primary. An axis applies only to the dimensions whose size fits. axis=0, used by SOLEIL for time stamps, is ignored.
• Auxiliary signals (@auxiliary_signals, and the other datasets with a legacy signal attribute, as all the counters and detectors of a SOLEIL scan) are shown as nodes of their own (AuxiliarySignalNode), with the axes that fit their dimensions.
• The browser selects and previews the default NXdata group (@default chain, default_nxdata_path).
• The title, start and end times of the NXentry group are recorded in experimental_source (details.entry_title, timestamps).
• Data mappings: axes may describe the first dimensions only, so that one pattern applies to the counters, spectra and images of a scan.

Compressed data

• hdf5plugin becomes a dependency, imported by sigimax.h5, so that bitshuffle/LZ4, Blosc, Zstandard... are read.
• Datasets compressed with a filter that is still unavailable are reported in the browser (unavailable_filters) instead of failing, including when the tree is built.
• Checked for DataLab's standalone build: the hdf5plugin hook of pyinstaller-hooks-contrib collects the filters, and a frozen program reads bitshuffle/LZ4 data.

Validation

• Tests: test_h5_views.py (views, legacy axes, auxiliary signals, frames, entry information, @default, filters, mapping axes), test_h5browser_views.py (view selector GUI). Full suite: 324 passed. Ruff clean, Pylint 10/10 with the project options.
• Checked on real files, which are not added to the repository (no license): SOLEIL 2008 (SWING, time scan), SOLEIL CRISTAL 2018 (12 Mythen detectors versus the goniometer angle) and Diffabs (counters versus the motor, XPAD stack of 81 frames), SLS STXM stack (4 frames, one per energy).

Documentation

User guide (hdf5_workspace.rst), release notes 1.1.0, French translations of the view selector.

Related

• HDF5 browser: multidimensional views and legacy NeXus scans (tests and docs) DataLab#444: application test, HDF5 browser documentation and release notes.
• Read real SPEC and XDI files as reference readers do, with validation corpora datalab-synchrotron#3: SOLEIL time-scan mapping using mapping axes on the first dimensions.