Repository navigation
Keep non-uniform image coordinates in NPZ transfers and HDF5 files - #4
Merged
Merged
Conversation
Images with non-uniform coordinates lost them in DataLab-Kernel: the NPZ serializer used by the Web API backend wrote and read only x0, y0, dx and dy, and so did the HDF5 save/load of the standalone and Web API backends. For such images Sigima keeps the default x0 = y0 = 0 and dx = dy = 1, so they came back with the pixel indices as coordinates, silently. NPZ archives now follow the format of DataLab (>= 1.4.0): the is_uniform_coords flag and, for non-uniform images, the pixel center coordinates in xcoords.npy and ycoords.npy, whose lengths are checked against the image shape. HDF5 image groups store the same flag and the xcoords/ycoords datasets (h5coords module shared by both backends). Archives and files without the flag (earlier versions) are still read with uniform coordinates. The NPZ serializer also accesses the attributes guaranteed by Sigima objects directly instead of through getattr fallbacks, and the round-trip tests cover compression, labels, units and metadata. The live round-trip test is skipped with DataLab < 1.4.0, which does not transfer non-uniform coordinates. Closes #3 Assisted-by: Claude Opus 5.5
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3
Summary
Non-uniform image coordinates were lost by DataLab-Kernel, both in NPZ transfers with DataLab and in HDF5 workspace files: images came back with the pixel indices as coordinates.
serialization_npz.py): same contract as DataLab (DataLab-Platform/DataLab@34ad31c):is_uniform_coordsinmetadata.jsonand, for non-uniform images,xcoords.npy/ycoords.npy(pixel centers). Lengths are checked against the image shape; archives without the flag (earlier versions) are read with uniform coordinates. Module docstring (format) updated.backends/standalone.py,backends/webapi.py): new sharedbackends/h5coords.pystoringis_uniform_coordsand thexcoords/ycoordsdatasets; files without the flag are read with uniform coordinates.getattrfallbacks), and the existing round-trip tests cover compression, labels, units and metadata.Tests
maininstalls DataLab from PyPI). Run locally against DataLabdevelop: passed.main. The other failures of the standalone suite in my local environment (87, missing matplotlib/plotly) are identical onmain.Note (pre-existing, not changed here)
DataLab
developreadsSIGIMAX_DO_NOT_QUITinstead ofDATALAB_DO_NOT_QUIT: with DataLab ≥ 1.4.0,conftest.py --start-datalabsees DataLab exit immediately unlessSIGIMAX_DO_NOT_QUIT=1is also set.