feat: add centralized reciprocal-space reconstruction (GUI, CLI, cluster batch) - #69
Conversation
- add a C++17 reciprocal-space kernel for deterministic HKL and Q mapping with adaptive pixel and exposure-footprint subdivision - propagate pixel variances through weighted voxel accumulation while preserving source-pixel correlations and recording marginal-variance limits - add parallel image loading, correction, tiling, native mapping, and byte-bounded in-memory accumulation - implement immutable Parquet map partitions, bounded external reduction, resumable manifests, checksums, and one-write-per-chunk HDF5 finalization - freeze detector calibration, crystal, UB, scan references, corrections, exclusions, masks, backgrounds, and repair data from authoritative orGUI state - add exact and fallback exposure-angle bounds for supported scan backends and serializable references for regular, sliced, interlaced, manual, simulated, and external-backend scans - add a reusable native pixel-repair plan with interpolation variance propagation for constant detector masks - add the reciprocal-space reconstruction dialog with grid coverage, editable intervals, file-size estimates, geometry-matched Jacobian steps, HDF5 settings, performance controls, preview, run, and resume workflows - add shared global HDF5 chunk settings and an optional reconstruction compression override using the central filter registry - add orGUI rsmap and orGUI-rsmap run, resume, and status commands that consume UI-prepared job descriptors - write standalone NeXus-style HDF5 results with axes, intensity, variance, weight, contributors, configuration, build metadata, and provenance - register completed files as external orGUI results without copying large reconstructed arrays into the active database - add focused numerical, variance, geometry, correction, determinism, storage, CLI, dialog, resume, and native-kernel regression tests - add mapping, alignment, storage, and reducer benchmarks - document the complete reconstruction workflow, every user and job parameter, scientific equations, storage architecture, CLI usage, and practical tuning
Add a minimal CLAUDE.md that imports AGENTS.md so Claude Code picks up the existing guidance tree. Extend the AGENTS.md files to cover code that had grown outside their scope: the top-level reconstruction pipeline, orgui/backend/, and the xrayutils cpp/ native kernels, plus refresh stale file/test lists (reconstruction.py, ReconstructionDialog.py, test_reconstruction* suite). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
|
ToDo: before merge: settings for normalization in the integration tab should be moved to the RS map dialog. Other option: Provide a normalization API for the backend, (deterministic or with variance). This would break the existing output of all integrations. The raw output would then only be available through reconstruction of the normalized outputs. |
|
Normalization in the integration tab is moved to the RS map dialog. Normalization rework is now opened as issue #70 |
|
Note: A
Only commit messages changed. Every rewritten commit keeps an identical tree, This PR itself is unaffected: it stays merged, and its commits and diff (served |
1 participant
Summary
orgui/datautils/xrayutils/reconstruction.py+ C++ kernel) that maps detector images directly into HKL/Q grids without materializing a full 3D array in memory.ReconstructionDialog— a GUI workflow (Experiment / Output grids / Performance / Cluster / Job-and-output / Preview tabs) to configure, preview, prepare, run, and resume reconstruction jobs, plus a sharedHDF5SettingsDialogfor output chunking/compression.reconstruction_cluster.py) with parallel submap reduction, and areconstruction_cli.pyentry point for headless/scripted runs.doc/source/reciprocal_space_reconstruction.rst, wired into the docs toctree, plusdoc/AGENTS.mddescribing the changelog/doc-sync workflow and a new automated doc-sync GitHub Action.Docs / known gaps
QScanSelector's new "Normalize by exposure time" / "Monitor corrections" fields live in the shared options panel (used by all scan workflows) but are currently only consumed by the reconstruction pipeline, not documented anywhere, and not mentioned as reconstruction-only inimage_integration.rst. Recommend a follow-up doc note (or scoping the fields to the reconstruction dialog) before release.CHANGELOG.md [Unreleased]entry for this feature yet.