Skip to content

Compressed slc forward - #745

Open
mirzaees wants to merge 4 commits into
isce-framework:mainfrom
mirzaees:compressed_slc_forward
Open

mirzaees wants to merge 4 commits into
isce-framework:mainfrom
mirzaees:compressed_slc_forward

Conversation

@mirzaees

@mirzaees mirzaees commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator
  • stack.py: MiniStackPlanner.plan() now chronologically merges existing compressed SLCs (keyed by reference_date) with each ministack's real CSLCs instead of blindly prefix-concatenating them,
    and chunks the top-level real-SLC positions directly (fixing the same contiguity assumption in ministack_starts). Added is_real_mask/last_compressed_slc_idx properties; fixed
    real_slc_date_range's end-side bug.
  • _compress.py: compress() now excludes old compressed SLCs via a boolean is_real_mask instead of a first_real_slc_idx slice boundary.
  • single.py: all first_real_slc_idx-based slicing (amplitude stats, output-file writing/naming, the length assertion) replaced with mask-based selection.
  • _core.py: first_real_slc_idx renamed to last_compressed_slc_idx throughout (including the JIT static_argnames), used directly as the CRLB reference index — a parameter swap, not an addition,
    since it was only used at one call site.
  • Tests: 8 new tests added across test_stack.py (interleaved merge + referencing + date-range fix), test_phase_link_compress.py (mask-based exclusion), test_phase_link_core.py (CRLB with
    last_compressed_slc_idx), test_workflows_single.py (end-to-end interleaved ministack).

mirzaees and others added 3 commits October 5, 2026 07:45
MiniStackPlanner previously assumed all compressed SLCs form a
contiguous prefix of each ministack, with a scalar first_real_slc_idx
boundary standing in for both "count of compressed SLCs" and "index of
the last compressed SLC". That assumption breaks in forward-mode /
incremental-reprocessing: a compressed SLC's reference_date can be
chronologically later than some real CSLC dates in the same input
list, since opera_utils.sort_files_by_date has no compressed-first
guarantee.

- stack.py: chronologically merge compressed + real SLCs in
  MiniStackPlanner.plan() instead of blindly prefix-concatenating;
  chunk ministacks by real-SLC position rather than a contiguous
  range; add is_real_mask/last_compressed_slc_idx; fix
  real_slc_date_range's end-side assumption.
- _compress.py / single.py: replace first_real_slc_idx slice
  boundaries with boolean is_real_mask selection, correct regardless
  of where compressed SLCs land.
- _core.py: rename first_real_slc_idx to last_compressed_slc_idx
  (including the jit static_argnames), used directly as the CRLB
  reference index -- it was only consumed at that one call site.

Behavior is unchanged in the fully-contiguous case (regression tests
pass unmodified); new tests cover the interleaved case end-to-end.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
jax 0.9.2 added a FutureWarning when jax.scipy.linalg.solve is given
an identity matrix as the RHS to compute a full inverse, since it hits
an ambiguous batched-solve path internal to cho_solve. Both call sites
in crlb.py were only ever using solve() this way to get a matrix
inverse, so switch them to jnp.linalg.inv directly.

Also cap numpy<2.5 in the dependency specs: gdal_calc.py (used via
_average_or_rename) still relies on an in-place ndarray.shape
assignment that numpy 2.5 deprecated. The fix landed upstream
(OSGeo/gdal#15127) but hasn't shipped in a GDAL release yet, so pin
around it until it does.

Since this repo's filterwarnings = ["error"] promotes both warnings to
hard failures, they were taking down ~54 tests across the CRLB and
GDAL-averaging code paths.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@mirzaees

mirzaees commented Oct 8, 2026

Copy link
Copy Markdown
Collaborator Author

@scottstanie can you please review this PR?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant