Skip to content

Fix strict array-API test bodies in test_combiner/test_ccdproc/test_image_collection - #998

Merged
mwcraig merged 6 commits into
astropy:mainfrom
mwcraig:test-hygiene-strict-2
Aug 25, 2026
Merged

Fix strict array-API test bodies in test_combiner/test_ccdproc/test_image_collection#998
mwcraig merged 6 commits into
astropy:mainfrom
mwcraig:test-hygiene-strict-2

Conversation

@mwcraig

@mwcraig mwcraig commented Aug 25, 2026

Copy link
Copy Markdown
Member

Part of #971 (section 2: strict test-body fixes).

Strict count on top of main 6724c8e: 31 failed → 15 failed (469 passed/36 skipped/43 xfailed/5 xpassed → 483 passed/36 skipped/45 xfailed/5 xpassed). numpy, jax, and dask show no regressions; jax additionally improves (541 passed/7 xfailed → 548 passed/0 xfailed/0 xpassed — see the third commit).

Add numpy_copy/numpy_ccddata test helpers for strict device1 arrays. The strict suite runs on array-api-strict's non-default Device("device1"), where np.asarray() deliberately raises, but several tests need to hand data to astropy.io.fits, which requires NumPy. Adds numpy_copy() (move to the default device, then convert) and numpy_ccddata() (a CCDData with data/mask/uncertainty all converted) to pytest_fixtures.py for the write-path fixes below.

Fix strict-incompatible test bodies in test_combiner.py. _make_mean_scaler called ndarray.mean()/np.ma.average() directly; rewritten to use each argument's own array namespace, since the reference image can be plain NumPy (read from FITS) while the scaled array is a namespace array. Several tests called CCDData.multiply(), iterated over an array, or called xp.asarray() on a list of arrays — all of which either route through astropy's arithmetic wrapper (which calls np.result_type and breaks on strict) or are flatly disallowed by array-api-strict; replaced with in-namespace construction, xp.stack(), and explicit arr[i, ...] indexing. combine() only honors array_package for filename/string input and the Combiner reference built from CCDData.read() defaults to NumPy, so several tests now pass array_package=xp/Combiner(..., xp=xp) (or convert the CCDData list's .data by hand) so both sides of each comparison are namespace arrays. Writing a device1 CCDData via CCDData.write() — which ImageFileCollection round-trips and combine(output_file=...) both do — needs numpy_ccddata(); comparisons against results that land on the namespace's default device use numpy_copy() + assert_allclose() instead of xp.all(...).

Two of this bucket's targets still fail after these fixes, on causes outside test-code's reach: test_combiner_result_dtype's dtype=int case, because combine() passes the bare Python int straight to xp.astype/xp.asarray, which array-api-strict rejects (AttributeError: type object 'int' has no attribute '_np_dtype') — a new gap, not previously reachable because an earlier .multiply() failure masked it; and test_combine_overwrite_output, because combine(output_file=...) writes the still-in-namespace result via ccd.write() at combiner.py:1273 (AttributeError: 'Array' object has no attribute 'astype'), the same class of bug as #935. Both are left failing, unmarked, for the coordinator to track in #971. Three more tests that call default median_combine() (test_combiner_with_scaling, test_combiner_with_scaling_uncertainty[median_combine], test_writeable_after_combine[median_combine]) remain failing on #929's sigma_func/astropy.stats densification, unrelated to and untouched by this PR — test_writeable_after_combine[median_combine] was already known to the plan; the other two are additional #929 instances this PR's fixes exposed by clearing their first-listed cause.

Because these fixes remove every CCDData.multiply() call from the affected tests, the jax-only backend_xfail markers on test_combiner_with_scaling, test_combiner_result_dtype, test_combine_overwrite_output, and test_combiner_with_scaling_uncertainty (added for a jax DeprecationWarning triggered by astropy arithmetic receiving a jax array as dtype=) no longer apply and are removed; jax now XPASSes all of them cleanly.

Fix test_subtract_overscan_model for strict; xfail the astropy.modeling path. The test built its overscan gradient with integer/float division array-api-strict rejects, then unpacked the resulting 3D array by iteration, which array-api-strict also disallows for arrays with more than one dimension; fixed with an explicit float64 dtype/device and [0, ...]/[1, ...] indexing. Also replaced two ndarray.copy() calls with xp.asarray(..., copy=True). With those fixed, both parametrizations reach subtract_overscan(model=...), which goes through astropy.modeling's LinearLSQFitter — NumPy-only and unrelated to this test's array handling (#933); marked backend_xfail("array-api-strict", ...) citing #933. Exact stop: TypeError: object of type 'Array' has no len() from xp.arange(len(oscan)) in ccdproc/core.py's subtract_overscan.

Reset leaked _config_ccd_requires_unit before test_generator_ccds_without_unit. astropy's _arithmetic decorator sets the module global _config_ccd_requires_unit = False around an arithmetic call and restores it only on success, with no try/finally; when an earlier test's arithmetic raises (test_unit_mismatch_behaves_as_expected does this deliberately, on every backend), nothing resets the flag before later tests run (astropy/astropy#20268). This previously only reproduced on jax (and, after this PR's strict fixes, on array-api-strict too) and was marked backend_xfail for both — but removing several successful .multiply() calls in the test_combiner.py commit above also removed incidental resets of that global, which surfaced the same leak as a genuine regression on plain NumPy. Rather than add NumPy to the xfail list, the test now resets astropy.nddata.ccddata._config_ccd_requires_unit = True itself and drops the marker entirely — it passes deterministically on all four backends regardless of what ran before it.

Leftover failures after this PR (for #971 bookkeeping):

  • test_combiner_result_dtypecombine(dtype=int) vs. array-api-strict's dtype objects (new gap, undocumented pre-PR).
  • test_combine_overwrite_outputcombine(output_file=...) writes a namespace array via CCDData.write() (same class as Array API: decide a policy for inherently CPU-only operations #935).
  • test_combiner_with_scaling, test_combiner_with_scaling_uncertainty[median_combine], test_writeable_after_combine[median_combine]Array API: Combiner.sigma_clipping and the default sigma_func densify via astropy.stats #929 (median_combine's default sigma_func densifies via astropy.stats).
  • The remaining ~10 failures are the pre-existing, untouched upstream bucket (test_unit_mismatch_behaves_as_expected, test_sigma_func_for_ccddata, test_combiner_dtype, test_combiner_sigmaclip_*, test_combiner_median, test_combine_result_uncertainty_and_mask[median_combine-*], test_rebin_ccddata[True-True]).

🤖 Generated with Claude Code

https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME

mwcraig and others added 4 commits August 25, 2026 15:11
The strict suite runs on array-api-strict's non-default Device("device1"),
on which np.asarray() deliberately raises. Several tests need to hand data
to astropy.io.fits (which requires NumPy), so add numpy_copy() to move an
array to the default device before converting, and numpy_ccddata() to
build a CCDData whose data/mask/uncertainty are all NumPy copies. Used by
the test_combiner.py fixes in the next commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
Several test bodies in test_combiner.py used NumPy-only operations or
astropy arithmetic that break on array-api-strict:

- _make_mean_scaler used ndarray.mean()/np.ma.average() directly; rewrite
  it to look up each argument's own array namespace, since the reference
  image may be plain NumPy (read from FITS) while the scaled array is a
  namespace array, or vice versa.
- CCDData.multiply()/xp iteration/xp.asarray(list) calls in the scaling,
  dtype, and overwrite-output tests go through astropy's arithmetic
  wrapper, which calls np.result_type on the operands and breaks on
  strict; replace them with in-namespace construction (CCDData(data *
  factor, ...)), xp.stack() instead of xp.asarray() on a list of arrays,
  and explicit ellipsis indexing (arr[i, ...]) instead of arr[i] on an
  array-api-strict array (bare single-axis indexing on a >1D array is not
  part of the Array API and array-api-strict rejects it).
- combine() only honors array_package for filename/string input, and the
  Combiner reference built from CCDData.read() defaults to NumPy, so pass
  array_package=xp to combine() and xp=xp to Combiner() (or convert the
  CCDData list's .data by hand when combine() ignores array_package for
  CCDData input) so both sides of each comparison are namespace arrays.
- Writing a device1 CCDData through CCDData.write() (as ImageFileCollection
  round-trips and combine(output_file=...) both do) fails because
  astropy.io.fits needs NumPy; write numpy_ccddata(ccd) instead, and
  compare results that land on the namespace's default device (rather than
  device1) via numpy_copy() + assert_allclose() instead of xp.all(...).

test_combiner_result_dtype's dtype=int case and test_combine_overwrite_output
both still fail strict after these fixes, on new causes not covered by this
PR: combine(dtype=int) passes the bare Python type straight to
xp.astype/xp.asarray, which array-api-strict rejects (it needs a real dtype
object), and combine(output_file=...) writes the still-in-namespace result
via CCDData.write() at combiner.py:1273, which needs the same NumPy-copy
treatment applied to combine()'s internals rather than to test code. Both
are left failing, unmarked, for the coordinator to track. Three tests that
call default median_combine() (test_combiner_with_scaling,
test_combiner_with_scaling_uncertainty[median_combine],
test_writeable_after_combine[median_combine]) also remain failing on
astropy#929's median_combine/sigma_func/astropy.stats densification, which this
PR does not touch.

Since these fixes eliminate every CCDData.multiply() call in the affected
tests, the jax-only backend_xfail markers on test_combiner_with_scaling,
test_combiner_result_dtype, test_combine_overwrite_output, and
test_combiner_with_scaling_uncertainty (added because astropy nddata
arithmetic passes a jax array as dtype=, triggering a jax
DeprecationWarning that this suite treats as an error) no longer apply and
are removed; the jax run now XPASSes cleanly on all of them.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
test_subtract_overscan_model built its synthetic overscan gradient with
integer/float division that array-api-strict rejects
(np_mgrid(...)/10.0 + 300.0 on an integer array), then unpacked the
resulting 3D array by iteration (yscan, xscan = ...), which
array-api-strict also disallows for arrays with more than one dimension.
Build the gradient with an explicit float64 dtype and device, and index
the two planes out with [0, ...]/[1, ...] instead of unpacking. Also
replace two ndarray.copy() calls (not part of the Array API) with
xp.asarray(..., copy=True).

With those fixed, the test reaches subtract_overscan(model=...), which
goes through astropy.modeling's LinearLSQFitter -- NumPy-only, and
unrelated to this test's own array handling (astropy#933). Mark both
parametrizations backend_xfail("array-api-strict", ...) citing astropy#933; the
exact stop is `TypeError: object of type 'Array' has no len()` from
`xp.arange(len(oscan))` in ccdproc/core.py's subtract_overscan.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
…hout_unit

astropy's _arithmetic decorator (astropy/nddata/ccddata.py) sets the
module global _config_ccd_requires_unit = False around an arithmetic call
and only restores it to True on success, with no try/finally. When an
earlier test's arithmetic call raises (test_unit_mismatch_behaves_as_expected
does this deliberately, on every backend), nothing resets the flag before
later tests run, so ccds() stops requiring a unit for the rest of the
process -- causing test_generator_ccds_without_unit's `with
pytest.raises(ValueError)` to fail with "DID NOT RAISE" (astropy/astropy#20268).

This previously only reproduced on jax and, per this PR's strict fixes,
array-api-strict, and was marked backend_xfail for both. But the fix in
the previous test_combiner.py commit removes several successful
CCDData.multiply() calls that happened to run between the leaking test and
this one and reset the flag back to True as a side effect on the default
NumPy backend, which surfaced the same leak as a genuine regression on
plain NumPy too. Rather than add a third backend to the xfail list (or
depend on incidental arithmetic calls elsewhere in the suite to reset
global state), reset astropy.nddata.ccddata._config_ccd_requires_unit to
True explicitly at the top of the test and drop the xfail marker entirely
-- the test now passes cleanly and deterministically on all four backends
regardless of what ran before it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
@codecov

codecov Bot commented Aug 25, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 97.76%. Comparing base (f61e08e) to head (2d34b6e).
⚠️ Report is 16 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #998      +/-   ##
==========================================
+ Coverage   97.75%   97.76%   +0.01%     
==========================================
  Files           9        9              
  Lines        1780     1788       +8     
==========================================
+ Hits         1740     1748       +8     
  Misses         40       40              
Flag Coverage Δ
dask 96.91% <ø> (+0.01%) ⬆️
jax 97.13% <ø> (+0.18%) ⬆️
numpy 97.65% <ø> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment thread ccdproc/tests/pytest_fixtures.py Outdated
Comment thread ccdproc/tests/pytest_fixtures.py
Comment thread ccdproc/tests/pytest_fixtures.py Outdated
Comment thread ccdproc/tests/test_combiner.py Outdated
mwcraig and others added 2 commits August 25, 2026 16:14
- Give the numpy_copy and numpy_ccddata test helpers proper numpy-style
  docstrings (Parameters / Returns / Notes).
- test_combine_average_ccddata converts the big-endian FITS data with
  ccdproc.core._native_numpy, which combine() and ImageFileCollection
  already use for this, instead of a hand-rolled .astype(float).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
Move the array to its namespace's own default device, found through the
standard __array_namespace_info__().default_device(), instead of naming
array-api-strict and its CPU_DEVICE. The 2025.12 standard allows
default_device() to be None (JAX), in which case there is nothing to
move to and np.asarray is used directly. No backend is named, so a new
one is handled automatically.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01S36ZzAAVXVm32vuTdtCQME
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant