Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 14 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,9 +35,9 @@ removals. A JOSS paper + citation remain a post-2.0 follow-up.
| --- | --- |
| `config.py` | `CameraConfig` (frozen dataclass of detector params) and `SensorType` enum. Pure data + validation + temperature scaling. No randomness. |
| `noise.py` | The physics. Pure functions: `CameraConfig` + exposure + temperature + seeded `Generator` → electrons/ADU. This is where noise models and the opt-in reusable `DetectorWorkspace` live. |
| `backend.py` | Optional NumPy/CuPy array and RNG boundary, explicit host conversion, and backend convolution. NumPy is the reference/default. Kept local rather than `aocore.Backend`: getframes draws per-frame noise from a device-native (cuRAND) stream, which `aocore.Backend.random` (host-side `Generator`) does not provide. |
| `backend.py` | Optional NumPy/CuPy array and RNG boundary, explicit host conversion, and backend convolution. NumPy is the reference/default. Owns the stack's `device` vocabulary (`"cpu"`, `"gpu"`, `"gpu:N"`, `"auto"`; parsed by `_parse_device`, resolved by `get_backend`) and `precision` vocabulary (`resolve_precision`: `"single"`/`"double"`, aliases `"float32"`/`"float64"`). A GPU `ArrayBackend` carries a `device_id`; `activate()` makes it current. Kept local rather than `aocore.Backend`: getframes draws per-frame noise from a device-native (cuRAND) stream, which `aocore.Backend.random` (host-side `Generator`) does not provide. |
| `frame.py` | `Frame` container: a NumPy array (ADU) plus metadata; array-like; optional FITS export. |
| `camera.py` | `Camera`, the main user-facing object. Orchestrates config + scene + noise into `Frame`s. Holds the RNG and high-level methods (`dark_frame`, `dark_series`, reset-correlated `nondestructive_series`, `correlated_double_sample`, `expose`, `observe`, `*_series`, `master_*`). Reset-correlated readout has one core, the private `_ramp_reads`, which walks a ramp on an arbitrary per-read interval pattern; `nondestructive_series` drives it uniformly and `correlated_double_sample` drives it as pedestal-then-signal. Put new ramp-readout modes there rather than in a second loop — the interval-scaled bias/settling/avalanche terms are easy to get subtly wrong twice. |
| `camera.py` | `Camera`, the main user-facing object. Orchestrates config + scene + noise into `Frame`s. Holds the RNG and high-level methods (`dark_frame`, `dark_series`, reset-correlated `nondestructive_series`, `correlated_double_sample`, `expose`, `observe`, `*_series`, `master_*`). Reset-correlated readout has one core, the private `_ramp_reads`, which walks a ramp on an arbitrary per-read interval pattern; `nondestructive_series` drives it uniformly and `correlated_double_sample` drives it as pedestal-then-signal. Put new ramp-readout modes there rather than in a second loop — the interval-scaled bias/settling/avalanche terms are easy to get subtly wrong twice. Every public method that touches device arrays is decorated with `backend._on_device`, which runs it (each step, for generator methods) with the camera's CUDA device current; decorate new ones too, or `device="gpu:N"` silently allocates on the wrong card. |
| `calibrate.py` | Master-frame builders (`combine`) and `calibrate` reduction — the raw → reduced → truth loop (phase 1.1). |
| `observation.py` | `Observation` / `ObservationTruth` / `Pointing`: the time-series driver, jitter/drift/dither, per-frame truth (phase 1.2). |
| `spectral.py` | Opt-in spectral mode: `QE`, `SED` (relative or absolute via `from_flux_density`), `Spectrum`, `SpectralBandpass`, effective-QE folding, transmission-product helpers (`product`, `from_file`/`from_product`), optional `astropy.units` coercion. |
Expand Down Expand Up @@ -109,9 +109,18 @@ indices and takes a background/threshold/window — a different contract from
offset-centred `_radial_grid` in `analysis/apertures.py`, and `backend.py` (see
the Architecture table). A bug in an aocore primitive is fixed in aocore, never
worked around here. `tests/test_conformance.py` runs the `aocore.conformance`
checks that apply to a detector package (image-plane centring, unit flux); the
OPD-driven ones (tilt, slopes, wind, Zernikes, RMS) do not apply because
getframes PSFs are analytic, not images of an OPD map.
image-builder checks that apply to a detector package, for every PSF model:
`check_point_source_centring` (also on the vignetting map),
`check_point_source_flux`, and `check_edge_flux_loss` (light off a detector edge
is lost, not renormalized). They need `aocore>=0.1.3`, pinned in the `dev`
extra only; the runtime pin stays `>=0.1.2`. The OPD-driven checks (tilt,
slopes, wind, Zernikes, RMS) do not apply because getframes PSFs are analytic,
not images of an OPD map. It also pins the section 8
software vocabulary: every `device=` takes `"cpu"`/`"gpu"`/`"gpu:N"`/`"auto"`
and every working-precision argument takes `precision="single"|"double"`
(`Camera`, `Scene.photon_rate_map`/`photoelectron_rate_map`, and the `noise`
functions with a `float_dtype`). The dataset `dtype` is a host *storage* type,
not a working precision, so it stays a NumPy dtype.

## Adding a camera preset

Expand Down
57 changes: 56 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,60 @@ to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [2.4.0] - 2026-10-07

### Added

- **`device="gpu:N"` and `device="auto"`.** Every `device` argument (`Camera`,
`get_backend`, and the CLI's `[camera]` table) now speaks the AO stack's
vocabulary (aocore CONVENTIONS 8.1): `"cpu"`, `"gpu"`, `"gpu:N"` for CUDA
device `N`, and `"auto"`, which picks the GPU when CuPy is installed and sees
a device and the CPU otherwise. A `"gpu:N"` beyond the devices CuPy sees
raises a `ValueError` naming the count. The old spellings (`"numpy"`,
`"cuda"`, `"cupy"`) still work, now case-insensitively. A GPU camera is
pinned to its card: the fixed-pattern maps and the cuRAND streams are created
on it and every camera method runs with it current, so a `"gpu:1"` camera
works whichever device is current at the call, and `with_config` keeps it.
New: `Camera.device_id`, `ArrayBackend.device_id`, `ArrayBackend.spec`
(`"cpu"` or `"gpu:N"`) and `ArrayBackend.activate()` (the device context,
for calling the low-level `noise` functions on another card).
- **`precision="single"` / `"double"`.** The working precision takes the
shared names (aocore CONVENTIONS 8.2), with `"float32"`/`"float64"` kept as
aliases: on `Camera`, as a new `precision` keyword on
`Scene.photon_rate_map`/`photoelectron_rate_map` (beside `dtype`), and on the
`noise` functions that take a `float_dtype` (`simulate_frame`,
`fixed_pattern_maps`, `dark_signal_map`, `photo_signal_map`). A `dtype` and a
`precision` that disagree raise `ValueError`. `getframes.resolve_precision`
maps any of these names to the NumPy dtype. `Camera.precision` still reports
the dtype name (`"float32"`/`"float64"`) whichever spelling was passed.
`dataset.pairs(dtype=...)` is unchanged: it is the host *storage* type of the
finished arrays, not a working precision.
- The CLI's `[camera]` table takes a `device` key.
- **Conformance tests** for the device and precision vocabulary, including that
each precision name selects the same dtype as in aocore.
- **Edge-flux conformance for every PSF model.** `tests/test_conformance.py`
now uses aocore 0.1.3's image-builder checks (`check_point_source_centring`,
`check_point_source_flux`) instead of feeding analytic PSFs through the
OPD-driven checks with a dummy OPD, and adds `check_edge_flux_loss` for
Gaussian, Moffat, elliptical Gaussian, Airy and array PSFs, guarding the
2.3.0 fix. The `dev` extra pins `aocore>=0.1.3,<0.2`; the runtime
requirement is unchanged.

### Changed

- An unknown `device` string now raises `ValueError` listing the accepted words
(`'cpu', 'gpu', 'gpu:N' ... or 'auto'`), and a non-string `device` a
`TypeError`. `device="gpu"` with CuPy installed but no CUDA device raises
`RuntimeError` at construction rather than failing at the first frame.
- `Camera.__repr__` shows the GPU number (`device='gpu:0'`).
- The noise functions' `float_dtype` default is now `None` (still float64).

### Fixed

- `dataset.pairs` with a GPU camera failed on an implicit CuPy-to-NumPy
conversion; it now copies each frame to the host through `to_numpy`, as do
the CLI's `.npy`/`.npz` writers now that the CLI can select a GPU.

## [2.3.0] - 2026-10-07

### Fixed
Expand Down Expand Up @@ -592,7 +646,8 @@ together in 1.0.
- Documentation, runnable examples, and CI (lint, type-check, test matrix, PyPI
release via Trusted Publishing).

[Unreleased]: https://github.com/jacotay7/getframes/compare/2.3.0...HEAD
[Unreleased]: https://github.com/jacotay7/getframes/compare/2.4.0...HEAD
[2.4.0]: https://github.com/jacotay7/getframes/compare/2.3.0...2.4.0
[2.3.0]: https://github.com/jacotay7/getframes/compare/2.2.0...2.3.0
[2.2.0]: https://github.com/jacotay7/getframes/compare/2.1.1...2.2.0
[2.1.1]: https://github.com/jacotay7/getframes/compare/2.1.0...2.1.1
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ frame = cam.with_config(resolution=(256, 256)).observe(scene, exposure=300.0, se

import cupy as cp # and the same path on a GPU

cam = gf.Camera.from_preset("andor_ocam2k", device="gpu", precision="float32")
cam = gf.Camera.from_preset("andor_ocam2k", device="gpu", precision="single")
rate = cp.full(cam.resolution, 2.0e6, dtype=cp.float32) # photons/s/pixel
frame = cam.expose(rate, exposure=1.0e-3, seed=0) # CuPy ADU, no host copy
```
Expand Down Expand Up @@ -140,8 +140,8 @@ for the methodology.
- **Scale & datasets** — a float32 fast path, vectorised multi-source rendering,
a streaming raw+truth `dataset` generator and a `getframes` CLI; see
**[Scale & datasets](https://jacotay7.github.io/getframes/guides/datasets/)**.
- **GPU-optional** — every camera takes `device="gpu"` (CuPy) and keeps the
detector path and truth arrays device-resident. CPU and GPU have independent
- **GPU-optional** — every camera takes `device="gpu"`, `"gpu:N"` or `"auto"`
(CuPy) and keeps the detector path and truth arrays device-resident. CPU and GPU have independent
RNG streams, so a `seed` repeats exactly on a fixed backend while parity across
backends means matching statistics, not identical pixels.
- **Reproducible and typed** — all randomness flows through a camera-owned seeded
Expand Down
28 changes: 19 additions & 9 deletions docs/guides/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,15 +8,17 @@ unchanged and remains the default.

## The float32 fast path

Pass `precision="float32"` when you build a `Camera` to run the whole signal chain
Pass `precision="single"` when you build a `Camera` to run the whole signal chain
— and each frame's ground truth — in single precision. That halves the memory of
the per-pixel arrays, which matters for large detectors and when you are generating
thousands of frames:
thousands of frames. `"single"`/`"double"` is the vocabulary shared across the AO
stack; `"float32"`/`"float64"` are accepted aliases, and `cam.precision` reports
the NumPy dtype name (`"float32"`) whichever spelling you passed:

```python
import getframes as gf

cam = gf.Camera.from_preset("zwo_asi2600mm", precision="float32")
cam = gf.Camera.from_preset("zwo_asi2600mm", precision="single")
frame = cam.expose(photon_rate=200.0, exposure=30.0, seed=0)

frame.dtype # uint32 — the digitised ADU stay exact integers
Expand All @@ -28,13 +30,20 @@ way. Persistent PRNU/DSNU, amplifier gain/offset, structured-bias, and per-pixel
read-noise maps use the selected precision too, so a `float32` camera does not keep
hidden double-precision detector-sized coefficients. The result matches the
`float64` path statistically and to single-precision tolerance for deterministic
maps. If you call the scene or noise layers directly, the same control is a `dtype`
/ `float_dtype` argument:
maps. If you call the scene or noise layers directly, the same control is a
`precision` keyword, or the older `dtype` / `float_dtype` argument (give one, or
both only when they agree):

```python
rate_map = scene.photon_rate_map(dtype="float32") # f32 photons/s/pixel map
rate_map = scene.photon_rate_map(precision="single") # f32 photons/s/pixel map
rate_map = scene.photon_rate_map(dtype="float32") # the same map
```

The `dtype` of [`pairs`][getframes.dataset.pairs] is different: it is the
*storage* type the finished `raw`/`truth` arrays are cast to on the host, not a
working precision, so it stays a NumPy dtype (the camera's `precision` sets how
they are computed).

## Vectorised catalog rendering

A [`Catalog`][getframes.scene.sources.Catalog] of many stars no longer loops in
Expand Down Expand Up @@ -73,7 +82,7 @@ re-iterable source of random fields:
```python
import getframes as gf

cam = gf.Camera.from_preset("zwo_asi2600mm", precision="float32")
cam = gf.Camera.from_preset("zwo_asi2600mm", precision="single")
scenes = gf.dataset.random_star_fields(n=10_000, shape=cam.resolution, seed=0)

ds = gf.dataset.pairs(camera=cam, scenes=scenes, exposure=60.0, dtype="float32", seed=1)
Expand Down Expand Up @@ -102,7 +111,8 @@ A `generate` config names a preset (or an inline camera) and a frame spec:
[camera]
preset = "andor_ikon_m934"
default_temperature_c = -60.0
precision = "float32"
precision = "single" # or "double"; "float32"/"float64" also accepted
device = "auto" # cpu | gpu | gpu:N | auto (GPU when CuPy sees one)

[frame]
type = "dark" # dark | bias | flat | light
Expand All @@ -117,7 +127,7 @@ requested `shape`:
```toml
[camera]
preset = "zwo_asi2600mm"
precision = "float32"
precision = "single"

[dataset]
n = 1000
Expand Down
44 changes: 43 additions & 1 deletion docs/guides/gpu.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import getframes as gf
camera = gf.Camera.from_preset(
"andor_ocam2k",
device="gpu",
precision="float32",
precision="single",
)
photon_rate = cp.full(camera.resolution, 2.0e6, dtype=cp.float32)
frame = camera.expose(photon_rate, exposure=1.0e-3, seed=0)
Expand All @@ -33,6 +33,48 @@ binning, truth, and ADU digitisation. Wavelength-resolved
truth on device. Static fixed-pattern maps are constructed in the camera's working
precision and cached once, so reuse the same `Camera` in a frame loop.

## Choosing a device

`device` takes the words shared across the AO stack:

| `device` | Runs on |
| --- | --- |
| `"cpu"` (default) | NumPy, the reference implementation |
| `"gpu"` | CuPy on the CUDA device current when the camera is built |
| `"gpu:N"` | CuPy on CUDA device `N` |
| `"auto"` | the current CUDA device when CuPy is installed and sees one, else the CPU |

Matching is case-insensitive, and the older spellings still work: `"numpy"` for
`"cpu"`, `"cuda"`/`"cupy"` for `"gpu"` (also `"cuda:N"`). An unknown word, or a
`"gpu:N"` beyond the devices CuPy sees, raises `ValueError` naming the device
count; `"gpu"` without CuPy raises `ImportError`. `"auto"` never raises for a
missing GPU, so one script runs on a laptop and a CUDA workstation alike:

```python
camera = gf.Camera.from_preset("andor_ocam2k", device="auto", precision="single")
camera.device # "gpu" or "cpu"
camera.device_id # CUDA device number, or None on the CPU
```

A GPU camera is pinned to one card. Its fixed-pattern maps and its cuRAND
streams are created on that device, and every camera method runs with it
current, so `device="gpu:1"` works whichever device is current when you call
it; `Camera.with_config` keeps the same card. The low-level
[`getframes.noise`](../reference.md) functions run on the current device; to use
them on another card, enter the backend's device context:

```python
backend = gf.get_backend("gpu:1")
with backend.activate():
... # noise.simulate_frame(..., backend=backend)
```

`precision` is `"single"` (float32) or `"double"` (float64, the default), with
`"float32"`/`"float64"` accepted as aliases; see
[Scale & datasets](datasets.md#the-float32-fast-path).

## Host transfers

`frame.data` is the zero-copy device interface. `np.asarray(frame)`,
`Frame.stats()`, and `Frame.to_fits()` are explicit host-facing operations and
copy GPU data. Use `getframes.to_numpy()` when a named host boundary is clearer.
Expand Down
2 changes: 2 additions & 0 deletions docs/reference.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@

::: getframes.backend.get_backend

::: getframes.backend.resolve_precision

::: getframes.backend.get_array_module

::: getframes.backend.to_numpy
Expand Down
2 changes: 2 additions & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ dependencies = [

[project.optional-dependencies]
dev = [
# The test suite runs aocore's image-builder conformance checks (0.1.3+).
"aocore>=0.1.3,<0.2",
"pytest>=7.0",
"pytest-cov>=4.0",
"ruff>=0.6",
Expand Down
2 changes: 1 addition & 1 deletion src/getframes/__about__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
# SPDX-License-Identifier: MIT
"""Single source of truth for the package version."""

__version__ = "2.3.0"
__version__ = "2.4.0"
3 changes: 2 additions & 1 deletion src/getframes/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@
nondestructive_stack_statistics,
ramp_photon_transfer,
)
from .backend import ArrayBackend, get_array_module, get_backend, to_numpy
from .backend import ArrayBackend, get_array_module, get_backend, resolve_precision, to_numpy
from .calibrate import calibrate, combine
from .camera import Camera
from .config import CameraConfig, SensorType
Expand Down Expand Up @@ -108,5 +108,6 @@
"load_preset",
"nondestructive_stack_statistics",
"ramp_photon_transfer",
"resolve_precision",
"to_numpy",
]
Loading
Loading