Repository navigation
Conversation
A plane the step that reached it already read crossed the state, reduced it and only then found nothing to record. A replay decided its reads from dz_init, which it never updates: a short one read a step late, past the spacing, and a long one at every step. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
An atomic per element of the state took most of a monitor reading on a device: 184 ms on an NVIDIA H200 and 1.24 s on an AMD Instinct MI325X at 768 x 768 x 600, against steps of 239 and 224 ms. A runner's points now share the walk-off table too, which is a sixteenth of a state to upload. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
pack_energy summed every mode on one GPU thread, and took 53 of the 55 ms of an energy call on an NVIDIA H200 at 768 x 768 x 600. statistics_peak has the same shape. A low-storage step copied its whole table of partials to read one row. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A first saved plane took the state and the whole history on the host to compare their size with the limit, then dropped them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
The method behind the device measurements of the boundary check, in the tree so that they can be repeated. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A copy to fresh pageable memory takes its first-touch page faults inside the driver, one thread at a time: 1.8 GiB/s on an NVIDIA H200 and 7.9 GiB/s on an AMD Instinct MI325X, for a 5.27 GiB state, against about 52 GiB/s pinned. A bare vendor test of this scheme reached 33 and 50 GiB/s into a fresh huge-page buffer. Field, stage, held-plane and Adams-row reads and field uploads now take it above 256 MiB on a device build. Only the first such crossing allocates the ring, so a field-free run holds none. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A mask of the whole state, on one thread, took twice the copy of it on a CPU and 0.73 s at 768 x 768 x 600 on an NVIDIA H200's host. A sum holding a NaN or an infinity isn't finite, so only one that overflows needs a second look. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
spectrum_to_optics allocated two state-sized arrays and took most of a saved plane once its crossing was fast. An in-memory store now converts into its own row, and its copy goes. Stored bits are unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
The adapter always takes the general engine, so these eight tests held only code nothing can reach. They keep their assertions and tolerances. Refs #88. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
BoundState built the other engine only for a problem the adapter never sends. Its refusals and fallbacks go with the dispatch. Two members that never checked `require_open` keep their test of the pointer, which is null after `close`. Refs #88. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Nothing chooses an engine any more, so the adapter's constant and the query it fed have no reader. `_core` is private: the kernel schema does not change. Refs #88. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
SEP 44 phase 1. A backend now hands the edge an EdgeReading, numbers and never the field, through StepDriver._edge_reading. HostEdgeReduction is the reference, on a host copy, and native takes it until phase 2, so nothing moves yet. READ_RESOLUTION goes, so a read now synthesises every plane. The launch's safety-ramp judgement reads clean columns formed at the load, which needs no state on the host. A resume now restores a collar's current, baselines and peaks (#512). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Nothing can reach it: the general engine runs every problem, and makes every check this engine's validator made. `fft`, `multiply_phase` and `compact_problem` stay in core.cpp, and `_core.State` keeps its name. OccupancyScatter and its ScatterView include had no other user. GeneralState's monitor comments deferred to the deleted class's, so they take over its text. Refs #88. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A load zeroes outside the active band, so a random field never came back bit for bit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
SEP 44 phase 2, for a run without a requested collar. The edge hands native its weight tables, four regions on the nodes, and native returns the region sums and each mode's energy. Native holds no face, mask or judgement, and a field-free default run brings no state to the host, which a test now holds it to. A collar still reduces a host copy. An inverse transform into the free scratch, then one sum: the same state read on both sides agrees within the transform's rounding on the radial, Cartesian, line and planar routes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A reduction's value has to fit the device's shared memory per block. HIP's smallest block is 64 threads, and 128 doubles asked for 67,584 bytes of its 65,536, so every reading with buckets raised on an AMD Instinct MI325X. CUDA shrinks its block instead, so the NVIDIA H200 didn't show it, and an offline HIP compile can't. On HIP a reading now takes two passes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A signal after a run's last poll stops nothing, so it no longer exits as stopped or says to resume. A signal during a recipe's build is documented as waiting for the build. Closes #521. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A whole state at a time missed the 10 % #361 asks of a default-memory change on both data-centre cards: at 768 x 768 x 600, 8 steps, x0.996 on an NVIDIA H200 and x0.978 on an AMD Instinct MI325X. So this reverts ae82a2a, with the seam field it added: `Transform.chunk_budgeted`, `split_rows` on the native problem, and the identity check translation needed for it. The collar stays in place on the state's rows. A device's choices are still tested on any build. The ledger keeps the numbers. `output/callback.py` goes into the layers figure, which CI found missing on every lane. It was the only failure of 18,500 tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
At each plane a lockstep scan drained every member's whole observation, field included, even under fields="none". On an NVIDIA H200 that took 22 % of a plane interval at 8 steps a plane, and on an AMD Instinct MI325X 13 to 16 %, against #299's 5 %. `observe_members` now takes `field=`, as `observe` does, and the scan asks under the rule a lone point of the same scan keeps. `_retained`'s reasoning moves to `_field_retention` so that both executors use one rule. Closes #299. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
… edge A runner's stated input took one thread's finiteness pass over the whole volume: 0.72 s of a 3.24 s point at 768 x 768 x 600 on an NVIDIA H200's host, 0.17 s on an AMD Instinct MI325X's. It now takes the threaded pass a loaded state does. The exact hard-edge fuzz that found #497's bug joins the tree (#504): columns placed on a circle's, a half-plane's and an ellipse's edge, and a float either side, against the reference's decisions, on CPU and in the device tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
… check Both came with the setup work, and this is the first coverage report since it merged. The rational residual on axes beyond `math.fsum`'s squares had no test. And a nested `def` straight after an `if` hides that `if`'s false arc from branch coverage under the type-checking import hook, so the mask's worker moves up beside the other one. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Seed 36 drew five inactive bins, which the load masks, so every row read zero and the test's guard against a vacuous answer failed on the NVIDIA H200's first pass, and on CPU when run that far. Each target now has to give both answers on its own. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
A new backend took 2.1 s of a 4.7 s point at 768 x 768 x 600 on an NVIDIA H200, and 1.0 s of 4.2 s at 384 x 384 x 600 over 51 steps, well past #298's 10 %. On a CUDA or HIP build a `Local` scan's worker now keeps it while each point passes the check a runner makes, which a builder using `with_input` gives. Any other point closes it first, so two are never held. `Simulation.run`'s body moves to `_run`, which can be handed a backend, so that a kept point takes the same path as one alone: its record differs in when it was made and how long planning took. Pool and Batch don't keep one, and nor does a CPU, where SEP 22 measured no gain and the memory goes back to the next point's build. Closes #298. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
`_device_build` stays private, as this work's new names do, and a test that unwraps the run to reach its refusal unwraps the body now. On a real device the first scan of the kept test kept a backend too, so its reference is now held to one a point. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
[skip ci] Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7c11633bb0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| ): | ||
| simulation.run(capture, progress=progress, stop=stop) | ||
| return |
There was a problem hiding this comment.
Close the kept backend before running a non-device point
When a local scan runs a device-backed point followed by a CPU-backed or TDSE point, this early return bypasses the compatibility check and leaves the previous runner in self._kept. The new point therefore constructs its backend while the old backend still owns its device allocations and pinned host buffers (potentially 1.5 GiB for the crossing ring alone), and those resources remain held through subsequent non-device points until the scan finishes. Call self.close() before this fallback run so that switching to a point that cannot reuse the runner releases the previous backend first.
Useful? React with 👍 / 👎.
A draft: one pull request for this work, at the maintainer's request, since each PR and each push costs a full CI run. It folds in #516, #517, #518, #522, #523 and #524, now closed, whose descriptions keep the full tables and per-section verification. The ledger entries in
docs/dev/benchmarks.mdhold the measurements. SEP 44's text, accepted in #515, is here too.The aim: a native run's field stays in its memory space from load to close, and only numbers cross, unless a caller asks to keep a field.
What's here
energyand the peak are real reductions;benchmarks.device_splitKERNEL_SCHEMA_VERSION; the deadnative::Stategoes; the run layer's duck-typed hooks are declaredstop=at each call's return, sosolprop runstops on SIGTERM rather than reporting a stop that didn't happenbeam_radius,fwhm_r,on_axis_intensityand theon_axis_*row reduce on the stateCallbackOutput, anOutputbuilt from a callable, so a reducing scan is a few lines; withfields="none"it holds no field volumeexpis skipped, input goes through tiles, and the accept is one reductionobserve_members(field=): a member's field crosses only at a plane the scan reads, under the rule a lone point keepsLocalscan on CUDA or HIP keeps its backend across points that differ only in inputsolprop runexits 0 and says so when a signal arrives too late to stop a runCloses #510, closes #511, closes #503, closes #502, closes #88, closes #385, closes #512, closes #519, closes #493, closes #499, closes #521, closes #299, closes #359, closes #362, closes #298. #346 is mostly done and stays open for the plane's record; #385's Metal part isn't touched. Relates to #347, #396, #365, #361, #103, #357, #504 and #388. #151 (#525) is separate.
What it does for a run
On the data-centre cards, an NVIDIA H200 at
8cd170e2and an AMD Instinct MI325X at9a5d2346, against the same probe before this work. Field-free,python -m benchmarks.device_split. The MI325X's commit still took the collar a whole state at a time, now dropped, which adds 2.2 % to a step, and it hasn't the threaded finiteness pass. A pass of that card at this head is waiting on capacity.field_to_host, 5.27 GiB, msenergy, msbeam_radius, msEvery target in the plan and SEP 44's gates are met on both cards.
python -m benchmarks ab --deviceagainstmainat8ed26dd7, 3 rounds, the same work:kerr_beam_384x0.051 on the H200 and x0.117 on the MI325X,kerr_beam_192x0.141 and x0.138, fields within 6.2e-16.A requested collar's read, at 384 x 384 x 600, native against the host's reduction of a host copy, with a step of 72 ms: the current 4.5 ms against 2.7 s on the H200 and 3.9 ms against 1.2 s on the MI325X, a read with its strip 18.7 ms against 12.9 s and 14.1 ms against 6.2 s.
A scan. A new backend costs 2.10 s of a 4.69 s point at 768 x 768 x 600 on the H200, and 0.98 s of 4.24 s at 384 x 384 x 600 over 51 steps, so a
Localscan of inputs on a device now keeps one (#298). A lockstep batch's plane no one reads falls from 5 to 7 ms of observing to 0.23 ms, of a plane interval of 23 to 28 ms (#299).A find, not fixed here (#529). FFT plans are the largest row of a device's memory, 21 of 39 GiB at 768 x 768 x 600, because each holds its own work area. At some transverse sizes a pair of them needs none: 900 x 900 holds less than 768 x 768 with 37 % more points, and 750 x 750 is 11 % faster with 5 % fewer. Sharing one area looks possible.
Against 0.1.0, on an AMD EPYC 7763, not interleaved, since 0.1.0 can't run the harness as it stands:
kerr_beam_192at one thread x1.060,kerr_beam_384at 16 threads x0.945. The plan's target was under x1.08. An interleaved run is queued, and the cluster's estimate is a day away, so these may be what you see first.On CPU,
python -m benchmarks abagainste49925d1, 3 rounds, the same work, fields within 4.2e-15: geometric mean 0.85, withkerr_beam_192at 16 threads x0.48. SEP 40's four-way control, default check against no ramp and no reads:line+15.8 %,planar+7.6 %,radial_filament_256+4.1 %,kerr_beam_192+8.3 %.Setup, at 768 x 768 x 600: on an Intel Xeon Gold 6240
from_free169–179 s → 14.9 s andtranslate28–30 s → 7.2–7.8 s. On the H200's host 5.31 s and 3.65 s, on the MI325X's 3.41 s and 1.63 s.The full tables, with what a run holds and the Python share, are in
docs/dev/benchmarks.md, "The native read on the data-centre cards".How it was verified
9a5d2346:pytest -m gpuovertests130 passed, 1 skipped, and overnative/tests31 passed; the edge's, collar's, widths' and statistics' modules with the transverse backends 269 passed, 3 skipped;test_transverse_reads.py,test_transverse_monitor.pyandtest_stop.py66 passed; the targeted device set 210 passed, 7 skipped. The MI325X is the first AMD card to run the read.8cd170e2, the node-chunk collar, the batch's field and the replaced input included: the edge's, collar's, split rows', batch's, widths', statistics' and transverse modules 326 passed, 3 skipped; the scan batch, runner, stop and transverse modules 135 passed;pytest -m gpuon the far field, widths and statistics 177 passed and 3 failed. All three are one seed of the new fuzz that drew only masked bins, a bug in the test, since fixed and passing on CPU in every seed.ccd701d5:pytest -m gpu tests130 passed, 2 skipped; the split rows', edge's and collar's read modules 172 passed; the transverse backends, widths, statistics and march lifecycle 118 passed, 3 skipped. Atec7b5bab, with the kept backend: the kept-backend, scan and batch modules 149 passed;pytest -m gpuon the far field, with the fuzz in all 120 cases, 134 passed.9fd4ceab: all green, 60 checks: every test lane, 100 % coverage, the docs, clang-tidy, the sanitizers, the slow lane and the wheels on every platform. The one commit after it changes only the ledger, with[skip ci], so the head shows no checks of its own.device-compileisn't on this branch, since it's Compile the CUDA and HIP targets in CI #525, which is green too.AGENTS.mdsays. One full gate was run here by mistake, early on.Three separate agents, given the code and not my conclusions, read SEP 44's collar, the statistics and the device step for faults. They found no wrong number. They did find a race on FFTW's planner, a resume test that couldn't fail, a device test that ran one block three times, a device-only attribute missing from the seam's tuple and several weak tests, all fixed. Seven deliberate breaks of the collar and 18 of the device step were each caught, the last two of those only after tests were added.
Not run
9a5d2346has run on HIP. The MI325X's pass atec7b5babkeeps retrying, and has found no capacity since the H200's finished.Decided by the maintainer
check_buildloses itsproblemargument, a public signature, with a fragment. An edit undernative/refuses an editable build until rebuilt, even a comment.solpropandsolprop-nativeare released together:_EdgeReading,_EdgeTables,_EdgeLine,_HostEdgeReduction,_edge_lines,_transverse_edge_bytesand_STRIP_BATCH_BYTESinsolprop.transverse_monitor, and_FluenceRings,_fluence_ringsand_ring_fluence_widthsinsolprop.spatial.numpy_refand read as rounding by native. The oracle is left alone.Backendstays a protocol for now. MakingStepDriverthe contract is noted on Write a SEP for the step loops and the kernel seam: one march protocol, the batch axis, the integrator and its tolerance #110, as a simplification of its own.Still for the maintainer
Final. Phases 1 to 4 are reported, and I've left the status.ae82a2a9. It missed The node chunk is sized at translation, whatever the device, and each chunk is a kernel launch #361's 10 % on both cards, x0.996 and x0.978, so a device keeps the node chunk andTransform.chunk_budgetedgoes. Say if you'd rather have it for the launches.CallbackOutput's design. It implementsOutputdirectly, notPlaneStore, whose save converts into a new volume and which counts as aCheckpointOutput, so a stop would log thatresume=Truecontinues. So it has nooutcome, and TDSE is refused.fieldsstays a run setting and not a handler argument. An emptystat_namesmeans optmat runs lose the peak-field report andMaterialRangeWarning, which is documented, with an info log at run time.check_cache,resumeandresumabilitywiden their annotations toOutputin order to refuse it by name.Localscans on CUDA and HIP, and a point's record differs only increatedandplanning_seconds. Say if you'd rather it were a setting.Known and left: a brief 128 MiB reallocation of far-field partials at 2048 x 2048 x 2049 with two polarisations, by arithmetic;
test_the_native_pass_does_not_depend_on_the_thread_countasserts the host's threads and fails on a CUDA build, from before this work.🤖 Generated with Claude Code
https://claude.ai/code/session_01XGENPtq9Gp7VCh5QmcDdzQ