Conversation
Brings the torch FRF stack onto a current dev base. Incoming changes were taken only under torchref/experimental/alignment/ and its tests; every other path keeps the alignment2/dev version, so the 197 commits of dev fixes since the 2026-06-15 fork are preserved. Taken from fix_alignment: - frf/ (13 modules), align.py, sh.py, wigner.py, ml_rotation.py, lattman_love.py, and the reworked __init__/pipeline/rigid_body/translation - removal of the JAX ball engine (ball_transform.py, jax_subpixel_peaks.py), which was never functional here -- jax/s2fft are not installed and its entry points raised ImportError - tests/unit/alignment, tests/unit/frf_separate, tests/integration/alignment Discarded (dev version kept): pyproject.toml, torchref/model/model.py, model_ft.py, sf_fft.py, torchref/scaling/solvent.py, and three new tests/unit/model tests that exercised the discarded model.py changes. Compatibility fallout between the ported package and current dev is expected and not addressed here. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
`_rebuild_sf_indices()` builds the iso/aniso partition (`_iso_indices`, `_aniso_indices` and the two fast-path flags) from `aniso_flag` and the heavy-atom mask. It ran only in `load()`, and the partition is neither a buffer nor a parameter wrapper, so `copy()` did not carry it: the copy raised `AttributeError` from `get_iso()`/`get_aniso()`. `Model.copy()` also assigned the space group as an object. `spacegroup` is a property, but `SpaceGroup` is an `nn.Module`, so `nn.Module.__setattr__` intercepted the assignment, stored it in `_modules` under the property's own name and never ran the setter -- registering the *original's* SpaceGroup as a second submodule of the copy, shared by identity. `ModelFT.copy()` already assigned `_spacegroup` directly; `Model.copy()` now matches it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Miller indices transform as `h' = h.S`, i.e. with the transpose of the symmetry rotation, which is what `SpaceGroup.apply_to_hkl` implements. Four sites re-derived the orbit expansion inline as `S.h`: align.py `_prepare_frf_inputs` obs unroll align.py `_run_frf_separate_rotation` non-epsilon unroll frf/preprocessing `epsilon_aware_unroll` sh.py `hkl_symops_to_cartesian` `S.h` and `h.S` agree whenever the symmetry matrices are orthogonal, which they are in every setting except trigonal and hexagonal -- so this was invisible on most of the benchmark. Where it is not orthogonal the unroll sites mix non-equivalent reflections into one orbit, and `hkl_symops_to_cartesian` returns matrices that are not rotations at all: orthogonality error 5.33 for P 3_1 2 1 and P 6_5 2 2, against 2e-7 with the transpose. `symmetrize_anisotropy` consumes those matrices, so the point-group projection of the anisotropy tensor was averaging over non-rotations and could increase the anisotropy it was meant to constrain. `compute_epsilon` already used the row-vector form and is unchanged. The new tests pin the premise (which settings are non-orthogonal), the observable consequence (the orbit differs as a set only when non-orthogonal), and the absence of new copies of the wrong contraction anywhere in the package -- the failure mode here was a shared helper being fixed while inline duplicates were missed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
…other `Model.rotate` and `Model.translate` mutate in place and return `self`, so every call site that treats them as returning a fresh model needs a `.copy()` first. `_make_rotated` is called once per rotation candidate off the same `self.model`, so candidate k+1 was evaluated at an orientation composed on top of candidate k, and `self.model` was destroyed in the process. Same pattern in `_dense_rotation_refine`, `_rigid_body_polish` (whose caller keeps the unpolished model when the polish does not improve R) and the post-placement translate. Also pass the space-group NAME rather than a SpaceGroup object at the three sites that build P1 copies. `spacegroup` is a property, but `SpaceGroup` is an `nn.Module`: object assignment is intercepted by `nn.Module.__setattr__`, filed under `_modules["spacegroup"]`, and the setter never runs -- so the "P1 search model" still carried the crystal symmetry. It also poisons the attribute, since a later correct string assignment then raises TypeError. Behaviour change to note: the `rescore_engine="none"` early return moved below the array extraction so sub-peak refinement runs in that branch too. It sharpens each orientation in place without reordering, so it is independent of which engine ranks the candidates -- but any baseline measured with `rescore_engine="none"` and `subpeak_refine=True` is not comparable across this commit. The new tests are fast, unlike the integration tests that previously covered this, which `--run-slow` gates off by default. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
The package was forked before several core changes and never caught up: - `Scaler.rfactor` was removed. `align._external_rwork` now calls `rfactor_work_free(data, abs(scaler.forward(fcalc)))` -- it takes scaled amplitudes, not complex F_calc -- and `RigidBodyRefinement` uses `XrayTarget.get_rfactor`, which scales through the target and so reports R against the same work/free partition the loss uses. - `SfFFT` no longer takes `radius_angstrom`; the splat radius is the global `torchref.sigma_cutoff_ed`, in sigmas. Dropped from `LattmanLoveInterpolator.__init__`, which no caller passed. - `torchref.alignment.ml_rotation` does not exist; the lazy import in `local_rotation_translation_refine` is now relative. - `rigid_body.py` imported three names twice and two it does not use, one of which (`MaximumLikelihoodXrayTarget`) no longer exists and made the whole package unimportable. `ModelFT.fit_to_data` does not exist either, so the merged integration tests and drivers now call `align_model_to_data` directly. One of them was comparing the aligned result against a reference that its own in-place `rotate` had already moved. Also drops the five `SpaceGroup` imports left dead by the switch to name-string assignment. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Named modules, files and methods that do not exist: `torchref.alignment.*` (the package is `torchref.experimental.alignment`), `phaser_frf`, `ball_search.py`, `benchmark_phaser_frf.py` with its `--engine` flag, `ModelFT.fit_to_data`, and three markdown/CSV paths that were never in the repo. The `fit_to_data:` prefix also appeared in nine runtime progress messages. Also drops the "v13" post-mortem prose from the sample-list and peak-finder module docstrings: what the deleted implementation got wrong is not the behaviour of this one. Adds a note in `bessel_sh_expand` that Phaser's radial band is per-`l` (`nmax = (lmax - l + 2)/2`, DataMR.cc:894) and that `N_radial` is only the allocated width, plus tests asserting the populated support equals that band rather than merely fitting inside it. The stale references inside `_run_frf_separate_rotation` are left alone; that function is being replaced. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
`build_adaptive_sample_list` clamped `pmax`/`qmax` to a minimum of 1. Phaser truncates toward zero with no clamp (FastRot.cc:214-215) and its `for (p = 0; p < pmax; p++)` body then never runs, so the beta section is genuinely empty; clamping invents a section Phaser does not sample. A section that comes out empty now contributes no samples, with `beta_starts` still carrying an entry so the slice stays representable. Behaviourally inert at every practical sampling step: the sample lists are bit-identical for grid_sampling_deg 2, 3, 4 and 5 degrees. The clamp could only bite for a step coarse enough to drive `pmax` to zero, and `qmax = 0` at beta = 0 is unused because that branch samples the alpha = gamma diagonal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Fifty-odd copy-pasted scripts drove the rotation-function investigation, with the same helpers reimplemented dozens of times and two of them in incompatible variants: `random_rotation` existed with and without the `sign(diag(R))` correction, so the same seed gave different true rotations depending on which script you ran, and rank-of-truth existed in four left/right x fractional/Cartesian flavours. `alignment_lab/lab` holds one definition of each, with the conventions as explicit arguments and recorded in every output row, and `alignment_lab/tests` pins the contracts whose violation silently changed results. `analysis/aggregate.py` reports paired per-trial differences and signs rather than a bare median: seed-to-seed truth-rank spread is +/-4-6 ranks, and three findings that looked strong below ten trials did not survive the full set. Run outputs (`runs/`, 457 MB) and scheduler logs (`slurm/`) are gitignored, as is the third-party Phaser source copy used for the FRF instrumentation. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
One diagnostic covering the four engine settings that are still switches because no value was ever chosen: `lmax_cap` (48 vs 64 vs Phaser's DEF_CLMN_LMAX of 100), the anisotropy estimator, `_orbit_unroll`, and the Patterson-radius union. Stage 1 is the lmax x anisotropy factorial, stage 2 takes the follow-ups one at a time from the winning cell. Every arm runs in one process per (structure, trial) cell, so the paired comparison against the shipped configuration is exact, and `production_dup` repeats the baseline verbatim to measure the engine's own run-to-run spread -- which bounds how small an effect the sweep can resolve at all. `merge_peak_lists` pools peak lists by z-score with SO(3) suppression. Absolute rotation-function values from two Patterson radii are not comparable; the per-run standardised heights are. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Two whole modules had no reader anywhere: `frf/bessel.py`, whose `spherical_bessel_table` duplicates the one in `data_mr.py` that the engine actually calls, and `frf/spherical_y.py`, a second implementation of the normalised Legendre recurrence and `Y_lm` that `sh.py` already provides. `frf/wigner_d.py` re-exported five names from `..wigner` and called none of them, and its `small_d_stable` was superseded when the same `J_y` eigendecomposition was inlined into `wigner_contraction_per_beta`. The one test that imported `small_d_packed` through the shim now takes it from `..wigner` directly, where it serves as the independent reference implementation the contraction is checked against. `WignerContraction` was never instantiated, `AdaptiveRotationFunction. total_samples()` never called, and `BesselSHCoefficients.N_radial` duplicates `coeffs.shape[0]`. `bessel_h_scale`, `beta_grid` and `beta_starts` are kept: they are not derivable from the arrays and a reader needs them to interpret the object. Also: `_build_beta_grid` (never called; its logic is inlined at the three use sites), `peak_finder._so3_angular_distance_deg` (the NMS re-inlines the cosine test, and the test that checks it carries its own copy), and a line in `adjust_gridding` that computed `primes` only for the next line to overwrite it. `_euler_to_matrix_edmonds_zyz` stays in `peak_finder` rather than deferring to `rotation_utils`: the two are algebraically equal but round differently in the last bit, which flips NMS suppression for pairs on the threshold. Its docstring now says so, so the duplication is deliberate rather than accidental. `bench_stages` timed `frf.bessel.spherical_bessel_table` and `wigner_d.small_d_stable`, neither of which the engine calls -- the README already recorded that they registered zero calls. It now times the stages that run. Verified behaviour-preserving: single-threaded, the peak list is bit-identical before and after on 1DAW and 3GR5. (Multi-threaded it is not reproducible even against itself -- see the following commit.) Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
The engine carries no RNG and no order-dependent atomics, so with one thread two identical calls give a bit-identical peak list. At the default thread count they do not: the structure-factor reduction is float32 and its parallel summation order varies, which on 3GR5 (P 6_5 2 2) moves peak scores by ~5e-8 relative and reorders about a dozen of 500 peaks. On 1DAW the same noise is ~8e-16 relative and nothing moves. That sets the resolution floor for any peak-list comparison, and it means a refactor has to be checked single-threaded to be checked at all. Truth rank was stable across threaded repeats on both structures, but two peaks within 5e-8 of each other are not ordered reliably. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
`FRFInputs` carried four fields no code anywhere reads: `s_vec_for_search`, `s_mag_sym`, `patt_obs` and `patt_calc`. The rotation search recomputes everything it needs from `data` and takes only `U_aniso` and `device` from the dataclass; the rescore and translation stages read the per-reflection arrays. Building the dead four cost a symmetry expansion of the whole Miller list plus a full Lattman-Love interpolation over it (`n_ops x N` reflections) on every run, with the result discarded. `_shellbin_norm_etrick` existed only to feed them. `MolecularReplacementPipeline` also accepted eight kwargs it stored in a `_vestigial` dict that nothing reads, and `align_model_to_data` forwarded all eight plus `L` from its own signature. Removed, along with the flags three integration drivers passed down to them -- including four `--use-*` CLI options whose help text describes experiments that no longer have an implementation behind them. Verified behaviour-preserving: single-threaded, the 1DAW peak list is bit-identical before and after. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
`--gate` scores each arm on what the pipeline actually needs: truth inside the top N candidates it carries forward, on most trials, for *every* structure. Rank 7 and rank 0 are the same outcome downstream and rank 223 is not, so a median rank hides the thing that matters and an average over structures lets one failing space group be cancelled by nine easy ones. It also pairs on `rank_for_compare` instead of `truth_rank`. `compare` drops any pair where either side missed, which silently discards exactly the cells an arm is being blamed for. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
`E[I(h) / <I>_shell] = c exp(-2 pi^2 s.U.s)` holds exactly in intensity space. The fit regressed it in log space instead, unweighted and with no constant term, so the `E[ln(I/<I>)] = -gamma` offset -- and `-gamma - ln 2` for centric reflections -- had nowhere to go but the quadratic form. Because centric reflections lie on the zones perpendicular to the symmetry axes, the resulting bias was direction-dependent rather than a harmless overall scale. Vanishing amplitudes were also clamped to 1e-30 and logged, turning each into a residual of about -69 in an unweighted least squares. Measured raw B spreads from the old fit, over the ten benchmark structures: 70 to 5461 A^2. `symmetrize_anisotropy` annihilated it for cubic lattices, where the invariant subspace is one-dimensional, and left 83 to 370 A^2 standing everywhere else. The replacement fits by Gauss-Newton with a free constant, weights from `Var(I/<I>)` (1 acentric, 2 centric), and drops non-positive amplitudes rather than clamping them. Its symmetrised spreads on the same panel are 0.0 to 73.7 A^2, and its residual spread on isotropic synthetic data falls as 1/sqrt(n) -- so what remains is estimation noise, about 7 A^2 at 40k reflections, not bias. That noise floor is why the correction is indistinguishable from no correction on most of the panel: the anisotropy it finds there is smaller than what it can resolve. 3GR5 (P 6_5 2 2) is the exception, at 73.7 A^2 uniaxial along c. Effect on the rotation search, ten structures x ten seeded orientations: the old fit puts truth outside the top 20 on 3GR5 in 10 of 10 trials at every bandwidth tried; the corrected fit is inside on 10 of 10. The signature gains `centric`, which both call sites already had in hand. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
… error) `rotation_search(model, data, model_error_A)` replaces a surface of 41, 32, 26 and 21 keyword arguments spread over `align_model_to_data`, `phaser_rotation_search`, `FastRotationFunction` and the engine wrapper. Nine of those were provably dead, fourteen had a non-default branch no code in the repo ever took, and five had a *default* production never used -- `align.py` flipped them on the way past, so no single file stated the shipped behaviour. Everything else is derived from the three inputs, following Phaser's own chain (runMR_FRF.cc:419-448): the bandwidth from the model's mean radius and the data's resolution, the sigma_A fall-off from the coordinate error, the Wilson normalisation and French-Wilson posterior from the observations and their sigmas. What remains fixed is six module constants, each with its provenance in a comment beside it. `model_error_A` is now honoured. The old entry point accepted a coordinate error and then overwrote it with the Oeffner estimate from the atom count whenever `vrms_strategy` was left at its default, so the caller's value was silently discarded. `MolecularReplacementPipeline` and `align_model_to_data` take the same argument and fall back to that estimate explicitly when it is None. `_run_frf_separate_rotation` is deleted; `rotation_search.search_peaks` is the implementation, and the pipeline calls it. Verified equivalent: single-threaded and given the Oeffner estimate, the new path reproduces the old peak list exactly on 1DAW, and on 3GR5 differs only through the engine's own run-to-run spread (which is not zero even single-threaded -- three fresh processes of the old path give two distinct peak lists). The lab keeps its ability to vary the constants, by rebinding them for the duration of one call rather than by passing arguments the API no longer has -- so the measurements that chose the values stay reproducible while the API stays switch-free. `FRFConfig.extra` now raises rather than silently ignoring knobs that no longer exist, and `lab/aniso.py` carries the superseded log-space fit as an explicit arm so the anisotropy comparison can be re-run. Also removes a `--lmax-cap` flag from the pose-recovery driver that no longer reached the pipeline, and stops that driver recording a bandwidth the engine never saw. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
`phaser_rotation_search` was a 32-parameter pass-through that constructed `FastRotationFunction` and called `score_model`. Its stated purpose -- signature parity with a module that no longer exists -- lapsed some time ago, and two of its parameters were documented as accepted and ignored. Callers now construct the engine directly. The engine keeps the parameters that describe the data, the bandwidth and the device, and loses the knockout-bisection toggles: `use_epsilon` with its `hkl_obs`, `obs_lmax`, `obs_solid_angle`, `patterson_radius_scale`, `n_var_shells`, `bessel_h_scale`, and the three flags production always set on. Two of those are now derived from the data instead of declared: French-Wilson runs when sigmas are present, and the m-filter reads the space group, which gives ZSYMM 1 for P1 anyway -- so the synthetic tests that switched them off were asking for the behaviour they already got. Also gone: the `FRF_DEBUG` environment switch, the three write-only attributes (`auto_lmax`, `_zsymm`, `_obs_lmax`, the last two advertising a calc-side reuse that `score_model` contradicts by hard-coding `zsymm=1`), and `preprocessing.solid_angle_weights`, whose only caller was the deleted quadrature-weight experiment. The lab captures the rotation function by wrapping `score_model` rather than the deleted function. Verified behaviour-preserving: single-threaded, the peak list is bit-identical before and after on 1DAW and 3GR5. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Ten benchmark structures at ten seeded orientations each, scored on whether truth lands inside the top twenty candidates -- the window the placement search carries -- for *every* structure, not on a median that lets one failing space group be cancelled by nine easy ones. A repeat-baseline arm in the same sweep puts the engine's own run-to-run spread at 1 cell in 100, which bounds what any of this can resolve. `LMAX_CAP = 64`, and the optimum is interior. All cells inside the top twenty: 95/100 at 48, 98/100 at 64, 98/100 at 100. The binding case is 1AK5 (P 4 3 2) at 6/10, 9/10 and 8/10, so only 64 clears nine of ten everywhere. Phaser's own ceiling of 100 is worse there, six to ten times slower, and needs more than 32 GB on three of the ten. Two candidates measured and rejected, both now documented where a reader will look for them: - the orbit-deduplicated obs unroll churns 28 of 100 cells in both directions (26 better, 13 worse) with the binding structure unchanged; - the two-radius Patterson union costs exactly double and changes one cell in a hundred. An earlier measurement had favoured it; that gain does not survive the anisotropy fix, which is what it had been compensating for. `aggregate.py --gate` now refuses to run when the cells carry different trial counts. It had been comparing ten trials of one structure against twenty of another -- the pre-OOM rows and their re-run had both landed -- which silently inverted which arms passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
…ntime One row per (structure, trial, arm) carrying all three, so a change cannot buy one at the silent expense of another -- a bandwidth that halves the runtime while dropping the true orientation out of the carried window is not a win, and neither is one that needs memory the machine does not have. `lab/profile.py` holds the primitives. Three hazards it is explicit about: - **Instrumentation points.** `frf/api.py` binds `bessel_sh_expand` and its neighbours into its own namespace at import, so wrapping them in the module that defines them intercepts nothing and the stage reports zero calls -- indistinguishable from a free stage. `FRF_STAGES` names the module where each call is *resolved*. Getting it wrong left 85% of the runtime unattributed; with it right, attribution is 97% and `bessel_sh_expand` is 59% of the run. - **Nested stages.** `evaluate_rotation_function` contains `build_dense_map_per_beta` contains `wigner_contraction_per_beta`, and `bessel_sh_expand` contains `spherical_bessel_table`. Reported exclusive alongside inclusive, so the column sums. - **Wall clock on a shared cluster measures the cluster.** Every row carries a fixed calibration workload timed in the same process plus the host identity, and the array script takes an exclusive node with a pinned thread count rather than inheriting one from the allocation. Peak memory is an RSS sampler, so a spike shorter than the interval is invisible and glibc may not return freed pages; both are stated where the numbers are, and `vm_hwm_mb` carries the process-lifetime high-water mark. `bench_stages.py` is retired: this subsumes it, including the "inner stages register 0 calls" gap its README recorded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
… copy to it The translation search ran on all data -- 228k reflections to 1.5 A on 2DQ6 -- and on the four largest panel structures it placed the model at the right orientation and 20-56 A from the true position, on every trial. Its own score is higher at the wrong place than at the deposited pose (0.665 against 0.350 on 2DQ6, where the likelihood is 1616 against 157865): the objective's calc side is raw |F_calc|^2, so at high resolution it follows whichever reflections carry the largest calculated intensity. The benchmark never saw this because it checked the rotation only. The window now defaults to the rotation search's [d_max, d_min], one resolution window and one Wilson normalisation for both stages; 0.0/inf remove a cut. The P1 copy is gridded at tf_d_min/1.5: coherence with the 1.0 A grid is 0.9995-1.0000 over the 15-4 A set on 1DAW, 2DQ6, 3K7M and 4BX9 (0.987 at tf_d_min itself), at 10-38 ms against 200-860. Measured with the pose gate (job 544884): 30/30 true poses within 0.32 A at the default, 18/30 with the window removed. Per-alignment wall clock 2.0-6.7 s against 5.8-54 s on shared nodes. The integration tests imported the deleted align module and passed removed keyword arguments, so they had not run since the refactor; they now use the package entry point and Cartesian mates, and the translation test checks the position. run_random_pdb_fit.py exercised only removed features and is gone. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
…hood The fast translation function divided raw |F_calc|^2 by its own sum. That is not a correlation: it is unbounded, and at high resolution it follows whichever reflections carry the largest calculated intensity. On the four largest panel structures it was higher 40 A from the true position than at the deposited pose (0.665 against 0.350 on 2DQ6), and the search went there on every trial. The observed side is now the rotation function's LERF1 coefficient, cw (E_obs^2 - 1) w sigma_A^2, and the calculated side is the candidate's transform divided by its own Wilson curve on the same abscissa, so every candidate's E_calc has unit mean per shell and the map is the covariance of two normalised intensities -- the rotation function's score equation, for translations. One FFT on a grid a third of the set's resolution apart, with parabolic peak refinement, replaces the 16-point coarse grid and the three 100-point local refines whose coarse half could miss the peak. The Rice/Woolfson likelihood at a fixed Luzzati sigma_A picks among the top peaks and ranks the candidates, so no candidate is scored against a model error fitted to itself and the per-candidate SigmaAEstimator fit is gone. The stage runs in the configured float and complex dtypes. use_llg_tf, n_translation_peaks and translation_grid_steps are removed; the lab diagnostics built on the old API are deleted. Measured with the pose gate (job 544899): 30/30 true poses at the default window and 30/30 with the window removed, maximum translation error 0.21 A, against 18/30 before. Warm on an exclusive EPYC 9335 node, 8 threads (job 544917): 1DAW 1.1 s, 2DQ6 1.9 s, 6G9X 2.2 s, 3K7M 2.7 s, 4BX9 3.8 s per alignment. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
…not its level empirical_sigma_a divides the observed Wilson curve by the calculated one and takes sqrt(min(R, 1/R)). The observed curve is on the data's arbitrary scale and the calculated one on the model's electron scale, and nothing removed that factor, so the ratio's level set the answer: measured 0.02-0.06 on 1DAW and 2DQ6 and 8-12 on 3K7M, giving a flat sigma_A of 0.15-0.35 with no resolution dependence -- not the resolution-dependent model deficiency the docstring describes. Each curve is now divided by its geometric mean over the supplied points before the ratio is taken. Its only caller is the rotation function's calc-side weight, where a per-shell factor is gauge in the correlation, so placements do not move: 30/30 at the default window and 30/30 uncut (job 544925). 191 tests pass across alignment, scaling and the alignment integration suite (544924). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
The greedy SO(3) suppression treated an orientation and its point-group mates as different peaks, so the shortlist handed to the translation search was mostly copies: 187 of the 300 pairs among 3K7M's top 25 peaks were mates of each other, 38 on 3GR5, 25 on 2DQ6. With the Cartesian symmetry rotations supplied, a kept peak suppresses its whole orbit and every returned peak is a distinct orientation. The group composes on the right, R R_g. Measured rather than assumed (alignment_lab/diagnostics/frf_orbit_side.py): on real peak lists the left orbit finds zero coincident pairs on every structure tried and the right orbit finds every mate. The lab's symmetry_orbit and orbit_rank defaulted to the left side, which is why the orbit-based truth rank disagreed with coordinate superposition; the default and the five harnesses that pinned the left side now use the right. 192 tests pass (job 544938). Placements unchanged: 30/30 at the default window and 30/30 uncut (job 544939). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Every success count quoted in the pipeline and the pose-recovery harness -- 30/30, 24/30, 37/40, 36/40, 32/40 -- gated on the rotation alone, with a metric that read two of the six trigonal mates of a correct solution as failures. Measured on poses over six structures x ten seeds (job 544953) the three ranking arms are 60/60 each and pick the same candidate in every cell; the likelihood stays the default as the right object for the question, not on a measured margin. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
…d model Each rotation candidate copied the search model twice -- once to rotate, once to set P1 on -- and a third time to build a placed model nobody read for 24 of the 25. On the 20k-atom structures that was a quarter of the run. One P1 copy is built with the translation set and re-oriented in place per candidate (the forward cache fingerprints parameters by pointer and version, so the next structure-factor call recomputes); the placed model is built for the winner, and place() builds it for any other solution. 192 tests (job 544963); 30/30 at the default window and 30/30 uncut (544964). Warm, exclusive EPYC 9335 node, 8 threads (544965): 1DAW 0.85 s, 2DQ6 1.35 s, 6G9X 1.30 s, 3K7M 2.35 s, 4BX9 2.55 s, from 1.1 / 1.9 / 2.2 / 2.7 / 3.8 s. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
…on function The pair (j, i) is the conjugate of (i, j) at -dh and the diagonal carries no t, so the map is twice the real part of the upper triangle's transform plus a constant. Half the scatter, which is what the stage costs on high-symmetry cells: 3K7M's translation stage 1.04 s to 0.72 s, the whole alignment 2.35 s to 2.01 s (job 544990). 192 tests (544988); 30/30 at both windows (544989). MRSolution.candidate_index records each solution's position in the rotation function's list, and the pose harness prints it, so the depth of shortlist a solution needed can be read off. Measured over 10 structures x 5 seeds (544991): with symmetry mates suppressed the rotation function's first peak is the true orientation in all 50 cells. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
With symmetry mates suppressed, the rotation function's first distinct peak is the true orientation in 50 of 50 pose-gated cells (10 structures x 5 seeds, job 544991), so the 25-deep shortlist was mostly cost. At 10 the panel is 30/30 at the default window and 30/30 uncut (545015), and the warm exclusive-node time per alignment is 1DAW 0.42 s, 2DQ6 0.67 s, 6G9X 0.67 s, 3K7M 1.02 s, 4BX9 1.11 s (545016), from 0.85 / 1.31 / 1.23 / 2.01 / 2.42. The depth is a safety margin measured on deposited models as search models; it is a parameter, and poorer models should raise it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Grid sizing on ModelFT is lazy and keyed on (cell, space group, max_res, explicit_gridsize); the alignment package's P1 copies set those in any order and rely on the forward cache's state fingerprint, which now folds in the grid key. SpaceGroup.epsilon keeps the friedel= parameter this branch added (the molecular-replacement likelihood needs the rotational count) and takes dev's dtype annotation. The three alignment modules dev touched (rigid_body, transform, clashscore) were removed on this branch as unreachable and stay removed; nothing on dev outside them imports them. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
DirectModelEvaluator wrapped a plain P1 ModelFT to present the old interpolator interface, with the rotation and cell arguments ignored, and to land the result on the default device. With the grid derived lazily from cell, space group and max_res the wrapper had nothing left to do; prepare_candidate takes the model and does the device move itself. The probe that asked whether a P1 copy was needed at all is answered by the one-template design and is removed. 192 tests (550841); 30/30 at both windows (550842). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
dev's dtype-conformance guard requires each torch.<dtype> literal outside the exempt kernels to say why it deviates from the configured dtype. Of the 109 sites it flagged after the merge, five were unjustified and now use the configured dtype: the two casts to double around the empirical sigma_A ratio, the dense P1 transform's amplitudes, and the translation stage's resolution mask and peak translations. The rest are annotated: index tensors that index_add_ and gather require in int64, host-side 3x3 rotation algebra in double, the rotation function's deliberate double accumulation of oscillatory sums, exact clustering keys, and the anisotropy fit. Full unit gate 2043 passed, 82 skipped (job 550854); alignment tests 192 (550852); panel 30/30 at both windows (550853). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
… package The rotation function's back half accumulated one step wider than the expansion -- complex128 for the radial sum, the Wigner contraction and the FFT -- and its inputs, the dense P1 transform, the shell sums and the expansion's clustering keys were cast to float64 on the device. None of it survives on a backend without float64, and none of it is needed: measured on the pose panel, single precision everywhere on the device recovers 30/30 at the default window and 30/30 uncut, with per-alignment times unchanged (0.44-1.12 s at depth 10, job 551437). The earlier measurement that motivated the wide accumulator showed the deep peak list reordering while the top peak stayed put; the placement search now consumes only the top few distinct orientations. The expansion's clustering keys need double -- _GROUP_SCALE_S keys |s| at 1e-7, below float32's resolution -- and are formed on the host, which always has it; only the integer keys reach the device. The Bessel ladder runs in its argument's dtype, kept in range by its power-of-two rescaling, so the bit-identity and scipy checks still exercise a double ladder. What remains in double is host-side: 3x3 rotation algebra, the Wigner-d eigendecomposition, the anisotropy fit and RotationSolutions. The two tests that pinned the wider accumulator now pin the configured dtype. 227 alignment, scaling and Bessel tests (552234); full unit gate 2043 passed (552236); panel 30/30 at both windows (552235). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015Q18eF1DJYrKv61uXowjeq
Refinement output used to copy the input file's header verbatim and then append its own REMARK 3, so a refined 3GR5 carried 420 header lines asserting two refinements at once: REFMAC 5.1.24 with R-work 0.213 at line 5, ours at line 389. A reader taking the first REMARK 3 it found got REFMAC. The inherited block also contradicted the data beside it -- it claimed a 5.1% / 1072-reflection test set while the MTZ shipped with it holds 9.85% / 2063, which torchref reads correctly. The passthrough was inverted as well, keeping the statistics refinement invalidates and dropping the chemistry it does not: SEQRES, SSBOND, DBREF, EXPDTA, COMPND, SOURCE, KEYWDS, SEQADV, HETNAM, FORMUL and SITE were all absent from the output. A whitelist now carries the crystal, sample and chemistry records through in mandated record order (TITLE used to land after REMARK 900), REMARK 2, 3 and 500 are dropped, and AUTHOR and JRNL are not inherited because they credit the deposition rather than this run. 283 header lines for the same file, 41 structural records preserved, one refinement block. add-metadata is exempt through supersede_refinement=False: annotating a file is not re-refining it, so nothing there supersedes the existing records. Prior refinements are tracked through mmCIF's _software loop, the only place either format has room for them -- _refine is singular by design. pdbx_ordinal was hardcoded to 1 and the incoming loop was never read, truncating the chain to one link on every write; it now reads the input's loop and appends at max(ordinal) + 1, carrying each entry's description so the chain says what every program did. Added _pdbx_initial_refinement_model, _refine.pdbx_starting_model and _refine.pdbx_R_Free_selection_details. from_cif_file no longer carries the input's _refine items through, which was the mmCIF form of the duplicated REMARK 3. Also fixed, all of the same kind: - mmCIF loop cells were written unquoted (only the pair path quoted), so any value containing whitespace split into extra columns when the file was read back. Latent while every loop column was a single token; a multi-word _software.description exposed it. - PDB coordinate and B-factor columns were written as str() of a rounded float rather than with an explicit precision, dropping trailing zeros: 18.3 and 31.51 where the format wants 18.300 and 31.510 (369 of 1329 atoms in a 3GR5 refinement), and 95.4 where it wants 95.40. Both writers affected. Columns held and values round-trip through any float()-based reader, so nothing was numerically wrong. - rfree_source was set only when the MTZ also carried a validation column, leaving the common FreeR-only case unattributed, and generated flags did not record their seed. It is also named after the reader now, since load() takes ReflectionCIFReader as well as MTZReader. The header records what was refined and how: refinement_method was only ever set by the difference-refinement CLI, so ordinary runs emitted no method line at all. TARGET and OPTIMIZER come from the settings that already reach refinement_history.json, including the scale target because it changes the R-factors the same header reports. Long values wrap onto a continuation line with the colon held in column 25, the convention REFMAC uses for its own author list. Author-supplied text goes in --output-remarks, rendered as REMARK 3 OTHER REFINEMENT REMARKS and _refine.details and emitted only when set. Nothing in the block is editorialised: every generated line is a measured quantity or a recorded setting. Removed the deprecated template= argument to pdb.write (nothing called it) and the never-populated custom_remarks field. RefinementMetadata had no test coverage at all, which is how the duplicated REMARK 3 shipped; 39 tests now cover the contract. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Rigid-body refinement stores its Euler angles pre-multiplied by the chain's radius of gyration, so a unit step in an angle and a unit step in a translation displace atoms comparably. In radians against Angstroms the rotation block of the Hessian carried 190-530x the curvature of the translation block on 1DAW and 3E98 -- the geometric Rg**2, 411 and 442/516 -- putting cond(H) at 1e3-5e3, which is why six parameters needed hundreds of L-BFGS iterations to place. Dividing the scale out in forward() brings the ratio to 0.4-1.3 and cond(H) to 3-18; the step then converges rather than exhausting its iteration budget, on about half the gradient evaluations, with R-free no worse on any of ten structures. RigidXYZTensor.rotation_radians returns the physical angle, and setting angle_scale to ones gives the unscaled parametrization. update_fixed_values recomputes the scale alongside the chain centres, since it accepts coordinates that are not a rigid re-pose; copy() carries it rather than re-deriving it, so an overridden scale does not change the copy's pose. Not a fix for the one or two negative Hessian eigenvalues at the finer cutoffs, and those counts are unchanged. The step also no longer co-refines the scaler in the same L-BFGS as the rigid parameters. The body target centres on alpha*|F_calc| and alpha absorbs a rescaling of F_calc exactly, so the scale has a flat direction there; SCALE_TARGETS already excludes every alpha-centred row from the scale fit for this reason. refine_scaler (objective ls) owns the scale, between cutoffs. And fixed refine_rigid_body leaving the caller's reflection data truncated. cut_res masks in place and returns self, so each cutoff stamped its resolution mask on the caller's own object and the restore had nothing to restore to -- it only looked correct because the default schedule ends at the native limit. With --rigid-body-cutoffs 6,4 on a 2.05 A dataset, 20138 of 23352 reflections stayed masked out for the rest of the run, R-factors included. Object identity cannot catch this, which is why the existing isolation test did not. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01XodKhB5rwTiiAWFbonP6v2
LossState owns hierarchical multiplication, aggregation, zero weights and cached reads. Default group weights remain separate. NLL checks move to base metrics; gradnorm smoke checks become exact RMS expectations. Validation: 36 passed on default MPS and CPU float64. Follow-up: all-zero aggregate returns torch float32 under configured float64; the retained zero-weight contract checks the value, not output dtype.
Remove local-only arithmetic and duplicate Target initialization; nn.Module inheritance remains in the comprehensive Target contract. Keep anisotropic DELU gradient routing, SIGD references and R-factor calls. Add deposited-coordinate bond, angle, chiral, plane and SIMU references plus exact LS weighting/mask values. Retain kernel/gradient boundary tests for torsion and DELU. Validation: 48 passed/2 CUDA skips including gradient guards; 34 passed on CPU float64 after accounting for the squared-distance regularizer. Eight zero-return fault injections detected.
CIF and MTZ integration tests own field shapes, crystal metadata, bin means and pair consistency; required assertions no longer depend on optional attributes. Existing CIF-to-PDB writing and device movement stay separate. Replace empty ModelFT state/forward/aniso checks with actual forward/cache checks, leaving restoration and anisotropic coverage in model unit tests. Remove unconsumed shared Model/ReflectionData fixtures. Validation: 13 default-MPS cases passed plus the corrected cache case; CPU float64 run 15 passed, 1 slow skip. Cache recomputation compares real magnitudes to allow backend reduction order.
Replace CIF/MTZ/SF-CIF loops with named per-file compatibility cases and an input inventory guard. Keep 1DAW reader contracts in the quick suite; require --run-slow for extras. Replace eager all_test_structures with one fresh named crystal per scaler/restraint case. Move the extra ModelFT loading cases and symmetry file sweep coverage into the compatibility panel. Merge space-group name cases under their unit owner without dropping parameter variants. Validation: 44 extended cases passed; final regression 477 passed/70 skipped, including 41 explicitly slow cases. Full collection: 2587 cases. No production files changed.
Test cleanup branch
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR to merge dev into main
changes so far:
Version 0.7.0
ModelContext. It now holds the unit cell, space group, atom table, link records, hydrogen settings, and input paths.Symmetryas a crystallography-free class with transform primitives, and madeSpaceGroupa specialised subclass.torchref.restraintswas removed, restraint dictionaries are now plain nested dicts, and residues are identified by(chain, resseq, icode)to fix insertion-code merging.Model.hydrogenatenow aligns monomer templates onto heavy atoms present, generation is the default, andAtomGraph.exclusions_12_13_14derives non-bonded exclusions from bond connectivity.Topologyas aResidueGraphover anAtomGraphwith typed edge blocks andsubset/copyoperations that reindex surviving edges.HydrogenTopologya dataclass, changedSymmetryclasses to dataclasses overDeviceMixininstead ofnn.Module, and removed unusedCellgradient plumbing and theReciprocalSymmetryGrid/ module-level expansion functions.