refactor(data): pin relative_angle to the angles' float dtype - #172
Merged
Conversation
janickm
force-pushed
the
dev/janickm/relative-angle-dtype
branch
from
August 31, 2026 12:55
aefc443 to
17759ec
Compare
relative_angle reduced the signed difference with a python-float 2*pi period. numpy promotes a numpy *scalar* mixed with a python float differently across major versions -- numpy 1 used value-based promotion and widened `float32_scalar % python_float` to float64, while numpy 2 keeps it float32 under NEP 50 -- so the scalar path silently depended on the installed numpy version, and disagreed with the array path (float32 either way) by ~1 ULP under numpy 1. This is reachable from RowOffsetStructuredSpinningLidarModelParameters. get_vertical_fov, which passes `row_elevations_rad.astype(dtype)[-1]` as a float32 scalar. The resulting `span_rad` shift moves the linspace angle grid in _init_angles_to_columns_map, which reassigns cKDTree boundary cells of angles_to_columns_map. Measured on the two Hesai models: 81 of 7372800 cells (Pandar128) and 164 of 2457600 (AT128) change between numpy 1.26.4 and 2.5.2. The current code already avoids this for those two sensors, but only by accident: their signed difference happens to land in [0, 2*pi), so the modulo is a no-op and its dtype cannot matter. On random float32 scalars the reduction still diverges in 44739 of 60000 cases, so a sensor whose elevation span wraps, or whose float32 subtraction rounds, would reintroduce the same failure. Carry the angles' float dtype into both the reference and the period so the result is version-independent by construction. torch needs no coercion: it is already dtype-preserving against python scalars. The angles carry the precision of the computation, so they must be a dtype-bearing float type rather than a bare python scalar -- otherwise there is nothing to pin the reduction to. Non-float dtypes assert as well, for numpy and torch alike, since the two fail differently but both silently: numpy truncates the 2*pi period to 6 and returns wrong angles, while torch promotes to float32 and narrows a float64-precision caller. The dtype is read duck-typed (numpy spells float-ness as dtype.kind, torch as dtype.is_floating_point, present since torch 1.12 -- the oldest version this package supports) so the function stays generic over both without importing torch. All existing callers already pass numpy arrays, numpy scalars or torch tensors. No behaviour change for the current sensors: the angles-to-columns maps are bit-identical to before (0 of 7372800 and 0 of 2457600 cells differ), and float32/float64 array paths are untouched. Validation: //ncore/impl/data:all, //ncore/impl/sensors:all and //tools/... pass on both python 3.8 (numpy 1.19.5, torch 1.12.1) and 3.11. The data/sensors/converter suites also pass unchanged (199 tests) under numpy 1.26.4 + torch 2.7.0 and numpy 2.3.5 + torch 2.13.0, where the angles-to-columns maps come out bit-identical across the numpy major versions.
janickm
force-pushed
the
dev/janickm/relative-angle-dtype
branch
from
August 31, 2026 13:53
17759ec to
9b37908
Compare
This was referenced Aug 31, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
relative_anglereduced the signed difference with a python-float2*piperiod. numpy promotes a numpy scalar mixed with a python float differently across major versions — numpy 1 used value-based promotion and widenedfloat32_scalar % python_floatto float64, while numpy 2 keeps it float32 under NEP 50 — so the scalar path silently depended on the installed numpy version, and disagreed with the array path (float32 either way) by ~1 ULP under numpy 1.This PR carries the operands' float dtype into both the reference and the period, so the result is version-independent by construction.
No behaviour change for the current sensors: the angles-to-columns maps are bit-identical to before.
Why it matters
The scalar path is reachable from
RowOffsetStructuredSpinningLidarModelParameters.get_vertical_fov, which passesrow_elevations_rad.astype(dtype)[-1]as a float32 scalar. The resultingspan_radshift moves thelinspaceangle grid in_init_angles_to_columns_map, which reassignscKDTreeboundary cells ofangles_to_columns_map.Measured on the two Hesai models, numpy 1.26.4 vs 2.5.2:
The current code is correct only by accident
Today's
relative_anglealready avoids this for those two sensors — but only because their signed difference happens to land in[0, 2*pi), making the modulo a no-op so its dtype cannot matter. On random float32 scalars the reduction still diverges in 44,739 of 60,000 cases.So a sensor whose elevation span wraps the
±piseam, or whose float32 subtraction rounds, would reintroduce the same failure. This PR closes that latent hole rather than a live one.What changed
ncore/impl/data/util.py— derive the float dtype fromangle_rad(falling back toref_angle_rad), coerce the python-scalar reference and the2*piperiod into it, then subtract and reduce once.angle_radmust be a dtype-bearing float type (numpy array/scalar or torch tensor), not a bare python scalar — otherwise there is nothing to pin the reduction to, and the result silently falls back to whatever python/numpy promotion yields. All existing callers already pass numpy arrays, numpy scalars or torch tensors.2*pito6and returning wrong angles, while torch promotes tofloat32and narrows a float64-precision caller. The dtype is read duck-typed (numpy spells float-ness asdtype.kind, torch asdtype.is_floating_point— present since torch 1.12, the oldest version this package supports), so the function stays generic over both without importing torch.ncore/impl/data/util_test.py— four regression tests: float32 scalar/array parity, float64 not narrowed, non-float/non-dtype angles rejected (numpy and torch), and torch float dtypes preserved.Validation
origin/mainangles_to_columns_mapunder numpy 2 + torch 2.13angles_to_columns_mapvs pre-change[0, 1]//ncore/impl/data:all,//ncore/impl/sensors:all,//tools/...bazel run formattyaspectThe float32 array and float64 array paths are untouched.
Notes for reviewers
test_float32_scalar_matches_float32_arraydocstring calls out that theccwdirection is what makes the reduction active — its signed differences are negative and must wrap. Acwdifference already inside[0, 2*pi)passes through the modulo unchanged and would not exercise the period's dtype at all. 63 of 64ccwcases discriminate the period dtype, so the test genuinely guards the fix.refactorrather thanfixbecause there is no user-visible behaviour change on current sensors.Follow-ups (deliberately not in this PR)
Two independent numpy 2 issues found while investigating, to be filed separately:
closest_index_sorted(util.py:66) —abs(value - uint64_scalar)wraps under NEP 50 (silently wrong index) or raisesOverflowErrorfor negative queries. The existing test passes a python list, so it does not catch this._init_angles_to_columns_map(lidar.py:410) —(idxs / n_rows).to(dtype)is exact below 2^24 but truncates incorrectly above it (e.g. 68,928 wrong indices atn_rows=128, n_columns=200000). Version-independent, buttorch.div(..., rounding_mode="floor")would make the line match its comment.