Skip to content

Fix NaN and Inf division by zero on EMU128 and SCALAR - #3511

Open
reginaldalfret wants to merge 1 commit into
google:masterfrom
reginaldalfret:fix/emu128-scalar-div-zero-nan-sign
Open

reginaldalfret wants to merge 1 commit into
google:masterfrom
reginaldalfret:fix/emu128-scalar-div-zero-nan-sign

Conversation

@reginaldalfret

Copy link
Copy Markdown

Summary

In commit 2ed1121 ("emu128, scalar: return +/-Inf on division by zero"), floating-point division by zero was rewritten to avoid division by zero sanitizers:

a.raw = (a.raw == T{0}) ? (inf * T{0}) : ScalarCopySign(inf, a.raw * den);

This introduced two edge-case defects on EMU128 and SCALAR:

  1. When a is NaN, a == T{0} evaluates to false, causing ScalarCopySign(inf, NaN * 0) to convert NaN into +Inf or -Inf instead of preserving NaN.
  2. When a is +Inf, a * den evaluates to Inf * 0 == NaN. ScalarCopySign then extracted an invalid sign bit from the generated NaN, returning -Inf for +Inf / +0 (and -Inf for -Inf / -0).

Solution

  • Check ScalarIsNaN(a) || a == 0 to produce NaN via a + (inf * 0) without dividing by zero.
  • For non-zero and non-NaN values, determine the quotient sign using ScalarSignBit(a) ^ ScalarSignBit(den) and return sign ? -inf : inf.
  • Extended TestFloatDiv in hwy/tests/div_test.cc to test +Inf / +-0, -Inf / +-0, +NaN / +-0, and -NaN / +-0.

Verification

  • Regression test failed before fix: Abort at div_test.cc:241: Assert quot(pos_inf, zero) == pos_inf.
  • All 21 tests in HwyDivTestGroup/HwyDivTest pass across all targets (AVX3_DL, AVX3, AVX2, SSE4, SSSE3, SSE2, EMU128).
  • Standalone HWY_COMPILE_ONLY_SCALAR and HWY_COMPILE_ONLY_EMU128 test binaries both pass all tests.
  • git diff --check passed cleanly.

In commit 2ed1121, division by zero was
handled using ScalarCopySign(inf, a * den).

This had two issues:
1. When a is NaN, a == T{0} evaluates to false, causing ScalarCopySign(inf, ...)
   to convert NaN into +/-Inf instead of preserving NaN.
2. When a is +/-Inf, a * den is (+/-Inf) * 0 which yields NaN. ScalarCopySign
   then extracted an unexpected sign from the NaN, returning -Inf for Inf / +0.

Fix by checking ScalarIsNaN(a) || a == 0 to return NaN without division by zero,
and computing the quotient sign via ScalarSignBit(a) ^ ScalarSignBit(den).
Add regression tests in TestFloatDiv for +/-Inf / +-0 and +/-NaN / +-0.

Signed-off-by: reginaldalfret <reginaldalfret@gmail.com>

@jan-wassenberg jan-wassenberg left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lots of corner cases here. Thanks for fixing.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants