Skip to content

cpu: raise the FP exceptions IRIX's diagnostics check for - #406

Draft
jamesbraid wants to merge 3 commits into
techomancer:mainfrom
jamesbraid:ide-fpu-fix
Draft

jamesbraid wants to merge 3 commits into
techomancer:mainfrom
jamesbraid:ide-fpu-fix

Conversation

@jamesbraid

@jamesbraid jamesbraid commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

IRIX programs and SGI's diagnostics that check FCSR or NaNs got the wrong answer in iris, because it raised the wrong FP exceptions and used modern IEEE NaNs. The Octane PROM's ide FPU test failed 2 of its 14 checks: 0/0 raised divide-by-zero instead of Invalid, and an overflowing add left FCSR empty. Now all 14 pass, under the interpreter and jitv2.

  • ADD, SUB, MUL and DIV raise Invalid, and Overflow plus Inexact. Results at the edge of the range go through one cold path both engines share. Normal results pay one compare, which replaces an existing one.
  • NaNs use the original MIPS encoding: a quiet NaN has the top fraction bit clear. iris's own R4400 hardware results in cpu-tests already showed this. Invalid results return the MIPS default NaN.
  • SQRT, RECIP, RSQRT, the MADD family and CVT.S.D return the same default NaN and raise Invalid for cases like inf×0 and sqrt(−1). This costs madd.s 60→66 host instructions under the JIT.
  • Fixes a JIT bug on main: with FCSR.FS set, compiled code trapped on denormals instead of flushing them to zero.

The FPU code is shared, so the R4400 and R5000 change too.

Testing

  • ide fpu: 14/14 under jitv2 and the interpreter. It was 12/14.
  • cargo test --release --lib --features jitv2: 2892 passed. Each fix has a test that fails without it.

The R4400, R5000 and R10000 use the original MIPS NaN encoding: a NaN
with the fraction MSB set is signalling, one with it clear is quiet. The
default NaN is 0x7FBFFFFF for that reason. iris used the IEEE 754-2008
convention, the reverse, in the interpreter and in compiled code.

The evidence for the original encoding is the NEC VR10000 instruction
manual, U12754E, first edition, August 2000, printed page 226, table
2-2, and the results of iris's own R4400 in cpu-tests: fpu/qnan_operand
feeds 0x7FC00000 to ADD and gets Invalid and the default NaN, not
Unimplemented, and the signalling-predicate compares agree. Upstream
issue 361 asked about it.

Swap the sense of the fraction-MSB test in is_snan and is_qnan and in
their compiled twins, including the compare path, and make the default
NaN 0x7FBFFFFF and 0x7FF7FFFF_FFFFFFFF. With the bit set the fraction
is nonzero by construction, so the interpreter's signalling test
shrinks to one mask and compare, and an ordinary operand skips the
subnormal and quiet-NaN checks after one exponent test. Compiled code
swaps a comparison and costs the same.

The tests that fix a NaN's bit pattern move with this commit. The new
ones cover the default NaN as an operand (Unimplemented, no Invalid), a
signalling NaN operand (Invalid, trapped and untrapped), and a
compiled-versus-interpreter check over ADD, MUL, DIV, SQRT and the
quiet and signalling compare predicates. Without this commit the rest
of the series still builds and passes its tests; the Octane PROM's
ide test then rejects the default NaN's bit pattern.
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from ad267e6 to c968e9c Compare November 2, 2026 15:47
@jamesbraid
jamesbraid marked this pull request as draft November 2, 2026 16:29
@jamesbraid
jamesbraid force-pushed the ide-fpu-fix branch 2 times, most recently from 4474a7f to 4bce574 Compare November 2, 2026 18:54
The Octane PROM's ide FPU test checks results the host FPU gets wrong
for MIPS: infinity minus infinity, zero times infinity, 0/0 and
infinity/infinity are Invalid Operations that deliver the default NaN
(0x7FBFFFFF), 0/0 is not a divide by zero, and MAX+MAX is Overflow with
Inexact. iris returned the host's NaN with no flags. 12 of the 14
subtests passed.

Classify those results on every CPU model, in both engines, at no cost
to an ordinary result.

ADD, SUB, MUL and DIV test the result's magnitude once. That test
replaces the old denormal-result test, and the top of the range, which
is where Invalid and Overflow live, goes to a cold classifier shared by
both engines. The operand flags those ops used to compute up front are
gone: an ordinary result proves there were none, and the classifier
finds a divide by zero itself.

Compiled code reaches the classifier through one call per instruction.
The same call takes a denormal, infinite or NaN operand and a denormal
result, where the interpreter's operand checks and denormal rule run, so
the instruction carries one call and one trap check, not a copy of those
rules at each of two exits. Zero, as an operand or a result, is common
in real code and stays inline. Generated code for the page holding
Whetstone and its transcendental kernel comes to 247 bytes per guest
instruction, against 404 before this series.

SQRT, RECIP, RSQRT, the multiply-add family and CVT.S.D swap the host's
NaN for the default NaN and add V. They do it the same way round: one compare of the result's
magnitude, and only an infinity or a NaN goes on to classify the
operands. A finite result cannot come from a NaN operand, a zero
divisor or a negative square root, so the signalling-NaN tests on up to
three operands no longer run for ordinary arithmetic. CVT.S.D checks its
source for a signalling NaN only after the conversion has already left
the exact path, since a NaN is never equal to itself.

Host instructions per FP op for this and the next commit together,
perf stat instructions:u, ordinary operands, ship profile, against
the commit before this series (slope
over a 120-op region, three alternating rounds, counts do not move
between rounds). R10000 numbers; R5000 and R4400 differ by a constant:
  compiled add.d, mul.d, div.d                    36 -> 35
  compiled sqrt.s, cvt.s.d                        no change
  compiled madd/msub/nmadd/nmsub .s and .d        56-63 -> 25-31
  compiled recip.s, rsqrt.s                       43 -> 25, 51 -> 27
  interpreter add.d, mul.d, div.d                 197-200 -> 171
  interpreter madd/msub/nmadd/nmsub               168-181 -> 153-159
  interpreter recip.s, rsqrt.s                    145 -> 136, 153 -> 137
  interpreter sqrt.s, cvt.s.d                     no change
Every measured op, in both engines, on the R4400, R5000 and R10000, is
within 0.1 instruction of the old count or cheaper.

The tests that check results run on the ordinary passthrough models in
both engines, and compiled code is checked against the interpreter for
each rule. This commit delivers the IEEE 754-2008 default NaN, 0x7FC00000. The
Octane's R10000 delivers 0x7FBFFFFF; the next commit moves the NaN
encoding, and with it the ide test's NaN checks pass.
With FS set and neither Underflow nor Inexact enabled, an R10000
replaces a denormal ADD/SUB/MUL/DIV/SQRT result with a signed zero and
sets Cause and Flag U|I. The interpreter does that. Compiled code
trapped with Unimplemented Operation instead, for every denormal
result, whatever FS said.

The trap decision negated the "FS is set" compare with bnot. A
Cranelift compare yields 0 or 1, and bnot(1) is 0xFE, which still
reads as true, so "FS clear or U|I enabled" was always true. The same
mistake in the round-to-integer path was fixed earlier. This one uses
a compare against zero too.

The new equivalence test halves the smallest normal number in both
formats with FS set, FS clear and FS set with Underflow enabled, and
checks the compiled result against the interpreter and the manual.

The trap and flush blocks are marked cold too. They hang off the cold
denormal block but were not cold themselves, so Cranelift laid them out
among the hot instructions of every FP op. On the benchmark suite that
cost fpu/linpack its fast mode in more runs (median 565 against 619
MIPS on main) and left fpu/transcend about 1% behind main; with the
blocks cold linpack matches main and transcend is 8% ahead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant