perf(arrow-cast): optimize parsing of decimals from strings

[neilconway]

Which issue does this PR close?

• Closes Spurious failure when casting long strings to decimal #10664
• Closes Incorrect results when casting large strings to Decimal256 #10665
• Closes Optimize parsing decimals from strings #10666

Rationale for this change

The previous implementation of parse_string_to_decimal_native used a string-manipulation approach: it trimmed the input string, split it based on ".", parsed both halves with i256::from_string, converted the result back to a string with format!, and parsed that again with i256::from_string.

Instead, we use a single pass over the input bytes. A simple state machine walks over the input digits, accumulating a running sum. This avoids all of the string manipulation and heap allocation of the previous approach.

We further optimize this by accumulating the running sum in a u64, and then periodically folding that partial value into the running decimal value (we do this often enough that there is no risk of overflowing the u64). That trades a bit of redundant computation for doing more work in u64 and less work in decimal; based on benchmarking, this is a clear win.

Finally, we don't need to accumulate digits from the suffix of the string. Values beyond the target type's scale don't contribute to the result value; only the first such digit influences rounding behavior.

This new approach also fixes two correctness bugs (#10664 and #10665) in the previous implementation.

Benchmarks (M4 Max)

Parser microbenchmarks (arrow-cast/benches/parse_decimal.rs):

• string decimal128 integer: ~111 ns → 15.8 ns, −85.8%
• string decimal128 exact scale: ~109 ns → 15.2 ns, −86.0%
• string decimal128 padded scale: ~116 ns → 15.3 ns, −86.8%
• string decimal128 rounded scale: ~175 ns → 15.7 ns, −91.0%
• string decimal128 signed: 201.8 ns → 15.7 ns, −92.2%
• string decimal128 38 digits: 116.8 ns → 27.2 ns, −76.8%
• string decimal256 76 digits: 174.4 ns → 61.9 ns, −64.5%
• string decimal256 rounded scale: 440.0 ns → 62.3 ns, −85.8%

End-to-end cast kernel (arrow/benches/cast_kernels.rs, 512-row string array, safe mode):

• cast string to decimal128(38, 3): 558 µs → 68.4 µs, −87.0%
• cast string to decimal256(76, 3): 519 µs → 80.1 µs, −84.4%

What changes are included in this PR?

• Rewrite parse_string_to_decimal_native as described above
• Add benchmark coverage
• Add unit tests
• Clarify / extend comments on accepted syntax and rounding behavior
• Replace a Vec + unsafe with PrimitiveBuilder, which saves an allocation
• Improve error message reporting

Are these changes tested?

Yes; existing tests pass, and new tests have been added. I also checked the new implementation against a naive oracle built using the num-bigint crate; the new implementation was consistent with num-bigint for 120M randomly generated inputs.

Are there any user-facing changes?

No, aside from fixed bugs.

AI usage

Iterated with the help of Claude Fable; I reviewed and understand the resulting code.

[Jefffrey]

i wonder if this will fix/affect some of the other issues we have open for decimal parsing/casting, see:

• arrow-cast: decimal parsing tracking issue #10477

edit: seems like none, those other issues are for a different decimal parsing function

[neilconway]

i wonder if this will fix/affect some of the other issues we have open for decimal parsing/casting, see:

• arrow-cast: decimal parsing tracking issue #10477

edit: seems like none, those other issues are for a different decimal parsing function

Yeah, I believe those issues are distinct. Although I think it might make sense to unify some of the code here, both to simplify the implementation and to address some inconsistencies (e.g., rounding vs. truncating when given more decimal digits than required for the target type's scale). I can take a look at some of that once this PR lands.

[alamb]

run benchmark parse_decimal