Skip to content

Replace ANTLR with a generated Parlot parser in the expression language - #375

Merged
lahma merged 18 commits into
spring-projects:mainfrom
sebastienros:feature/parlot-expressions
Nov 4, 2026
Merged

lahma merged 18 commits into
spring-projects:mainfrom
sebastienros:feature/parlot-expressions

Conversation

@sebastienros

@sebastienros sebastienros commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

This replaces the ANTLR 2 parser of Spring.Expressions with a parser generated at build time by the source generator of Parlot, and removes the ANTLR 2.7 runtime that Spring.Core embedded, the generated lexer and parser, the Antlr build target and its Windows-only tool.

The quirks of the grammar and the PEG notes stay in src/Spring/Spring.Core/Expressions/README.md.

What changes

ANTLR 2 Parlot
Runtime dependency 101 files embedded in Spring.Core None: PrivateAssets="all", the parser is generated into the assembly
Build tooling antlr-2.7.6.exe, Windows only Roslyn source generator, which needs the .NET 10 SDK the repository already uses
Expression.Parse, .NET 8 Baseline 8 to 48 times faster on Arm64, 8 to 18 times on x64
Expression.Parse, .NET Framework Baseline 3 to 7 times faster
Allocations per parse 20 KB to 81 KB 536 B to 12 KB
((((1)))) at depth 22 (45 chars) 49 s, doubling with each level 7 ms, including the JIT of the first parse
----1 on a 1 MB stack Process terminated by a stack overflow at 5,000 SyntaxErrorException beyond 2,000 levels

The second to last row is a denial of service: the grammar decides between (a) and (a;b) by parsing the expression with a syntactic predicate and then again, so nesting doubles the time, even for an expression that ends up rejected.

Equivalence

The history of this PR has ParserDifferentialTests (commit eecad8a5, removed when ANTLR could no longer run). It
runs the two parsers side by side, through all four entry points (Parse, ParsePrimary, ParseProperty and
ParseLambda), and compares the printed trees (class, token type, text, children) and the errors. The inputs are 4,204
strings from the test suite and a hand-written list of edge cases, generated expressions covering each grammar rule, and random mutations
of both with a vocabulary of the tokens and their near misses.

  • An early run of 3.9 million parses found 21 differences, all from one lexer rule (. followed by a digit always starts a number,
    even an invalid one), now fixed. Before that, smaller runs found the other lexer and follow-set quirks listed below.
  • The final run: 1,571,952 parses, 615,238 accepted with identical trees, 956,714 rejected by both, no other result.
  • Run again on the final grammar, with the recursion limit left out, it gives the same counts.
  • The rejected ones have the same column in 91% of the cases, and the same column and message in 81%.

ExpressionGrammarTests stays, with the trees and errors ANTLR gave for 216 expressions, chosen for the quirks of the
grammar, written to ExpressionGrammarGolden.json by ANTLR itself. They are compared exactly, for Parse and ParsePrimary: the 61
entries where an error has another position or text hold both, this parser's as the expected one and ANTLR's for reference.

The ANTLR grammar has unusual behaviour that this port keeps deliberately, because it decides what is accepted: a dot between
nodes is optional (a b is a.b, 1 .a and a. parse), T( is one token only without a space, T(...) reads any token
up to ), a real number with a fraction has to have a valid exponent after e while an integer doesn't (1.5ex fails, 1ex
is 1 then ex), Ul is not an integer suffix but lU is, keywords can't be property names, the relational and power
operators don't chain, the line of an error is always 1 and its column counts a tab as moving to the next multiple of 8.

Differences, all deliberate:

  • Positions and messages of syntax errors. An error is reported at the furthest token reached, where ANTLR's LL(2)
    prediction often reports the token before it: for #hope.floats# it reported floats, and now #. Where an error
    named the expected token, such as expecting "RPAREN", found 'x', it is usually now unexpected token: ["x",<33>,line=1,col=5].
    Most lexical errors, and the messages of ParseLambda and ParseProperty are as they were.
  • � in the input. ANTLR took that char for the end of the input and ignored what followed. It is now an
    invalid char.
  • QualifiedIdentifier has no children. ANTLR kept one child per token of the name, which only served to rebuild its
    text; the text is now whole. getText() returns the same value.
  • The nesting limits above. The parser doesn't return why it failed, so at 32 or more open brackets or prefix operators an error reads
    unexpected token or expression nested too deeply: "(", which can also be a plain syntax error at that depth.

Breaking changes

Also in BreakingChanges.txt:

  • The public types of Spring.Expressions.Parser.antlr are gone, and SpringAST has no base class. getFirstChild, getNextSibling, addChild, setFirstChild and setNextSibling use SpringAST instead of AST, and SpringAST(IToken) and the unused members of BaseAST are removed. This needs at least a recompilation for code walking the tree.
  • SyntaxErrorException is now public and derives from Exception: catch it instead of RecognitionException.
  • Too deep an expression is a SyntaxErrorException, and evaluating too deep a tree throws InsufficientExecutionStackException, instead of a stack overflow ending the process.
  • Spring.Core now depends explicitly on System.Memory, which it already had transitively on .NET Framework and .NET Standard 2.0.

Costs to weigh

  • The evaluation has a stack check. BaseNode calls EnsureSufficientExecutionStack() for each node, which also fixes an existing crash on long chains such as 1+1+1+.... Prepared GetValue is about 4% to 21% slower in short runs on Arm64.
  • Spring.Core.dll is 8% larger, 908 KB to 983 KB, as the generated code is larger than what it replaces.
  • The nesting message is a guess. From 32 open brackets or prefix operators, an error reads unexpected token or expression nested too deeply, because the generated parser doesn't return why it failed.

Benchmarks

These figures are from .NET 8 and the .NET Framework job (net472), measured before the rebase onto the 4.0 groundwork. They were not measured again on .NET 10.

test/Spring/Spring.Benchmark/ExpressionBenchmark.cs only uses the public API, so that the same file runs against both versions.
The commit that adds it in this PR (343f26f7) still has ANTLR:

git worktree add ../spring-antlr 343f26f7
cp test/Spring/Spring.Benchmark/*.cs ../spring-antlr/test/Spring/Spring.Benchmark/
# In each checkout:
dotnet run -c Release -f net10.0 --project test/Spring/Spring.Benchmark/Spring.Benchmark.csproj -- --filter '*ExpressionBenchmark*' '*ObjectWrapperBenchmark*' --job short

The expressions: Small is Name.Length + 1 (15 chars), Medium a conditional with two property paths (86 chars), Large seven terms with selections,
projections, list and map initializers, a type, a constructor, a lambda and an expression list (491 chars).

BenchmarkDotNet v0.15.2, --job short, one launch of each version, two machines:

  • Arm64: macOS 15.8.2, Apple M4 Pro, .NET 8.0.30, Arm64 RyuJIT, with other work running (load average 5 to 18). Two rounds, the lower of the two for each cell,
    as one measurement of the Parlot Large parse was disturbed (7.5 µs and 16.8 µs, with a deviation of 7.5 µs).
  • x64: Windows 11 (Hyper-V), AMD EPYC 7763, .NET 8.0.31 and .NET Framework (net472 job), one round. This machine has shown
    differences of up to 27% between runs of identical code: read the ratios as orders of magnitude, not as precise values.

Parsing only

Expression.Parse, mean time and allocations:

Size ANTLR Parlot Faster Fewer allocations
Small Arm64, .NET 8 1,922 ns, 20,785 B 216 ns, 536 B 8.9x 39x
x64, .NET 8 4,357 ns, 20.3 KB 550 ns, 536 B 7.9x 39x
x64, .NET Framework 5,541 ns, 20.4 KB 1,695 ns, 546 B 3.3x 38x
Medium Arm64, .NET 8 53.2 µs, 28,770 B 1.13 µs, 2,040 B 47x 14x
x64, .NET 8 49.1 µs, 28.1 KB 2.81 µs, 2,040 B 17x 14x
x64, .NET Framework 59.9 µs, 28.7 KB 8.42 µs, 2,086 B 7.1x 14x
Large Arm64, .NET 8 363 µs, 81,181 B 7.5 µs, 12,088 B 48x 6.7x
x64, .NET 8 352 µs, 79.3 KB 19.2 µs, 12,088 B 18x 6.7x
x64, .NET Framework 418 µs, 83.0 KB 60.3 µs, 12,300 B 6.9x 6.9x

Most of ANTLR's 20 KB for the smallest expression is its lexer and token buffers, created per parse. A parse on .NET Framework is
about three times slower than on .NET 8, as it reads through the System.Memory package.

The layers above

ExpressionEvaluator.GetValue(root, expression) parses each time, with no cache. The first evaluation of a parse also does its reflection
lookups and creates dynamic methods, which dominate the Large expression, so its time depends less on the parser.

GetValue, uncached ANTLR Parlot Faster
Small Arm64, .NET 8 2,139 ns, 21,193 B 421 ns, 944 B 5.1x
x64, .NET 8 4,619 ns, 20.7 KB 992 ns, 944 B 4.7x
x64, .NET Framework 7,222 ns, 20.9 KB 2,978 ns, 963 B 2.4x
Medium Arm64, .NET 8 70.7 µs, 35,523 B 17.3 µs, 8,792 B 4.1x
x64, .NET 8 74.8 µs, 34.9 KB 23.6 µs, 9,008 B 3.2x
x64, .NET Framework 128 µs, 38.3 KB 65.3 µs, 11,882 B 2.0x
Large Arm64, .NET 8 615 µs, 104 KB 270 µs, 35,348 B 2.3x
x64, .NET 8 1,441 µs, 102 KB 1,042 µs, 35,563 B 1.4x
x64, .NET Framework 2,176 µs, 115 KB 1,678 µs, 44,753 B 1.3x

Expression.Parse then 100 GetValue, as the documentation recommends. The cost of the parse is spread over the evaluations, and
what remains is the first evaluation, so the gain shrinks as expected:

Parse + 100 GetValue ANTLR Parlot Faster
Small Arm64, .NET 8 6,984 ns 5,220 ns 1.3x
x64, .NET 8 15.3 µs 11.2 µs 1.4x
x64, .NET Framework 24.6 µs 19.5 µs 1.3x
Medium Arm64, .NET 8 154 µs 90.5 µs 1.7x
x64, .NET 8 299 µs 216 µs 1.4x
x64, .NET Framework 520 µs 453 µs 1.1x
Large Arm64, .NET 8 848 µs 458 µs 1.9x
x64, .NET 8 1,976 µs 1,527 µs 1.3x
x64, .NET Framework 3,005 µs 2,459 µs 1.2x

A prepared GetValue doesn't depend on the parser, but it pays for the stack check BaseNode now does. Arm64, .NET 8, ANTLR then Parlot without
the check then with it, one short run of each: Small 49 ns, 50 ns, 56 ns; Medium 816 ns, 749 ns, 806 ns; Large 2,042 ns, 1,972 ns, 2,382 ns, with the
same allocations. The last run gave 57 ns, 781 ns and 2,354 ns. So about 4% to 21%, within the noise of this machine for the smallest gaps. An application that evaluates prepared expressions in a
hot loop gives back that much, for not crashing on a deep tree.

The tables above were measured before the last fixes, which added the check, the guard on named arguments and a Then closing each primary expression,
with the same allocations. One new run on Arm64 is 5% to 12% slower at parsing than the tables (Small 243 ns, Medium 1,265 ns, Large 8,616 ns). That is within
what this machine varies under load, though it wasn't isolated, and the ratios to ANTLR stay at 8x to 42x there. x64 wasn't measured again.

ObjectWrapper.GetPropertyValue parses the name on each call. Name takes the path that only reads a name (ParseProperty), Friends[1].Address.City the one
that parses a primary expression (ParsePrimary):

ANTLR Parlot Faster Fewer allocations
Name Arm64, .NET 8 1,142 ns, 19.3 KB 84 ns, 240 B 14x 82x
x64, .NET 8 2,241 ns, 19.3 KB 173 ns, 240 B 13x 82x
x64, .NET Framework 3,342 ns 477 ns, 249 B 7.0x
Friends[1].Address.City Arm64, .NET 8 2,643 ns, 21.6 KB 617 ns, 1,456 B 4.3x 15x
x64, .NET 8 5,812 ns, 21.6 KB 1,402 ns, 1,456 B 4.1x 15x
x64, .NET Framework 8,939 ns 4,011 ns, 1,492 B 2.2x

Tests

Spring.Core.Tests, where ExpressionEvaluatorTests only compiles on .NET Framework:

Before After
net462, Windows 2,741 passed, 0 failed 2,773 passed, 0 failed
net8.0, macOS 2,458 passed, 5 failed 2,490 passed, 5 failed

The 5 failures are the same on both sides and don't involve expressions: formatters that depend on the culture of the machine. These figures predate the rebase onto the 4.0 groundwork. After it and the review, net10.0 on macOS has 2,498 passed and the same 5 failures. net472, which has ExpressionEvaluatorTests, was compiled but not run locally, so CI is its first run.

Parlot gaps

Nothing here blocked the port. Parlot 2.2.1:

  1. net462 is refused (PARLOT026, floor 4.7.2), and the package's netstandard reference doesn't resolve on its reference assemblies, while the generated code works there. This took four
    MSBuild targets until the 4.0 floor moved to net472. Same finding as Lucene.NET.
  2. PARLOT021 is misleading when a call doesn't bind. If(() => rule == x, a, If(...)) with parsers whose types made inference fail reported "may only be read inside supported parse-time callbacks"
    rather than the inference error, as the check requires a bound argument. Giving the type, If<BaseNode>(...), fixed it.
  3. Unary can't mix operator types, although docs/parsers.md shows .Unary((Terms.Char('-'), ...), (Terms.Text("not"), ...)): Unary<TInput>(params ...) can't be inferred.
  4. A scanner is called with an empty span at the end of the input. The docs of Scan don't say so, and a scanner reading span[0] throws IndexOutOfRangeException.
  5. No non-associative operator level (a == b == c and a ^ b ^ c have to fail), and no last operand of Ternary that is a whole expression. These force a chain of tables and sequences here.
  6. No way to commit to an alternative without losing the failure position. ElseError reports where it was reached, not the furthest failure, so a choice that must not backtrack takes a negative lookahead (Not(lparen)).
  7. An entry point can't return where it stopped. A rule without Eof needs When((context, x) => ...) reading context.Scanner.Cursor to check what follows.
  8. The stack per level is high before tier-up: in a 1 MB thread, between 200 and 400 levels of parentheses on a first parse in Release, and 96 in the Debug test host. It is far more than ANTLR could reach in practice, and it fails cleanly.
  9. The generated code is large: the assembly grows by 75 KB even though 101 ANTLR files and 6,000 generated lines go away. Parsers used twice (methodOrProperty, arguments) didn't look shared across the choices that use them; this wasn't investigated further.
  10. The failure overload doesn't say why a parse failed. Reaching MaxRecursionDepth or the end of the stack returns false and a position, like a syntax error. The diagnostics overload has the reason but generates the grammar twice. So the message here is a guess from the depth at the failure: from 32 open brackets or prefix operators it says unexpected token or expression nested too deeply, including for a plain typo at that depth, and a real stack failure below it reads as an unexpected token.

@sebastienros
sebastienros marked this pull request as ready for review November 3, 2026 00:50
@sebastienros

Copy link
Copy Markdown
Contributor Author

@lahma

@sebastienros
sebastienros force-pushed the feature/parlot-expressions branch from 29e839c to e5ec63f Compare November 3, 2026 02:00
@lahma

lahma commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

@sebastienros we're planning to take this into 4.0. Before it goes in, I'll merge a smaller PR that raises the supported frameworks, so this branch can lose its net462 workarounds instead of shipping them:

  • The .NET Framework floor goes from 4.6.2 to 4.7.2. TargetFullFrameworkVersion becomes net472 in both src/ and test/ Directory.Build.props.
  • The modern test target goes from net8.0 to net10.0, because .NET 8 and 9 are leaving support. Library targets stay netstandard2.0;net472.

After you rebase onto it, this PR should be able to drop:

  • The four _Spring* MSBuild targets in Spring.Core.csproj. With net472, PARLOT026 is satisfied and the netstandard.dll facade is part of the 4.7.2 reference assemblies.
  • The System.ValueTuple reference and its PackageVersion, since ValueTuple is built into 4.7+. The BreakingChanges.txt line about it also goes.

System.Memory stays: neither net472 nor netstandard2.0 includes it.

The CoreRuntime.Core80 job in FrameworkAndCoreConfig would move to .NET 10 like the rest. I'll link the framework PR here once it's up.

@lahma

lahma commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

The framework PR is up: #376 (.NET Framework 4.7.2 floor, net10.0 test target). Once it merges, rebasing this branch should let the four _Spring* targets and System.ValueTuple go.

@lahma

lahma commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

@sebastienros the 4.0 groundwork is now on main (#376 to #383), so this branch can rebase. Here's what changes for it.

  • .NET Framework floor is 4.7.2 (Start 4.0: .NET Framework 4.7.2 floor, .NET 10 tests, preview version and agent definitions #376). The four _Spring* targets in Spring.Core.csproj and System.ValueTuple can go.
  • Spring.Core also targets net10.0 (Add a net10.0 target to the libraries and remove code access security #382), as netstandard2.0;net472;net10.0, so the generator now runs for three TFMs.
    • System.Memory is only needed on netstandard2.0 and net472.
    • Framework conditions are keyed on '$(TargetFrameworkIdentifier)' == '.NETFramework' / NETFRAMEWORK, never "not netstandard".
  • Legacy serialization on net10.0 (Add a net10.0 target to the libraries and remove code access security #382): code access security is gone, and on net10.0 every (SerializationInfo, StreamingContext) constructor and GetObjectData carries [Obsolete(... DiagnosticId = "SYSLIB0051")] under #if NET, using the constants in src/Spring/Obsoletions.cs.
    • If the now-public SyntaxErrorException keeps a serialization constructor, it needs the same annotation, or the net10.0 build fails with SYSLIB0051.
  • Tests run on NUnit 5 + Microsoft.Testing.Platform, on net10.0 and net472 (Move the tests to NUnit 5 on Microsoft.Testing.Platform, and MSTest 4 #383).
    • Every classic assert in the repo was converted to the constraint model (Assert.That), so ExpressionGrammarTests and the other new tests need that style too.
    • Run them with dotnet test --project <csproj>.
    • The Core80 job in FrameworkAndCoreConfig should move to .NET 10.
  • BreakingChanges.txt now has 4.0 items 1–9. Please add yours as 10 onwards.
  • Legacy build-support/ content is gone, apart from tools/antlr-2.7.6, which this PR deletes. No conflict is expected there.

@lahma

lahma commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

@sebastienros, a short addendum to my rebase note. Four more PRs merged since then (#384–#387), and two of them affect this branch:

The rest of the earlier note still applies: BreakingChanges numbering for this PR starts at 10.

🤖 Generated with Claude Code

@lahma

lahma commented Nov 3, 2026

Copy link
Copy Markdown
Collaborator

@sebastienros, one more small rebase note. #389 took BreakingChanges item 10 (LogExceptionHandler), so this PR's entries now start at 11. Everything else in the earlier notes still applies.

🤖 Generated with Claude Code

Parsing alone, an uncached ExpressionEvaluator.GetValue, parsing then evaluating, and the
property paths of ObjectWrapper, through the public API only so that the same file can
compare two versions of Spring.Core.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
ExpressionGrammar builds the same nodes from a grammar that the Parlot source generator
turns into code of this assembly, without a runtime dependency on Parlot. Nothing uses it
yet: ParserDifferentialTests compares the two on the strings of the test suite, on
generated expressions and on mutations of both. SyntaxErrorException becomes public, as
the exception to catch once ANTLR's is gone.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
ExpressionGrammarTests keeps what the differential tests showed: the trees and errors of the
ANTLR parser on 216 expressions, from a file they wrote, and the cases where it failed,
such as the exponential time it took on nested parentheses. The last run of the differential
test compared 1,572,000 parses: 615,238 accepted with the same tree and 956,714 rejected by
both, 81% of them with the same message and position, 91% with the same position.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Nodes keep the child and sibling links, method names and serialized form they had from
antlr.BaseAST, now in SpringAST itself, and SyntaxErrorException derives from Exception.
The generated parser, the embedded ANTLR 2 runtime, the tool and its build target go.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Just reading the [ClrJob] attribute throws off Windows, and BenchmarkDotNet reads those of the
whole assembly.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
… and gaps

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
- A failed named argument isn't read again as an assignment, which took exponential time.
- Appending to a primary expression keeps its tail, as long paths took quadratic time.
- BaseNode checks the stack when it evaluates a node, as a deep tree could overflow it.
- A syntax error doesn't parse the input again, and says when nesting can be its cause.
- Tests cover the primary, property and lambda rules, and these cases.
- The follow sets name their tokens, the helpers building nodes are fewer, and
  SyntaxErrorException has documentation.

The differential test of 40b4398, run again on this grammar, gives the same result: 615,238
accepted with identical trees and 956,714 rejected by both.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
- BaseNode also checks the stack when it sets a value.
- A primary expression keeps its tail in its child links while it is read, rather than in a field.
- The scanners of the tokens that start a longer one read them with NextToken.
- The golden file has the errors of this parser, compared exactly, and is an embedded resource.
- The tests about the time parsing takes fail when it takes too long.
- The benchmarks only ask for .NET Framework on Windows.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Move the breaking changes to the 4.0 items, suppress the API differences of
removing the ANTLR runtime, and benchmark .NET 10. A 128 KB stack is less than
the parser needs on Windows, where it failed before the tree could be evaluated.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
@sebastienros
sebastienros force-pushed the feature/parlot-expressions branch from e5ec63f to f7d09b6 Compare November 3, 2026 23:10
@sebastienros

Copy link
Copy Markdown
Contributor Author

Rebased onto main (66454bb), with these changes from your notes:

  • The four _Spring* targets and System.ValueTuple are gone. System.Memory is only referenced on netstandard2.0 and net472.
  • The breaking changes are now item 11 of the 4.0 list.
  • CompatibilitySuppressions.xml is regenerated. Spring.Core packs against the 3.1.0 baseline, and the new entries are the antlr types, SpringAST(IToken) and the node types that lose the AST interface.
  • SyntaxErrorException has the SYSLIB0051 annotations, and the benchmark job moved to .NET 10.

I also changed EvaluatingTooDeepATreeThrows, which failed on build-windows in the previous run: on its 128 KB stack, the parser appears to run out of room on Windows before the tree can be evaluated. It now uses 256 KB, which this CI run will confirm.

On macOS, Spring.Core.Tests on net10.0 has 5 failures, all in the culture-dependent formatter tests. net472 compiles but I couldn't run it locally, so this CI run is its first test on Windows, including ExpressionEvaluatorTests.

@lahma lahma left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sebastienros, thanks for the rebase. It's in good shape. The review compared the Parlot grammar rule by rule with the old Expression.g and found no semantic differences: precedence, call predicates, T(, @(, new, initializers, lexing and suffixes all match. The golden file's coverage of ANTLR's quirks is excellent, and the DoS and stack-overflow hardening is a real improvement. Every point from the rebase notes was applied. Before merge, there are four must-fixes, two small should-fixes, and some items I'll handle myself after merge.

Must-fix before merge

  1. Windows CI fails in CompileExamples with NU1109. The examples pin transitive packages (CentralPackageTransitivePinningEnabled in examples/Directory.Build.props, added in #381 for a CS1705). That pins the new central System.Memory 4.5.5, but NUnit3TestAdapter → … → System.Collections.Immutable 10.0.0 needs System.Memory ≥ 4.6.3. Keep 4.5.5 as the library floor: MEL.Abstractions 8.0.3 already brings it to consumers. Override it only for the examples, with a new examples/Directory.Packages.props:

    <Project>
      <Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Packages.props', '$(MSBuildThisFileDirectory)../'))" />
      <ItemGroup>
        <!-- The examples pin transitive packages, and their test adapters need a newer System.Memory than Spring's 4.5.5 floor. -->
        <PackageVersion Update="System.Memory" Version="4.6.3" />
      </ItemGroup>
    </Project>
  2. .fallout/build.schema.json:27 still lists "Antlr". That file is committed and kept in sync, and the next build rewrites it and leaves the tree dirty. Remove the entry.

  3. Move PARLOT_PORT.md out of the repo root. It's a PR narrative that is already going stale:

    • It cites 25c6a0de, which no longer exists after the rebase.
    • It says -f net8.0, but benchmarks now target net472;net10.0.
    • Its figures are from net462 and .NET 8.

    Put "What changes", Equivalence, Benchmarks and "Parlot gaps" in the PR description, since that becomes the squash commit body. Keep the maintainer-relevant design notes (deliberate grammar quirks, PEG vs LL(2)) as a short src/Spring/Spring.Core/Expressions/README.md next to the grammar, and point AGENTS.md at it.

  4. BreakingChanges item 11 needs to be complete, because package validation's suppressions now list the removals:

    • Name the four removed namespaces: antlr, antlr.debug, antlr.collections, antlr.collections.impl.
    • List what SpringAST lost: the protected down/right fields, ICloneable/Clone, SpringAST(IToken)/initialize(IToken), and the inherited BaseAST helpers (Equals*, findAll*, ToTree, xmlSerialize*, encode/decode, setVerboseStringConversion).
    • The sentence "their trees are the same" isn't accurate for QualifiedIdentifier, which no longer has child nodes; only getText() is the same.
    • Optionally, mention that � in the input is now rejected. ANTLR treated it as end of input and silently dropped the rest.

Should fix in this PR (small)

  1. Contradictory error message for incomplete input. ExpressionGrammar.cs:72 sets expected = "EOF" even when the failure is at the end of input. So a?b, new a and new a.b report expecting "EOF", found '<EOF>', and the golden file freezes that at lines 341-342. Adding && offset < input.Length makes these fall through to unexpected end of expression. That needs three golden updates.
  2. SyntaxErrorException serialization. The serialization constructor and GetObjectData skip base(info, context)/base.GetObjectData. That was only needed while it derived from RecognitionException. It's public now and its serialized shape freezes with 4.0, and as written it drops InnerException, the stack trace and HResult. Call the base members and drop the custom "Message" key. Small style point: line/column should be _line/_column.

After merge, on my side

  • Measure the cost of EnsureSufficientExecutionStack per node evaluation on net472. On prepared expressions on Arm64 .NET 8 it costs 4–21%. If it matters, check depth every N levels instead.
  • Measure and document the supported nesting depth on a 256 KB thread (IIS x86).
  • ExpressionEvaluatorTests is Compile Removed on net10.0 because of its BinaryFormatter parts. I'll split those out so the main expression suite also runs the new parser on .NET 10.
  • 5L parses and then IntLiteralNode throws; this bug predates the PR.
  • Nits: ExpressionBenchmark field naming, a blank line in StandardsComplianceTest.cs:77, and the fully qualified RuntimeHelpers in BaseNode.
  • The Parlot gaps you listed are probably better tracked as Parlot issues.

Ping me when it's updated and I'll merge once CI is green.

🤖 Generated with Claude Code

…essions

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
The examples pin transitive packages, and their test adapters need a newer
System.Memory than the floor of the libraries. The Antlr target is gone from the
schema, which the next build would rewrite.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
An incomplete expression said it expected the end of the input while being at it.
SyntaxErrorException called the base members of serialization only while it
derived from RecognitionException: without them it loses its stack trace and HResult.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
@sebastienros

Copy link
Copy Markdown
Contributor Author

@lahma updated, as a merge of main rather than another rebase. All six points are in:

  1. examples/Directory.Packages.props raises System.Memory to 4.6.3 for the examples only. Primes.Tests restores on macOS.
  2. Antlr is out of .fallout/build.schema.json.
  3. PARLOT_PORT.md is gone. The PR description has What changes, Equivalence, Benchmarks and Parlot gaps, and src/Spring/Spring.Core/Expressions/README.md has the quirks and PEG notes, linked from AGENTS.md. The benchmark figures are still the earlier .NET 8 and net472 ones, labelled as such.
  4. BreakingChanges item 11 names the four namespaces and what SpringAST lost, corrects the QualifiedIdentifier sentence and mentions U+FFFF.
  5. Incomplete input now reports unexpected end of expression, with the three golden updates and a test.
  6. SyntaxErrorException calls the base serialization members, drops the custom Message key and uses _line/_column, with a round-trip test.

On macOS, Spring.Core.Tests on net10.0 has 2,498 passed and the same 5 culture-dependent formatter failures. net472 compiles but hasn't run: the last Windows run stopped at CompileExamples, so this CI run is the first to test it, including the 256 KB stack change and ExpressionEvaluatorTests.

…hanges

PARLOT_PORT.md moved to the description of the pull request, apart from the
quirks and the PEG notes, which maintainers of the grammar need.

Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
@lahma

lahma commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

@sebastienros, thanks, every point is addressed. I approved the CI run, since the branch is on a fork. Compile and examples now pass on Windows. One test fails on the Windows leg, Release build, net472:

failed EvaluatingTooDeepATreeThrows(1900,"-")
  Assert.That(exception, Is.TypeOf<InsufficientExecutionStackException>())
  Expected: <System.InsufficientExecutionStackException>
  But was:  null

With JIT optimizations on .NET Framework x64, 1,900 nested unary minuses evaluate within the 256 KB thread, so nothing throws. The parser caps prefix operators and parentheses at 2,000 levels, so a unary case can't go deep enough to be reliably too deep for evaluation in every configuration. The (100_000, "1+") case doesn't have that limit, and it's the one that really proves "too deep to evaluate throws instead of crashing". I'd drop the (1_900, "-") case. The 2,000-level parse limit is already covered on the parsing side. The alternative is to find a depth that fails on net472 Release too, but that stays fragile.

Once that's in and CI is green, I'll merge.

🤖 Generated with Claude Code

On the Windows leg's Release net472 run, 1,900 nested unary minuses
evaluated within a 256 KB stack, so the case got no
InsufficientExecutionStackException and failed. The parser caps prefix
operators at 2,000 levels, which DeepNestingOfUnaryOperatorsIsASyntaxError
covers, so the case can't be made to overflow reliably on every runtime.
The 100,000-term chain keeps the evaluation-side check.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01LGkLQuAuUjvRjrU1AcTsmk
Signed-off-by: Marko Lahma <marko.lahma@gmail.com>
@lahma

lahma commented Nov 4, 2026

Copy link
Copy Markdown
Collaborator

@sebastienros, I pushed 32894f6 to your branch. It removes only the [TestCase(1_900, "-")] line. The 2,000-level cap on prefix operators is already covered by DeepNestingOfUnaryOperatorsIsASyntaxError. The (100_000, "1+") case keeps checking that evaluating a tree that is too deep throws. I approved the CI run, and I'll merge once it's green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01LGkLQuAuUjvRjrU1AcTsmk

@lahma lahma left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All the review points are addressed, and CI is green on build-windows, build-linux, integration-linux and DCO at 32894f6. Thanks, @sebastienros!

@lahma
lahma merged commit 98db0e0 into spring-projects:main Nov 4, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants