Repository navigation
Replace ANTLR with a generated Parlot parser in the expression language - #375
Conversation
29e839c to
e5ec63f
Compare
|
@sebastienros we're planning to take this into 4.0. Before it goes in, I'll merge a smaller PR that raises the supported frameworks, so this branch can lose its
After you rebase onto it, this PR should be able to drop:
The |
|
The framework PR is up: #376 (.NET Framework 4.7.2 floor, |
|
@sebastienros the 4.0 groundwork is now on
|
|
@sebastienros, a short addendum to my rebase note. Four more PRs merged since then (#384–#387), and two of them affect this branch:
The rest of the earlier note still applies: BreakingChanges numbering for this PR starts at 10. 🤖 Generated with Claude Code |
|
@sebastienros, one more small rebase note. #389 took BreakingChanges item 10 ( 🤖 Generated with Claude Code |
Parsing alone, an uncached ExpressionEvaluator.GetValue, parsing then evaluating, and the property paths of ObjectWrapper, through the public API only so that the same file can compare two versions of Spring.Core. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
ExpressionGrammar builds the same nodes from a grammar that the Parlot source generator turns into code of this assembly, without a runtime dependency on Parlot. Nothing uses it yet: ParserDifferentialTests compares the two on the strings of the test suite, on generated expressions and on mutations of both. SyntaxErrorException becomes public, as the exception to catch once ANTLR's is gone. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
ExpressionGrammarTests keeps what the differential tests showed: the trees and errors of the ANTLR parser on 216 expressions, from a file they wrote, and the cases where it failed, such as the exponential time it took on nested parentheses. The last run of the differential test compared 1,572,000 parses: 615,238 accepted with the same tree and 956,714 rejected by both, 81% of them with the same message and position, 91% with the same position. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Nodes keep the child and sibling links, method names and serialized form they had from antlr.BaseAST, now in SpringAST itself, and SyntaxErrorException derives from Exception. The generated parser, the embedded ANTLR 2 runtime, the tool and its build target go. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Just reading the [ClrJob] attribute throws off Windows, and BenchmarkDotNet reads those of the whole assembly. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
… and gaps Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
- A failed named argument isn't read again as an assignment, which took exponential time. - Appending to a primary expression keeps its tail, as long paths took quadratic time. - BaseNode checks the stack when it evaluates a node, as a deep tree could overflow it. - A syntax error doesn't parse the input again, and says when nesting can be its cause. - Tests cover the primary, property and lambda rules, and these cases. - The follow sets name their tokens, the helpers building nodes are fewer, and SyntaxErrorException has documentation. The differential test of 40b4398, run again on this grammar, gives the same result: 615,238 accepted with identical trees and 956,714 rejected by both. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
- BaseNode also checks the stack when it sets a value. - A primary expression keeps its tail in its child links while it is read, rather than in a field. - The scanners of the tokens that start a longer one read them with NextToken. - The golden file has the errors of this parser, compared exactly, and is an embedded resource. - The tests about the time parsing takes fail when it takes too long. - The benchmarks only ask for .NET Framework on Windows. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
Move the breaking changes to the 4.0 items, suppress the API differences of removing the ANTLR runtime, and benchmark .NET 10. A 128 KB stack is less than the parser needs on Windows, where it failed before the tree could be evaluated. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
e5ec63f to
f7d09b6
Compare
|
Rebased onto
I also changed On macOS, |
lahma
left a comment
There was a problem hiding this comment.
@sebastienros, thanks for the rebase. It's in good shape. The review compared the Parlot grammar rule by rule with the old Expression.g and found no semantic differences: precedence, call predicates, T(, @(, new, initializers, lexing and suffixes all match. The golden file's coverage of ANTLR's quirks is excellent, and the DoS and stack-overflow hardening is a real improvement. Every point from the rebase notes was applied. Before merge, there are four must-fixes, two small should-fixes, and some items I'll handle myself after merge.
Must-fix before merge
-
Windows CI fails in
CompileExampleswith NU1109. The examples pin transitive packages (CentralPackageTransitivePinningEnabledinexamples/Directory.Build.props, added in #381 for a CS1705). That pins the new centralSystem.Memory4.5.5, but NUnit3TestAdapter → … → System.Collections.Immutable 10.0.0 needsSystem.Memory≥ 4.6.3. Keep 4.5.5 as the library floor: MEL.Abstractions 8.0.3 already brings it to consumers. Override it only for the examples, with a newexamples/Directory.Packages.props:<Project> <Import Project="$([MSBuild]::GetPathOfFileAbove('Directory.Packages.props', '$(MSBuildThisFileDirectory)../'))" /> <ItemGroup> <!-- The examples pin transitive packages, and their test adapters need a newer System.Memory than Spring's 4.5.5 floor. --> <PackageVersion Update="System.Memory" Version="4.6.3" /> </ItemGroup> </Project>
-
.fallout/build.schema.json:27still lists"Antlr". That file is committed and kept in sync, and the next build rewrites it and leaves the tree dirty. Remove the entry. -
Move
PARLOT_PORT.mdout of the repo root. It's a PR narrative that is already going stale:- It cites
25c6a0de, which no longer exists after the rebase. - It says
-f net8.0, but benchmarks now targetnet472;net10.0. - Its figures are from net462 and .NET 8.
Put "What changes", Equivalence, Benchmarks and "Parlot gaps" in the PR description, since that becomes the squash commit body. Keep the maintainer-relevant design notes (deliberate grammar quirks, PEG vs LL(2)) as a short
src/Spring/Spring.Core/Expressions/README.mdnext to the grammar, and point AGENTS.md at it. - It cites
-
BreakingChanges item 11 needs to be complete, because package validation's suppressions now list the removals:
- Name the four removed namespaces:
antlr,antlr.debug,antlr.collections,antlr.collections.impl. - List what
SpringASTlost: the protecteddown/rightfields,ICloneable/Clone,SpringAST(IToken)/initialize(IToken), and the inheritedBaseASThelpers (Equals*,findAll*,ToTree,xmlSerialize*,encode/decode,setVerboseStringConversion). - The sentence "their trees are the same" isn't accurate for
QualifiedIdentifier, which no longer has child nodes; onlygetText()is the same. - Optionally, mention that
�in the input is now rejected. ANTLR treated it as end of input and silently dropped the rest.
- Name the four removed namespaces:
Should fix in this PR (small)
- Contradictory error message for incomplete input.
ExpressionGrammar.cs:72setsexpected = "EOF"even when the failure is at the end of input. Soa?b,new aandnew a.breportexpecting "EOF", found '<EOF>', and the golden file freezes that at lines 341-342. Adding&& offset < input.Lengthmakes these fall through tounexpected end of expression. That needs three golden updates. SyntaxErrorExceptionserialization. The serialization constructor andGetObjectDataskipbase(info, context)/base.GetObjectData. That was only needed while it derived fromRecognitionException. It's public now and its serialized shape freezes with 4.0, and as written it drops InnerException, the stack trace and HResult. Call the base members and drop the custom"Message"key. Small style point:line/columnshould be_line/_column.
After merge, on my side
- Measure the cost of
EnsureSufficientExecutionStackper node evaluation on net472. On prepared expressions on Arm64 .NET 8 it costs 4–21%. If it matters, check depth every N levels instead. - Measure and document the supported nesting depth on a 256 KB thread (IIS x86).
ExpressionEvaluatorTestsisCompile Removed on net10.0 because of its BinaryFormatter parts. I'll split those out so the main expression suite also runs the new parser on .NET 10.5Lparses and thenIntLiteralNodethrows; this bug predates the PR.- Nits:
ExpressionBenchmarkfield naming, a blank line inStandardsComplianceTest.cs:77, and the fully qualifiedRuntimeHelpersinBaseNode. - The Parlot gaps you listed are probably better tracked as Parlot issues.
Ping me when it's updated and I'll merge once CI is green.
🤖 Generated with Claude Code
…essions Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
The examples pin transitive packages, and their test adapters need a newer System.Memory than the floor of the libraries. The Antlr target is gone from the schema, which the next build would rewrite. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
An incomplete expression said it expected the end of the input while being at it. SyntaxErrorException called the base members of serialization only while it derived from RecognitionException: without them it loses its stack trace and HResult. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
|
@lahma updated, as a merge of
On macOS, |
…hanges PARLOT_PORT.md moved to the description of the pull request, apart from the quirks and the PEG notes, which maintainers of the grammar need. Signed-off-by: Sébastien Ros <1165805+sebastienros@users.noreply.github.com>
|
@sebastienros, thanks, every point is addressed. I approved the CI run, since the branch is on a fork. Compile and examples now pass on Windows. One test fails on the Windows leg, Release build, net472: With JIT optimizations on .NET Framework x64, 1,900 nested unary minuses evaluate within the 256 KB thread, so nothing throws. The parser caps prefix operators and parentheses at 2,000 levels, so a unary case can't go deep enough to be reliably too deep for evaluation in every configuration. The Once that's in and CI is green, I'll merge. 🤖 Generated with Claude Code |
On the Windows leg's Release net472 run, 1,900 nested unary minuses evaluated within a 256 KB stack, so the case got no InsufficientExecutionStackException and failed. The parser caps prefix operators at 2,000 levels, which DeepNestingOfUnaryOperatorsIsASyntaxError covers, so the case can't be made to overflow reliably on every runtime. The 100,000-term chain keeps the evaluation-side check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LGkLQuAuUjvRjrU1AcTsmk Signed-off-by: Marko Lahma <marko.lahma@gmail.com>
|
@sebastienros, I pushed 32894f6 to your branch. It removes only the 🤖 Generated with Claude Code |
lahma
left a comment
There was a problem hiding this comment.
All the review points are addressed, and CI is green on build-windows, build-linux, integration-linux and DCO at 32894f6. Thanks, @sebastienros!
This replaces the ANTLR 2 parser of
Spring.Expressionswith a parser generated at build time by the source generator of Parlot, and removes the ANTLR 2.7 runtime thatSpring.Coreembedded, the generated lexer and parser, theAntlrbuild target and its Windows-only tool.The quirks of the grammar and the PEG notes stay in
src/Spring/Spring.Core/Expressions/README.md.What changes
Spring.CorePrivateAssets="all", the parser is generated into the assemblyantlr-2.7.6.exe, Windows onlyExpression.Parse, .NET 8Expression.Parse, .NET Framework((((1))))at depth 22 (45 chars)----1on a 1 MB stackSyntaxErrorExceptionbeyond 2,000 levelsThe second to last row is a denial of service: the grammar decides between
(a)and(a;b)by parsing the expression with a syntactic predicate and then again, so nesting doubles the time, even for an expression that ends up rejected.Equivalence
The history of this PR has
ParserDifferentialTests(commiteecad8a5, removed when ANTLR could no longer run). Itruns the two parsers side by side, through all four entry points (
Parse,ParsePrimary,ParsePropertyandParseLambda), and compares the printed trees (class, token type, text, children) and the errors. The inputs are 4,204strings from the test suite and a hand-written list of edge cases, generated expressions covering each grammar rule, and random mutations
of both with a vocabulary of the tokens and their near misses.
.followed by a digit always starts a number,even an invalid one), now fixed. Before that, smaller runs found the other lexer and follow-set quirks listed below.
ExpressionGrammarTestsstays, with the trees and errors ANTLR gave for 216 expressions, chosen for the quirks of thegrammar, written to
ExpressionGrammarGolden.jsonby ANTLR itself. They are compared exactly, forParseandParsePrimary: the 61entries where an error has another position or text hold both, this parser's as the expected one and ANTLR's for reference.
The ANTLR grammar has unusual behaviour that this port keeps deliberately, because it decides what is accepted: a dot between
nodes is optional (
a bisa.b,1 .aanda.parse),T(is one token only without a space,T(...)reads any tokenup to
), a real number with a fraction has to have a valid exponent afterewhile an integer doesn't (1.5exfails,1exis
1thenex),Ulis not an integer suffix butlUis, keywords can't be property names, the relational and poweroperators don't chain, the line of an error is always 1 and its column counts a tab as moving to the next multiple of 8.
Differences, all deliberate:
prediction often reports the token before it: for
#hope.floats#it reportedfloats, and now#. Where an errornamed the expected token, such as
expecting "RPAREN", found 'x', it is usually nowunexpected token: ["x",<33>,line=1,col=5].Most lexical errors, and the messages of
ParseLambdaandParsePropertyare as they were.�in the input. ANTLR took that char for the end of the input and ignored what followed. It is now aninvalid char.
QualifiedIdentifierhas no children. ANTLR kept one child per token of the name, which only served to rebuild itstext; the text is now whole.
getText()returns the same value.unexpected token or expression nested too deeply: "(", which can also be a plain syntax error at that depth.Breaking changes
Also in
BreakingChanges.txt:Spring.Expressions.Parser.antlrare gone, andSpringASThas no base class.getFirstChild,getNextSibling,addChild,setFirstChildandsetNextSiblinguseSpringASTinstead ofAST, andSpringAST(IToken)and the unused members ofBaseASTare removed. This needs at least a recompilation for code walking the tree.SyntaxErrorExceptionis now public and derives fromException: catch it instead ofRecognitionException.SyntaxErrorException, and evaluating too deep a tree throwsInsufficientExecutionStackException, instead of a stack overflow ending the process.Spring.Corenow depends explicitly onSystem.Memory, which it already had transitively on .NET Framework and .NET Standard 2.0.Costs to weigh
BaseNodecallsEnsureSufficientExecutionStack()for each node, which also fixes an existing crash on long chains such as1+1+1+.... PreparedGetValueis about 4% to 21% slower in short runs on Arm64.Spring.Core.dllis 8% larger, 908 KB to 983 KB, as the generated code is larger than what it replaces.unexpected token or expression nested too deeply, because the generated parser doesn't return why it failed.Benchmarks
These figures are from .NET 8 and the .NET Framework job (
net472), measured before the rebase onto the 4.0 groundwork. They were not measured again on .NET 10.test/Spring/Spring.Benchmark/ExpressionBenchmark.csonly uses the public API, so that the same file runs against both versions.The commit that adds it in this PR (
343f26f7) still has ANTLR:The expressions: Small is
Name.Length + 1(15 chars), Medium a conditional with two property paths (86 chars), Large seven terms with selections,projections, list and map initializers, a type, a constructor, a lambda and an expression list (491 chars).
BenchmarkDotNet v0.15.2,
--job short, one launch of each version, two machines:as one measurement of the Parlot Large parse was disturbed (7.5 µs and 16.8 µs, with a deviation of 7.5 µs).
net472job), one round. This machine has showndifferences of up to 27% between runs of identical code: read the ratios as orders of magnitude, not as precise values.
Parsing only
Expression.Parse, mean time and allocations:Most of ANTLR's 20 KB for the smallest expression is its lexer and token buffers, created per parse. A parse on .NET Framework is
about three times slower than on .NET 8, as it reads through the
System.Memorypackage.The layers above
ExpressionEvaluator.GetValue(root, expression)parses each time, with no cache. The first evaluation of a parse also does its reflectionlookups and creates dynamic methods, which dominate the Large expression, so its time depends less on the parser.
GetValue, uncachedExpression.Parsethen 100GetValue, as the documentation recommends. The cost of the parse is spread over the evaluations, andwhat remains is the first evaluation, so the gain shrinks as expected:
GetValueA prepared
GetValuedoesn't depend on the parser, but it pays for the stack checkBaseNodenow does. Arm64, .NET 8, ANTLR then Parlot withoutthe check then with it, one short run of each: Small 49 ns, 50 ns, 56 ns; Medium 816 ns, 749 ns, 806 ns; Large 2,042 ns, 1,972 ns, 2,382 ns, with the
same allocations. The last run gave 57 ns, 781 ns and 2,354 ns. So about 4% to 21%, within the noise of this machine for the smallest gaps. An application that evaluates prepared expressions in a
hot loop gives back that much, for not crashing on a deep tree.
The tables above were measured before the last fixes, which added the check, the guard on named arguments and a
Thenclosing each primary expression,with the same allocations. One new run on Arm64 is 5% to 12% slower at parsing than the tables (Small 243 ns, Medium 1,265 ns, Large 8,616 ns). That is within
what this machine varies under load, though it wasn't isolated, and the ratios to ANTLR stay at 8x to 42x there. x64 wasn't measured again.
ObjectWrapper.GetPropertyValueparses the name on each call.Nametakes the path that only reads a name (ParseProperty),Friends[1].Address.Citythe onethat parses a primary expression (
ParsePrimary):NameFriends[1].Address.CityTests
Spring.Core.Tests, whereExpressionEvaluatorTestsonly compiles on .NET Framework:net462, Windowsnet8.0, macOSThe 5 failures are the same on both sides and don't involve expressions: formatters that depend on the culture of the machine. These figures predate the rebase onto the 4.0 groundwork. After it and the review,
net10.0on macOS has 2,498 passed and the same 5 failures.net472, which hasExpressionEvaluatorTests, was compiled but not run locally, so CI is its first run.Parlot gaps
Nothing here blocked the port. Parlot 2.2.1:
net462is refused (PARLOT026, floor 4.7.2), and the package'snetstandardreference doesn't resolve on its reference assemblies, while the generated code works there. This took fourMSBuild targets until the 4.0 floor moved to
net472. Same finding as Lucene.NET.PARLOT021is misleading when a call doesn't bind.If(() => rule == x, a, If(...))with parsers whose types made inference fail reported "may only be read inside supported parse-time callbacks"rather than the inference error, as the check requires a bound argument. Giving the type,
If<BaseNode>(...), fixed it.Unarycan't mix operator types, althoughdocs/parsers.mdshows.Unary((Terms.Char('-'), ...), (Terms.Text("not"), ...)):Unary<TInput>(params ...)can't be inferred.Scandon't say so, and a scanner readingspan[0]throwsIndexOutOfRangeException.a == b == canda ^ b ^ chave to fail), and no last operand ofTernarythat is a whole expression. These force a chain of tables and sequences here.ElseErrorreports where it was reached, not the furthest failure, so a choice that must not backtrack takes a negative lookahead (Not(lparen)).EofneedsWhen((context, x) => ...)readingcontext.Scanner.Cursorto check what follows.methodOrProperty,arguments) didn't look shared across the choices that use them; this wasn't investigated further.MaxRecursionDepthor the end of the stack returnsfalseand a position, like a syntax error. The diagnostics overload has the reason but generates the grammar twice. So the message here is a guess from the depth at the failure: from 32 open brackets or prefix operators it saysunexpected token or expression nested too deeply, including for a plain typo at that depth, and a real stack failure below it reads as an unexpected token.