Skip to content

Codechange: Assign PLY parse tree lists in-place - #454

Closed
LordAro wants to merge 1 commit into
OpenTTD:masterfrom
LordAro:la-speed
Closed

LordAro wants to merge 1 commit into
OpenTTD:masterfrom
LordAro:la-speed

Conversation

@LordAro

@LordAro LordAro commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Profiling nmlc against the very, very large iron-horse.nml shows that a very large amount of time is taken in parsing the input. Not exactly a surprise as it's huge - somewhere around 80MB. But, it can be faster. My local testing indicates the 'parsing' phase drops from around 50s to 25s (with cpython 3.15). Which isn't bad.

The main improvement comes from "unrolling" the PLY accessors in the first commit. It'll be partly due to a large of intermediate objects, and from the unrolling itself.

Very large disclaimer: This change was originally written by an LLM. I have since (hopefully) suitably torn it apart and rewritten as needed. Please review as brutally as you see fit.

The previous code uses the nice helper function that can handle various
different forms of arguments (slices & negative numbers for stacks) that
we don't use. Remove this indirection and assign the output directly.

Additionally uses in-place modifications where feasible to avoid having
to construct intermediate objects.
@PeterN

PeterN commented Oct 26, 2026

Copy link
Copy Markdown
Member

Well that's problematic because I don't think we should get a free pass.

@LordAro

LordAro commented Oct 26, 2026 •

Copy link
Copy Markdown
Member Author

Probably not. It's an admittedly thin argument, but this is not OpenTTD itself - as far as I'm aware we do not have a org-wide policy.

And sometimes, there is only one way of doing something

@PeterN

PeterN commented Oct 26, 2026

Copy link
Copy Markdown
Member

If we don't then we should. It does not make sense to only apply to one piece.

I am not going to mess about with definitions, exceptions or whatnot.

@LordAro

LordAro commented Oct 26, 2026

Copy link
Copy Markdown
Member Author

Alright, fine.

So... what do we do next? 2x parsing performance improvement isn't something that should be ignored. Presumably I can't do anything, as I'd just reproduce the above (but slower)

@PeterN

PeterN commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Put it into perspective, we have literally one person complaining about build times, and their parsing time with Pypy without this change is 5.3 seconds. I don't think it's worth the taint.

@andythenorth

Copy link
Copy Markdown
Contributor

If the choice was as stark as project unity, or 2x parsing improvement, I'd pick project unity every time.

@LordAro LordAro closed this Nov 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants