Skip to content

Merge train: #9794, #9780, #9774, #9814, #9756, #9796, #9775 (all previously conflicting) - #9863

Merged
proggeramlug merged 27 commits into
mainfrom
train128
Sep 6, 2026
Merged

Merge train: #9794, #9780, #9774, #9814, #9756, #9796, #9775 (all previously conflicting)#9863
proggeramlug merged 27 commits into
mainfrom
train128

Conversation

@proggeramlug

Copy link
Copy Markdown
Contributor

Merge train: #9794, #9780, #9774, #9814, #9756, #9796, #9775 — seven PRs that had been stuck on conflicts, all resolved here.

The conflicts, and what each actually needed

#9756 presented as 43 conflict hunks across 13 files of GC internals and needed zero hand-merging. Its description opens with "Stacked on #9755 (its commit is included; review only the last commit)", and #9755 landed in train #9817. Comparing commit subjects against main showed 6 of its 8 commits already there; picking the 2 genuinely missing ones applied clean. Reading the PR body first would have saved the most work of anything today.

#9814 was the one that needed real judgement. boxed_primitives.rs had a mid-function conflict where neither side alone was correct: #9794 adds boxed_string_wrapper_utf16_len, #9814 makes string indices virtual and roots the length install — and #9814's own descriptor_state.rs calls #9794's helper. --ours dropped the virtual indices (its test then crashed with TypeError: Cannot assign to read only property '0'); --theirs dropped the helper (unresolved import). The merge keeps #9814's rooted install plus #9794's helper, and drops install_string_wrapper_indices, which virtual indices make obsolete.

That merge then created a real bug, caught by the suite rather than the compiler: it left two synthesize paths, and #9814's string_wrapper::has_index early-return short-circuited above the stored-descriptor table probe, so Object.defineProperty on a wrapper index could never win. Synthesis now happens only in the fallback below the table. All five string_wrapper tests pass.

#9796 — union for two independent regex.rs additions, then #9801 (landed since) had added lazy.rs call sites using the old (String, String) cache key against #9796's new (Arc<str>, Arc<str>). Four stale sites in total, each surfacing at a different stage: compile, test-compile, and gate.

#9775main already has try_import_meta_require, so HEAD wins that file; its inprocess.rs work is kept.

#9794, #9780, #9774 — a gc/mod.rs entry-list union, a two-comment concat, and the census hot-TLS conversion respectively. check_thread_locals is now green on every file for the first time.

File-cap splits

Four files crossed the 2000-line gate: descriptor_state.rsdescriptor_state/gc_scan.rs; regex.rsregex/program_key.rs and regex/replace_expand_fancy.rs; inprocess.rsinprocess/optimize_emit.rs; lower/tests.rslower/tests/ui_widget_add_child.rs. Each verified in both feature configurations — a --features regex-engine build passes while the default one fails when a moved item sits behind that gate.

The raw_handle_debt --no-raise-vs invocation then correctly refused to let the new replace_expand_fancy.rs start with a ceiling: a brand-new file must have zero bare reads. Rather than grandfather it, all four moved sites became with_mut_ptr — preserving the deliberate per-iteration re-read after an allocating js_string_from_str — so total debt dropped by four instead of relocating.

Gate verdicts

SHAPE_CACHE_YOUNG's verdict was deleted because #9756 removes that log — the inventory failing a stale entry is it working as designed. NEVER_MATCH (Arc<regex::Regex>, a Rust-allocator program) classified. PASS1_MARKED's window re-pinned after #9794's diagnostics touched gc/mod.rs and gc/policy.rs: both gain declarations and counters only, and the census bracketing in step_mark_propagation / step_sweep is unchanged.

Validation

64/64 lint gates; perry-runtime, perry-codegen, perry-hir, perry-stdlib, perry-transform, perry-ext-http — run serially (RUST_TEST_THREADS=1).

Ralph Küpper added 27 commits September 6, 2026 05:53
…l origins, allocation-site sampling

Three instruments for the cc-perf campaign, all inert unless asked for.

`PERRY_GC_DIAG=1` gains the lines that say WHY the collector ran:
`[gc-trigger]` (every predicate input at each decision site), `[gc-full]`
(the arm behind each synchronous full mark-sweep, counted per site),
`[gc-budgeted] start/done` (steps, per-phase step time, root-scan share),
`[gc-charge]` (mutator-assist / synchronous-full time per calling site,
resolved to JS display names) and `[gc-survival]` (per copying minor, the
root that first reached each surviving byte — shadow stack, native stack
map, named scanner, remembered set by old-parent type — with transitive
reach charged to the originating root through a parallel worklist origin
vector).

`PERRY_ALLOC_SITE_SAMPLE=<bytes>` samples the arena allocation sites byte-
proportionally across the runtime allocators AND the codegen inline bump
path (the mirrored inline block limit is capped at one interval while
sampling, so the fast path returns to the runtime once per interval).

The survival test is sabotage-checked: disabling the drain propagation
charges the 40 elements to `worklist_drain` and the test fails on that row.
The knob's OFF state and magnitude parse are pinned next to the other GC
knobs. `gc_diag_enabled()` gets the per-thread test override the census
already has, so the diag paths are testable without touching the process
environment.

Claude-Session: https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
…iptors on the primitive-string path

Three allocations a JS program can never observe, found with the
allocation-site sampler (`PERRY_ALLOC_SITE_SAMPLE`) on the compiled
claude-code TUI, where they are the largest attributed source of garbage in
both the streaming turn and the render pass that follows it.

1. A one-ASCII-character string is now the canonical per-thread header.
   `js_string_char_at` minted a fresh 32-byte heap string per character read,
   and everything that walks a string a character at a time goes through it:
   `s[i]`, `charAt`, string spread, the String-wrapper index installer. There
   are 128 possible contents. The table has the same residency contract as the
   small-integer string table next to it (longlived arena, `refcount = 0` so it
   is never mutated in place, pinned out of the young generation) and rides
   that table's existing root scanner rather than registering a 96th one.

2. Runtime-internal constant property names resolve through the intern table.
   The `globalThis` builtin lookup, `x.constructor`, `toString` resolution and
   primitive-method dispatch each built a fresh heap string for a literal name
   on every call; `js_get_global_this_builtin_value` alone accounted for 133 MB
   of the 990 MB one 3300-character reply allocates. `string::canonical_key`
   routes them through the content-keyed per-thread table that
   `js_string_materialize_to_heap` already uses, which is also what the
   property read/write fast paths require of a key.

3. A `String` wrapper no longer stores a property descriptor per character.
   ECMA-262 §10.4.3 gives every in-range index of a String exotic object
   `{ writable: false, enumerable: true, configurable: false }` — a fact of the
   class and the boxed length, not per-object state — so `get_property_attrs`
   answers it from the wrapper's payload. Storing it cost, per boxed character,
   a Rust `String`, a `PROPERTY_DESCRIPTORS` entry only a full collection's
   dead-owner prune could reclaim, an owner-index entry, and one program-wide
   `prop_plan_epoch_bump()`. A sloppy method call on a string primitive boxes
   its receiver, so the TUI paid all of it per rendered line. A real stored
   descriptor still wins, so `Object.freeze`/`defineProperty` on a wrapper are
   unchanged.

`PERRY_GC_DIAG=1` also gains `[gc-primitive-dispatch]`: which
`<Builtin>.prototype.<method>` names reach the primitive-method fallback, how
often, and how many wrapper index properties they cost — the counter that says
whether a boxing fix ran.

Claude-Session: https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
… % more

The five tables #9754 converted were valued individually with a
measurement-only `PERRY_YOUNG_LOG=0` gate on
`RuntimeRootVisitor::young_scope()` (all five scanners fall back to their full
walk together inside one binary), plus a third arm — `cc_base_new`, main
`1d63fa91f`, no logs at all. Three interleaved rounds, `stream_scale` len 3300,
identical collection schedule in every arm (minors 196/194/196, budgeted steps
60/59/59), so these are scan costs:

| scanner, ms per turn         | main            | log, full walk  | log, minor walk |
|------------------------------|-----------------|-----------------|-----------------|
| all 95 scanners              | 14761/14105/19411 | 23368/23286/26974 | 2667/2852/3674 |
| scan_shape_table_rekey_mut   | 10884/11077/14458 | 18893/18655/22125 | 1426/1567/1947 |
| scan_descriptor_roots_mut    |   1807/1192/2223  |  2257/2467/2548   |  127/136/145   |
| scan_closure_dynamic_props   |    1014/985/1347  |    890/902/959    |  224/236/209   |
| transition_cache scanner     |     121/123/159   |    254/221/246    |   85/94/145    |
| shape_cache scanner          |      89/89/113    |    159/153/167    |  121/122/151   |

The shape cache is the one table where the log loses to the walk it replaced:
+34 ms (+35 %) against main, having skipped **0.0 % of 3.85 M entry visits in
every one of 107 collections**. The cause was already documented — the
canonical keys arrays are allocated in the LONGLIVED arena, which
`addr_is_minor_relevant` must answer `true` for because a longlived parent is
not write-barriered, and a longlived object is never promoted, so no entry ever
leaves the log.

So it goes back to the plain `values_mut()` walk: the arm helper, its
production and test-seam call sites, the thread-local log, the name constant
and the `debug_assert_logged` re-derivation are all deleted. An inert log is
not free — it is a permanent arming obligation on every future writer of that
cache plus a suppression audit that has to keep proving each site — and it
should not land on the promise of a longlived remembered set that does not
exist yet. When that set exists and makes this table skip something, the log
can come back with a measurement.

The test is kept as a scanner test (a young entry reachable only through the
cache still moves and is re-keyed in both the inline slot and the overflow map)
and now asserts that NO `[gc-young-log]` row exists for the table, so re-adding
a log here without re-measuring is a red test.

Note for anyone repeating this on another table: the two-arm version of this
experiment gives the wrong answer. With the log merely disabled, the full-walk
arm still pays its upkeep — a `take_sorted()` whose sorted result is discarded
and an `addr_is_minor_relevant` probe per entry to rebuild `kept` — so every
"off" row above is worse than main, by +7.8 s on the shapes table alone. Only
the third arm says whether a log should exist at all.

Claude-Session: https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m
`ShapeIndex.slots` — the content-hash to slot accelerator built for every keys
array past `KEYS_INDEX_THRESHOLD` — was a `PtrHashMap<u64, SlotList>` per
shape: a 33-byte hashbrown bucket per key in a power-of-two table (2.1 KB for a
40-key object). The compiled claude-code TUI holds thousands of these.

Every hit the index produces is re-validated against the key bytes
(`shape_slot_lookup_verdict`), so a colliding answer is a miss and never a
wrong property — which is what lets the stored hash be narrow. `SlotIndex` is
an open-addressing table of (16-bit tag, 16-bit slot) cells (4 B; a 16-bit tag
and 32-bit slot only past 65,535 keys); the tag is the top of a golden-ratio
fold of the FNV-1a hash (FNV's own high bits barely move for short keys), the
probe position is a function of the tag alone so cells re-place themselves on
growth and after a delete's `retain_shift`, load kept at or below 7/8. 40
keys: 64 x 4 B = 256 B against the 2.1 KB hashbrown table. Same O(1) probe; a
repeated note-hit no longer appends a duplicate.
…es, engine prototype switch

Rebased onto main after #9764 landed as ddbe0b1; the site cache,
header-authoritative lookups and find-only global `test` are main's now and
are gone from here. The program-cache coherence fix moved to #9801.

Three changes remain.

* The capture-group cliff. `repeat_matcher::capture_layout` takes a pattern
  off the linear engine when ECMA-262's RepeatMatcher capture semantics are
  observable — a capture directly under a quantifier, or a capture inside a
  negative lookaround. That routing is a correctness requirement, but the
  engine it routes to is a classical backtracker with no step budget, so
  adding parentheses fell from linear time to exponential
  (`/^(a+)+$/.test("a"*28 + "!")`: 16,522 ms; node 4,798 ms). 6.3 % of 4,463
  real literals take that route. Both engines accept the same LANGUAGE and
  differ only in capture ASSIGNMENT, so `linear_rules_out_match` asks the
  linear program first and a subject it rules out — which is what every ReDoS
  input is — never reaches the backtracker. This removes the reachable
  exponential case; it does not bound the worst case (that needs the step
  budget open upstream as ridiculousfish/regress#177).

* Allocation-free cache probes. The three compiled-program caches were
  `HashMap<(String, String), _>`, so every probe allocated two Strings and
  copied the pattern text, once per RegExp OBJECT. `ProgramKey =
  (Arc<str>, Arc<str>)` makes a probe two refcount increments; the remaining
  materialisations are cold (`RegExp.prototype.compile`, the syntax-error
  fallback).

* `PERRY_REGEX_ENGINE=regress`, off by default: routes every pattern through
  the ECMAScript backtracker and installs a shared never-match placeholder as
  the standard program, so the tier-0 engine architecture can be measured in a
  real binary. Not a supported configuration — the backtracker has no budget.

Tests: quantified_capture_pattern_does_not_backtrack_on_a_non_matching_subject,
plus the `capture_layout` predicate assertions rewritten around the
`(layout, needed)` pair.

Claude-Session: https://claude.ai/code/session_01YPfnmWZmSpSWpmnoXvH8z2
changelog.d/README.md asks for `<PR-number>-<slug>.md`; the three fragments
landed unnumbered. Renames only — no entry text and no code changes, so the
measured candidate binary is unaffected.

Claude-Session: https://claude.ai/code/session_014UZWia6L37DpA93VLtNK9m
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant