Skip to content

[2.x] perf: load limited BelongsToMany includes in two narrow phases - #4867

Merged
imorland merged 1 commit into
2.xfrom
im/narrow-limited-includes
Jul 31, 2026
Merged

[2.x] perf: load limited BelongsToMany includes in two narrow phases#4867
imorland merged 1 commit into
2.xfrom
im/narrow-limited-includes

Conversation

@imorland

Copy link
Copy Markdown
Member

Discussion-view performance arc, part 5 (#4862, #4863, #4865, #4866) — the last heavy query on a discussion view: the limited mentionedBy include window.

Changes proposed in this pull request

Laravel compiles a limited eager load into row_number() OVER (PARTITION BY …) over the relation's full select — every column of every candidate row, content blobs included, is materialized through the window sort just to keep a few per parent. Mentions keeps 4 mentioning posts per post; on a post mentioned 1,403 times, that's 1,403 full-width post rows into a temp table to keep 4. Likes' per-post user window has the identical shape.

EloquentBuffer now loads limited BelongsToMany includes in two phases:

  1. Narrow window — the same groupLimit window Laravel would run, but selecting only the related key (pivot keys are appended automatically). Resource scoping (visibility) applies here, so the surviving ids are exactly the ones the one-phase query would keep, in the same order.
  2. Full fetch — one whereIn for the surviving rows only, carrying the endpoint's nested eager loads (mentionedBy.user, …).

Anything that isn't a limited BelongsToMany falls through to the previous path untouched. Implementation note for reviewers: in eager context, a relationship scope's ->limit() lands on the base query builder as groupLimit, not limit — checking the wrong property makes the fast path silently never engage (caught by inspecting the live SQL, which is also the honest way to verify this PR does what it says).

Numbers (dev install, post with 1,403 mentions)

  • Window-query cost: median 9.3ms → 6.1ms (−35%); the remainder is visibility-scoping the candidate rows inside the window, which is bounded work per candidate rather than per column.
  • Response payloads byte-identical (verified live: same 4 include ids, same order).
  • Combined with [2.x] perf: materialize post visibility once; flat grouped relation aggregates #4866, the two mention queries on a discussion view went from 5.5 + 9.3ms to 1.75 + 6.1ms on the worst post in the database.

A UNION-per-parent (or LATERAL) shape could collapse the window cost further via per-parent early termination, but needs dialect branching (MySQL 5.7 has no LATERAL) — left as a documented escalation if real-world profiles ever demand it.

Reviewers should focus on

  • EloquentBuffer::loadLimitedBelongsToMany(): the probe relation + $loader application, the groupLimit/limit detection, id grouping via pivot key, and empty-collection setRelation (prevents lazy reload).
  • The new mentions test pinning that the limit partitions per post, not per page — two heavily-mentioned posts on one page each get their own window; this was the coverage gap a global-limit regression would have slipped through.
  • Existing pins that carried the semantics: exact include ids in oldest-first order with invisible posts excluded, across show/list/default-include endpoints (mentions 82 — also green under a flarum_ table prefix — likes 22, messages 12).

Confirmed

  • Backend changes: tests are green (run composer test).

Laravel compiles a limited eager load (mentions' four mentioning posts
per post, likes' user window) into row_number() over the relation's
full select: every column of every candidate row — content blobs
included — is materialized through the window sort just to keep a
handful per parent. On a post mentioned 1.4k times that is 1,403 full
post rows into a temp table to keep 4.

The buffer now runs the window over the related key and pivot columns
only, then fetches the surviving rows in one whereIn with the
endpoint's nested eager loads. Resource scoping applies to the narrow
phase, so the kept ids match the one-phase query exactly, in the same
order. Anything that isn't a limited BelongsToMany falls through to
the previous path untouched.

Note: in eager context a relationship scope's ->limit() lands on the
base builder as groupLimit, not limit.

Window cost on the stress post drops ~35%; response payloads are
byte-identical. A new mentions test pins that the limit partitions per
post, not per page — the coverage gap a global-limit regression would
have slipped through.
@imorland
imorland requested a review from a team as a code owner July 31, 2026 11:33
@imorland imorland added this to the 2.0.0-rc.6 milestone Jul 31, 2026
@imorland
imorland merged commit 6af637c into 2.x Jul 31, 2026
25 checks passed
@imorland
imorland deleted the im/narrow-limited-includes branch July 31, 2026 11:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant