Skip to content

fix(model): clear the whole per-request query cache on any ORM write and on rollback - #4437

Merged
bpamiri merged 5 commits into
developfrom
peter/4429-request-cache-invalidation
Oct 6, 2026
Merged

bpamiri merged 5 commits into
developfrom
peter/4429-request-cache-invalidation

Conversation

@bpamiri

@bpamiri bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator

The bug

With cacheQueriesDuringRequest on (the default), a finder's result is cached in the base model's slot keyed by its SQL. An ORM write cleared only the writing model's slot, so a query that joins another model through include= — cached under the base model — kept returning the joined model's pre-write columns for the rest of the request (#4429). Reads made inside a transaction that then rolled back had the same shape: the rolled-back rows stayed cached as phantoms.

Fix (Approach A, approved by rev1-r1 + orch1)

  • $clearRequestCache() now clears the entire request.wheels["$queryCache"], not just the calling model's slot. Every ORM write path already funnels through it — create / update / delete / the *All and *ByKey forms / the fix(model): keep soft deletes reversible and make softDelete=false a real delete #4397 soft-delete UPDATE via $runDeleteStatement — so none can forget the cross-model case, and there's no new merge race. Whole-cache-on-write is the correct semantics for a request-scoped cache: a write ends the read-consistency window, and re-running a read afterward is both correct and cheap. (B — per-model/table tagging — was rejected: its dependency set can't be made complete for nested includes or raw where= cross-table refs without fragile SQL parsing, and a gap silently reintroduces the bug.)
  • $runInTransaction clears the request cache when it resolves a rollback — both the thrown-body path and the returned-false / transaction="rollback" path. A commit doesn't need it; its reads committed.
  • forgetCachedQueries (feat(model): add public forgetCachedQueries() to clear the per-request finder cache #4364) is unchanged. Its docs now point at all=true after a raw write to a table other models join through include=; the named form stays for single-table raw writes.

Tests

The core runner forces cacheQueriesDuringRequest off (tests/runner.cfm), so neither staleness was ever suite-visible. A new tests/specs/requestcache/ group turns the cache on per spec (restored in afterEach, which also restores the one post + author the specs mutate, so a failing spec can't leak state) and pins:

  • same-model write clears that model's cached finder;
  • cross-model include= write clears the base model's cached query — RED on develop;
  • transaction rollback clears reads cached inside it (no phantom rows) — RED on develop (verified RED against base, GREEN with the fix);
  • a read with no ORM write between is still served from cache (the cache isn't simply off).

Two existing specs that asserted the old per-model-clear behaviour (requestQueryCacheSpec, requestQueryCacheTenantCollisionSpec — one literally "clears only its own model's cache") are updated to the new clear-all behaviour.

Verification

  • Full core suite (Lucee 7 + SQLite): green.
  • tests/specs/requestcache/: RED on develop for the two cross-cutting cases, GREEN with the fix.
  • Guides build (pnpm build): 852 pages, Complete — the caching guide now documents the per-request cache.

Closes #4429

🤖 Generated with Claude Code

…and on rollback

The per-request finder cache (cacheQueriesDuringRequest, on by default) stored each finder's result
in the base model's slot keyed by its SQL, but an ORM write cleared only the writing model's slot. A
query that joined another model through include= lives in the BASE model's slot, so a write to the
included model left it returning the joined model's pre-write columns for the rest of the request
(#4429). Reads inside a transaction that then rolled back had the same shape: the rolled-back rows
stayed cached as phantoms.

- $clearRequestCache() now clears the entire request.wheels["$queryCache"], not just the calling
  model's slot. Every ORM write path already funnels through it (create / update / delete, the *All
  and *ByKey forms, the #4397 soft-delete UPDATE via $runDeleteStatement), so none can forget the
  cross-model case and there is no new merge race.
- $runInTransaction now clears the request cache when it resolves a rollback — both the thrown-body
  path and the returned-false / mode=rollback path. A commit does not need it; its reads committed.
- forgetCachedQueries (#4364) is unchanged; its docs now point at all=true after a raw write to a
  table that other models join through include=.

The core runner forces cacheQueriesDuringRequest OFF, so neither staleness was suite-visible. A new
tests/specs/requestcache group turns the cache on per spec (restored in afterEach, with the mutated
rows restored too) and pins the same-model clear, the cross-model include= clear (RED on develop),
the rollback phantom case (RED on develop), and that a read with no write between is still served
from cache. The guides' caching page documents the per-request cache.

Closes #4429

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@bpamiri

bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Review: head ee04e34. One MEDIUM to fix; the rest is APPROVE.

What's right

  • $clearRequestCache() itself clears everything. Every ORM write path reaches it:

    Clearing everything is correct for a request-scoped cache, and the guide's list of write methods matches those call sites.

  • Owner rollback clears it on both paths. On the throw path (transactions.cfm:174, guarded by closeTransaction, so only the owner clears) and on the returned-false / transaction="rollback" path (:195). Commit is untouched. Nested alreadyopen calls correctly leave clearing to the owner's rollback.

  • Specs. The new requestcache group turns the cache on itself (it's read at call time in read.cfm) and restores the setting, the rows and the cache in afterEach.

    • The cross-model include= case and the rollback-phantom case are RED on develop, as stated.
    • The "cache really is on" spec (a raw write isn't seen) guards against the group passing because the cache is simply off.
    • The two old specs that pinned per-model clearing are correctly updated.
  • Docs. The guide section, the forgetCachedQueries docblock (all = true after raw writes to joined tables) and the fragment are accurate.

MEDIUM: a savepoint rollback doesn't clear the cache

  • transaction="savepoint" (Nested transactions: add a savepoint API, and document Lucee dropping a savepoint set before the first query #3958) rolls back to the savepoint in $runSavepoint (transactions.cfm:428 on a throw, :433 on a false return). Neither path calls $clearRequestCache().
  • That's the same phantom-read case this PR fixes. A savepoint unit that writes and then reads (caching the uncommitted row) and is then rolled back leaves that row cached for the rest of the request, while the outer transaction carries on and may commit.
  • Suggested fix: call $clearRequestCache() after both $rollbackToSavepoint* calls, plus a RED spec modelled on $cacheRollbackProbe4429 but run with transaction = "savepoint" inside an outer invokeWithTransaction.

LOW

  1. Only the returned-false rollback path has a spec. One case where the probe throws inside a commit transaction (captured into var state, invariant 11) would pin :174 too.
  2. Optional docs note: a raw CFML transaction {} that rolls back isn't seen by Wheels, so forgetCachedQueries(all = true) applies there too, just as after raw writes.

The PR needs a full matrix (shared plumbing), which orch1 is holding until Actions recovers. PR CI at review time: 28 pending, 2 skipping. The final follows the MEDIUM fix, the matrix, and PR CI.

Addresses rev1-r1's review of #4437: a transaction="savepoint" rollback goes through
$runSavepoint, not $runInTransaction, so a savepoint unit's reads stayed cached as phantoms
after the savepoint rolled back while the outer transaction carried on and committed.

- $runSavepoint now clears the request cache after both rollbackToSavepoint calls (the
  exception path and the returned-false path).
- New specs: a savepoint rollback clears its cached reads (RED on the prior head), and a
  rollback through the exception/throw path of $runInTransaction clears too.
- The guides note that a raw transaction{} block rollback needs forgetCachedQueries(all=true),
  since the automatic clear only fires for ORM transactions.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@bpamiri

bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Re-review: head dac3232. The MEDIUM and both LOWs are resolved; code APPROVE.

  • Savepoint rollback. $runSavepoint now calls $clearRequestCache() after $rollbackToSavepointQuietly on the throw path (before rethrow) and after $rollbackToSavepoint on the false-return path. The outer transaction carries on with a clean cache. The new spec runs a savepoint unit that writes, reads and returns false inside an outer commit. Its RED on ee04e34 (Expected [Per] but received [PhantomSavepoint4429]) is exactly the phantom case.
  • Throw-path rollback spec. $throwRollbackProbe4429 throws inside a commit transaction, so the rollback goes through the exception branch at :174. The spec captures the throw into var state (invariant 11) and checks that the phantom isn't served.
  • Docs. The raw transaction {} rollback note is added.

This is shared plumbing, so the final needs a full compat matrix on this head (orch1 is holding dispatches until Actions recovers) and green PR CI.

@github-actions

github-actions Bot commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Wheels Test Results

35 785 tests   35 060 ✅  11m 12s ⏱️
 2 830 suites     725 💤
     5 files         0 ❌

Results for commit 4f08511.

♻️ This comment has been updated with latest results.

@bpamiri

bpamiri commented Oct 5, 2026

Copy link
Copy Markdown
Collaborator Author

Matrix 37379702292 (head dac3232): 30/32 legs green. One MUST-FIX in the new spec.

  • RequestCacheInvalidationSpec is 6/6 on 30 legs, and requestQueryCacheSpec 8/8 / requestQueryCacheTenantCollisionSpec 6/6 on all 32. Every PostgreSQL, Oracle and SQL Server leg and both H2 legs are green, as are the savepoint and rollback specs.

MUST-FIX: the spec's raw-write helper binds a CockroachDB id as a 32-bit integer (adobe2023-cockroachdb and adobe2025-cockroachdb).

  • "serves a repeated read from cache when no ORM write intervenes" errors with Invalid data 1216256709799313409 for CFSQLTYPE CF_SQL_INTEGER.
  • $rawPostTitle() binds id = {value = arguments.id, cfsqltype = "cf_sql_integer"}. CockroachDB's unique_rowid() ids are 64-bit; Adobe validates the bind type, while Lucee and BoxLang let it pass. That's why only the two Adobe × CockroachDB legs fail.
  • Fix: use cfsqltype = "cf_sql_bigint" (or leave the type off), and declare the helper's id argument as any or a string so it doesn't round-trip through a numeric. Spec-only; no framework change.

PR CI has one failure, "Visual regression". It's the blog page (42,811 pixels differ), which this PR doesn't touch. It looks like blog content changed on develop after this branch's baseline was taken; merging develop or refreshing the baseline should clear it. Every other PR check passed (30 pass, 3 skipping).

After the bigint fix, a narrowed re-run (Adobe 2023/2025 × CockroachDB, plus a SQLite compile check) on the new head should be enough, read together with this full run.

bpamiri and others added 2 commits October 5, 2026 16:38
…ot integer

Addresses the #4437 matrix review: $rawPostTitle bound the post id as cf_sql_integer, so
Adobe 2023/2025 x CockroachDB errored on 64-bit ids ("Invalid data ... for CF_SQL_INTEGER").
Bind cf_sql_bigint and take the id as an untyped argument so a 64-bit value is not truncated
through a numeric cast.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@bpamiri

bpamiri commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator Author

FINAL: APPROVE. Head d8861ac.

  • Full compat matrix 37379702292 on dac3232 (the code): 30/32 legs green. RequestCacheInvalidationSpec (including the savepoint, throw and returned-false rollback cases), requestQueryCacheSpec and requestQueryCacheTenantCollisionSpec passed on every leg, apart from the one spec error on the two Adobe × CockroachDB legs.
  • That error was in the spec, now fixed: $rawPostTitle() binds the id as cf_sql_bigint and takes an untyped id. Against the merge base, the PR's own diff changed only by those lines. The rest of the d8861ac delta is develop (for the blog visual baseline).
  • Narrowed 37389705646, Adobe 2023 / 2025 × CockroachDB: 2/2 green, with RequestCacheInvalidationSpec 6/6 on both.
  • Narrowed 37389715473, 5 engines × SQLite + RustCFML (merge check): 5/5 green, 6/6 on each. RustCFML: 6958 pass, 0 fail, 0 error.
  • PR CI on d8861ac: 31 pass, 0 fail, 0 pending, 3 skipping. Visual regression now passes.

…-cache-invalidation

Signed-off-by: Peter Amiri <peter@alurium.com>

# Conflicts:
#	web/sites/guides/src/content/docs/v4-2-0/digging-deeper/caching.mdx
@bpamiri

bpamiri commented Oct 6, 2026

Copy link
Copy Markdown
Collaborator Author

FINAL (re-confirmed after the develop conflict): APPROVE. Head 4f08511.

@bpamiri
bpamiri merged commit a722945 into develop Oct 6, 2026
45 checks passed
@bpamiri
bpamiri deleted the peter/4429-request-cache-invalidation branch October 6, 2026 00:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Request query cache keeps stale include= results after a write to the joined model

1 participant