Skip to content

Invalidate object cache after raw SQL delete in idx_clean_transients() - #560

Open
pmallek wants to merge 1 commit into
idxbroker:masterfrom
pmallek:fix/clean-transients-object-cache-invalidation
Open

Invalidate object cache after raw SQL delete in idx_clean_transients()#560
pmallek wants to merge 1 commit into
idxbroker:masterfrom
pmallek:fix/clean-transients-object-cache-invalidation

Conversation

@pmallek

@pmallek pmallek commented Sep 1, 2026

Copy link
Copy Markdown
Contributor

Summary

Idx_Api::idx_clean_transients() deletes idx_*_cache options with a raw SQL
DELETE, bypassing delete_option(). On installs backed by a persistent object
cache (Memcached, Redis) the cached copies survive the delete. The affected
options then become permanently unwritable, which on at least one production
site turned into a sustained 412 loop that took all IDX content down.

idx_clean_transients() runs on plugin update (Initiate_Plugin::update_triggered()),
so any site on such a host can enter this state after upgrading.

Root cause

idx/idx-api.php:687

DELETE FROM $wpdb->options WHERE option_name LIKE '%idx_%_cache'

Once the row is gone but the object-cache entry is not:

  1. get_option() returns the stale cached value, so it is not false;
  2. update_option() only falls back to add_option() when get_option()
    returns false (wp-includes/option.php:928);
  3. otherwise it runs an UPDATE that matches zero rows, hits
    if ( ! $result ) return false; and never reaches wp_cache_set().

The value can never be refreshed or repaired. It stays expired forever, and
every request re-queries the API.

Impact

The trapped entry that matters is idx_clients_accounttype_cache.
engage_account_type() has 14 call sites, 8 of them on the normal bootstrap
path (initiate-plugin.php x2, register-impress-shortcodes.php x2,
register-blocks.php, create-impress-widgets.php,
register-shortcode-for-ui.php, idx-pages.php), and it has no static
memoization. With the cache trapped, one WordPress request can mean several
/clients/accounttype calls.

On a production site on Pressable (IDX Broker Engage, 500 calls/hour) we
measured hourly-access-key-usage: 968 roughly ten minutes into the hour —
about 16 requests per minute, every one of them returning 412.

Observed timeline on that site, from its own options:

when (UTC) what
2026-08-28 19:56 3.3.0 update ran, idx_api_cache_repaired_at = 1787946960
2026-08-28 19:56 idx_clients_accounttype_cache written, expiration = 1788033361
2026-08-29 19:56 that entry expired
2026-08-29 20:09 first 412 recorded, 13 minutes later
2026-09-01 09:07 still looping at 968 calls/hour

Why this looks intermittent

We found four options present in Memcached but absent from wp_options, all of
them small. Every large cache entry had recovered on its own — featured
(410 KB), leads (149 KB), postalcodes (48 KB), cities (38 KB). Memcached
evicted those under LRU pressure, so get_option() returned false and
add_option() rebuilt the row correctly.

Only small, cache-resident entries stay broken. That makes the bug
host-dependent and load-dependent, which may be why it has been hard to pin down.

Reproduction

  1. WordPress with a persistent object cache drop-in (Memcached or Redis).
  2. Let IMPress populate idx_clients_accounttype_cache.
  3. Trigger idx_clean_transients() (a plugin update does it).
  4. Observe: the row is gone from wp_options, get_option() still returns the
    old value from the object cache, and update_option() on that key returns
    false forever.

The fix

Collect the matching option names before the DELETE, then wp_cache_delete()
each one plus the notoptions and alloptions buckets. Both branches of the
function are covered. Passes php -l.

Recovering an already-affected site

The fix prevents new occurrences but does not heal existing ones, since the
trapped entries are already orphaned. Dropping the orphaned keys from the object
cache lets add_option() recreate the rows. We did this on the affected site and
the call rate dropped to zero immediately, with no downtime.

Related

  • Refs Stop http requests once hourly API limit is reached. #57 (open since 2016) — this is one concrete mechanism behind
    "Stop http requests once hourly API limit is reached". The 412 cooldown in
    idx_api() is real, but it cannot help while its own state option is trapped.
  • Related to Cron Jobs causing issues with sites going down #70idx_create_idx_pages / idx_delete_idx_pages run every
    three minutes by default (idx-pages.php:19), and each run bootstraps the
    plugin, adding ~40 more accounttype checks per hour on top of organic traffic.

Two related observations, not in this PR

Happy to open separate PRs for either:

  1. engage_account_type() would benefit from static memoization — even with a
    healthy cache it does 8+ get_option() calls per request.
  2. The '%idx_%_cache' pattern has a leading wildcard and an unescaped
    underscore (a single-character wildcard in LIKE), so it can match and
    delete options belonging to other plugins. $wpdb->esc_like() with an
    anchored prefix would tighten it.

idx_clean_transients() removes idx_*_cache options with a direct DELETE
query, bypassing delete_option(). On installs backed by a persistent
object cache (Memcached, Redis) the cached copies survive the delete, so
get_option() keeps returning a value for a row that no longer exists.

update_option() only falls back to add_option() when get_option()
returns false. Otherwise it issues an UPDATE that matches zero rows,
hits "if ( ! $result ) return false;" and never reaches wp_cache_set().
The option is then permanently unwritable and stays expired forever.

In practice this traps idx_clients_accounttype_cache, which
engage_account_type() consults on every bootstrap, so each WordPress
request re-queries /clients/accounttype until the account exhausts its
hourly API limit and every IDX call returns 412.

Collect the matching option names before the DELETE and drop their
object-cache entries afterwards, along with the notoptions and
alloptions buckets.

Refs idxbroker#57

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant