Skip to content

fix(plugins): keep NoSQL edits on the record, key and TTL they read, page Kafka by records and bound Redis string reads - #4220

Merged
datlechin merged 3 commits into
mainfrom
fix/audit-w6-nosql-drivers
Nov 12, 2026
Merged

datlechin merged 3 commits into
mainfrom
fix/audit-w6-nosql-drivers

Conversation

@datlechin

@datlechin datlechin commented Nov 12, 2026 •

Copy link
Copy Markdown
Member

Summary

Audit 2026-11-12, lane nosql-drivers (NQ-1 to NQ-10).

  • SurrealDB: rows carry their record id as a row locator, so editing or deleting a record whose id is a UUID, array or object reaches that record. A query result row whose id text could be one of those is refused instead of saving nothing (NQ-1).
  • Cassandra and ScyllaDB: browsing reads each cell's TTL. An edit writes USING TTL with what is left (0 for a cell that never expires when the table has a default), cells with different TTLs go in one conditional batch, undo delete puts the TTLs back, and an edit of a cell that expired since the read is refused (NQ-2).
  • Kafka: a NEWEST or TAIL page walks back by records instead of offsets, each step sized by the last range's density. A reply holding only commit markers, aborted or compacted batches moves the read on instead of reporting truncated results. Past the page's own range a walk reads at most 256 MiB per partition, then shows what it has, marked truncated (NQ-3).
  • MongoDB: grid sorts end with "_id": 1, as the shared core's pager does, so equal values keep one order across pages (NQ-4).
  • Byte order mark: Redis, Cassandra, Bigtable, SurrealDB and Memcached decode wire text exactly, so a key starting with U+FEFF is shown and written as itself. Bigtable writes name the row by its locator (NQ-5).
  • Redis: the key browse reads a string with GETRANGE up to 64 KiB and marks a longer one. A Value edit of it, or a new row copied from a preview, is refused. Export, including KEYS, reads whole values; console KEYS and SCAN summarize in batches (NQ-6).
  • Elasticsearch: rows carry _index, _id and _routing, and updates, deletes and the read before an update use them, so routed and child documents save on their shard (NQ-7).
  • Elasticsearch: in a text or keyword field, an edit that reads as a JSON array stays a string unless the stored value is an array. A new row keeps the array unless the field held such a string (NQ-8).
  • MongoDB: an edited number keeps the BSON type its cell was read as when the new value fits that type, the rule the core took in fix(core): bound result bytes, keep MongoDB number types, hold FFI secrets in secrecy, keep long iOS query values whole and give each iOS connection one session owner #4201 (NQ-9).
  • Memcached: the grid reads with gets, a Flags-only or Value-only edit is a cas, and EXISTS or NOT_FOUND fails the save (NQ-10).

Tests changed on purpose:

  • MongoDBOnIOSTests.pageQueryCombinesFiltersAndSearch expects the _id tiebreak.
  • RedisKeyMetadataReadTests pass the new stringByteLimit: argument.

Needs the maintainer:

  • MongoDB: a sort on an indexed field now also sorts by _id, which a single-field index cannot serve, so a large collection sorts in memory. MongoDB's Sort Consistency guidance and the core's pager do the same.
  • SurrealDB: Data Rewind cannot find a record whose id is a UUID, array or object, and refuses the rewind rather than writing. It needs a locator query.
  • Cassandra: editing only a static column of a table with clustering columns is still refused by the server, as before.
  • Kafka: a partition whose newest data holds more than 256 MiB of aborted transactions above its newest committed records shows a short page marked truncated, as it showed an empty one before. Finding that boundary needs probe reads.

Docs: TableProApp/docs#211 (draft).

Test plan

  • swift test in TableProCore for TableProRedisCoreTests, TableProCassandraCoreTests, TableProKafkaCoreTests, TableProBigtableCoreTests and TableProMemcachedCoreTests: 371 passed. New: TTL grouping and batch text, Kafka tail walk and next offset past markers, exact UTF-8 decode, Redis string preview, Bigtable locator keys, Memcached cas choice.
  • RedisDriverTests: 654 passed, with the new key browse value, copied preview and byte order mark cases.
  • verify.sh plugins builds every registry plugin. MongoDB, SurrealDB, Elasticsearch, the Cassandra browse glue and the Kafka engine have no test target; their paths were traced by hand and reviewed in four rounds.
  • The iOS test (MongoDBOnIOSTests) runs on CI only: this Mac has no iPhone simulator.

@datlechin
datlechin enabled auto-merge (squash) November 12, 2026 05:27
@datlechin
datlechin merged commit bf591ee into main Nov 12, 2026
11 checks passed
@datlechin
datlechin deleted the fix/audit-w6-nosql-drivers branch November 12, 2026 08:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant