perf(remote): avoid full journal snapshots during event accounting - #285
Open
sambitcreate wants to merge 1 commit into
Open
sambitcreate wants to merge 1 commit into
sambitcreate wants to merge 1 commit into
Conversation
Contributor
There was a problem hiding this comment.
✅ No new issues found.
Reviewed changes I reviewed the stream-accounting implementation, its behavioral tests, and the reproducible benchmark documentation.
- Exact budget accounting Replaces full retained-journal serialization on append with per-event UTF-8 sizes and bounded metadata-envelope accounting.
- Journal ownership and ordering Clones payloads, accounts normalized restore data, and applies final stream state and timestamp before checking the budget.
- Behavioral validation Adds byte-oracle coverage for Unicode, restore and cleanup, trimming, aggregate pressure, and delivery, plus a synthetic benchmark.
GPT Luna | 𝕏
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Every Remote event previously deep-cloned and serialized the entire retained journal to enforce its byte budget, even with persistence disabled. Account for immutable owned event bytes plus the bounded metadata envelope instead, retaining exact UTF-8 JSON limits and existing replay/terminal-delivery behavior. Apply final state/timestamp before checking the budget.
The identical synthetic workload now performs zero full snapshots for 64 appends, down from 64, with byte-identical final snapshots at both retention sizes. Timings are exploratory; persistence still snapshots normally and no GPU/energy savings are claimed.
Validation: 62 stream tests; broader Remote suites 533 passed with one environmental occupied-port skip; TypeScript and scoped lint passed. Fresh-context GPT-6 Astra medium review found no actionable issues, independently passed the 62 tests, and passed 7,452 exact-byte oracle comparisons across Unicode, escaping, timestamps, resets, terminal transitions, pruning and revocation. Native consumers were inspected; wire/recovery contracts are unchanged.
Evidence and reproducible fixture:
docs/performance/remote-stream-accounting.mdandremote-stream-accounting-benchmark.mts.