Skip to content

Trimmed the Redis streams by retention on every append. - #79

Closed
moetemp wants to merge 3 commits into
moe/AI-198-o7-py-8-redis-providerfrom
moe/AI-198-o7-py-9-redis-trimming
Closed

moetemp wants to merge 3 commits into
moe/AI-198-o7-py-8-redis-providerfrom
moe/AI-198-o7-py-9-redis-trimming

Conversation

@moetemp

@moetemp moetemp commented Oct 5, 2026

Copy link
Copy Markdown
Owner

This PR trims the Redis provider's logs by age, and optionally by count, on every append.

What changed?

RedisStreams(retention=, max_len=) takes DEFAULT_RETENTION at seven days and no count cap by default, and checks both at construction. The append script trims exactly, in the same call as the append. A stage at or above max_len is refused where it's staged. A replay that reaches a trimmed range fails its task with StreamIntegrityError, which the transport no longer wraps as a storage error. An outside cursor below the window is refused with StreamCursorError. The Redis conformance case opts out of the shared trimmed-cursor case, since this provider refuses a trimmed cursor on the first step rather than at the call, and its own live module covers that floor. New tests cover the bounds, the default, the loud replay failure, the age trim and the batch the window can't hold.

Part of AI-198 (epic AI-37).

Why?

Without a trim, a Redis log grows forever. The trim has no consumer floor, because keeping one would need a consumer registry in Redis. So the failure modes are made loud instead: a replay past the window fails rather than replaying something else. Trims are exact because Redis's approximate trim never touches a log shorter than one macro node.

How did you test it?

Link to a test plan if any -

  • Unit Tests
  • Staging
  • End to End Tests

The lint set is clean. The external stream suite and tests/streams pass on the dev server the fixtures start and with -E against a local channel server. tests/streams with STREAMS_LIVE=redis passes against that server and a local Redis.

A key is trimmed to a seven-day window by default, with an optional count cap,
on every append the provider makes. A replay that reaches a trimmed range fails
its task loudly, and an outside cursor below the window is refused.
The cases check the constructor's bounds, the seven-day default, a replay and a
read past the window, the age trim, and a batch the window cannot hold.
@moetemp moetemp added the skip-changelog Changelog entry rides another PR label Oct 5, 2026
@moetemp

moetemp commented Oct 6, 2026

Copy link
Copy Markdown
Owner Author

Replaced by #107 in the v5 series (phase 1, Signal wake only). The branch stays as a pin.

@moetemp moetemp closed this Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

skip-changelog Changelog entry rides another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant