Conversation
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
shangyian
changed the title
`MaterializationSpec.retention` was read on export but never written. #2403 wired the field through the API path, the config builder and the job that consumes it, but not through `internal/materializations.py`, so a block declared in YAML reached `UpsertCubeMaterialization` without it and the upsert took the field default. The stored config held 400 DAYS whatever the author asked for. Two things followed. The cube was materialized with a retention nobody declared, which for a block covering years of history means Druid dropping data the cube is supposed to serve. And because the export reads retention back out of that config, a cube declaring anything else compared unequal to its own YAML on every deploy -- reported as an update, rescheduled, forever. All three places that build an upsert dropped it: the initial build, the revision-swap rebuild, and `_upsert_from_materialization`, which recovers intent from a stored row. That last one loses retention with no YAML change involved at all -- a cube materialized correctly reverts to the default on its next revision swap -- so it reads the stored value with the same default the export uses, and a config predating the field rebuilds as export already reports it. The deployment test asserts the round trip rather than the stored value, since it is export-equals-declaration that the comparison depends on. Existing rows keep the retention they were built with, so the first deploy after this lands is a real update for any cube that declared another value.
shangyian
force-pushed
the
cube-retention-dropped
branch
from
September 20, 2026 15:38
2fe30aa to
905829f
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.