Repository navigation
feat(cubejs): warn when update_window is set without incremental - #97
Merged
Merged
Conversation
Cube honours refresh_key.update_window only with incremental: true. Without it every partition in the build range is rebuilt on each refresh tick. abler's BigQuery models had 20 such pre-aggregations since April and rebuilt ~270 partitions an hour on dev, closed months included, every 30 minutes for the busiest two. Cube's compiler is silent about this, so /api/v1/validate (editor save) and /api/v1/validate-in-branch (Model Management API, FraiOS) now return a warning on the update_window line. Non-blocking: valid is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Cube honours
refresh_key.update_windowonly whenincremental: trueis also set (PreAggregationPartitionRangeLoader.js:96,BaseQuery.js:4089). Without it, every partition in the build range is rebuilt each time the refresh key fires, and Cube's compiler doesn't warn.On dev, 20 of abler's BigQuery pre-aggregations had been written like this since April (18 in BQ Prod, 2 in BQ QA). In 11.5 h that came to 3,147 BigQuery builds over 661 partitions. Closed months from January 2026 were rebuilt every 30 minutes. Those models were fixed directly on 2026-10-13 (new versions
c937249eand4a152764). This PR stops the same mistake from being saved silently again.What
utils/preAggregationWarnings.js: walks the compiled cubes and returns one warning per pre-aggregation that hasupdateWindowbut noincremental. Each warning carries the cube'sfileNameand the line of itsupdate_window, so the editor can place the marker./api/v1/validate(client-v2 editor, on save) and/api/v1/validate-in-branch(Model Management API / FraiOS) add these warnings when the model compiles cleanly.validis unchanged, and FraiOS passes warnings through.Test
src/utils/__tests__/preAggregationWarnings.test.js: warns once at the right line, and stays silent afterincremental: trueis added.dailyClubTotalsshape to/api/v1/validatereturnsvalid: trueand the warning ontransactions_daily.yml:14.npm test: 705/707 pass. The 2 "production dependency hardening" failures fail on main too, against my localnode_modules.🤖 Generated with Claude Code