Conversation
✅ Deploy Preview for thriving-cassata-78ae72 canceled.
|
shangyian
force-pushed
the
feat/typed-default-value
branch
from
September 24, 2026 05:24
1b4cd3a to
1838573
Compare
A dimension link's `default_value` could only ever be a string. The field was typed `str` at every layer, so `default_value: 0` failed validation, and the emitted literal was always single-quoted, so the workaround of authoring `"0"` produced COALESCE(int_col, '0'). The feature was unusable on any non-string dimension column. The stored value is now JSON (jsonb on postgres) and accepts a string, number, or boolean. How it renders is decided by the type of the column it wraps, not by how it was authored, so links already carrying "0" against a numeric column start emitting COALESCE(col, 0) with no migration of anyone's YAML. A link's default applies to every dimension column reached through it, so one link can serve columns of different types and there is no coherent write-time check to make. Resolution happens per column at build time instead, and it never raises: a value the column's type cannot hold falls back to the quoted string it produces today. Spark implicitly casts COALESCE(int_col, 'Unknown') to string, so raising there would break setups that currently work. Supersedes DataJunction#2406, whose JSON-storage and column-type-wins approach this carries forward. Fixes DataJunction#2377 Co-Authored-By: Beto Dealmeida <bdealmeida@netflix.com>
Python counts 1, 1.0 and True as equal, so widening default_value to accept numbers and booleans meant a swap between them read as no change at three points: the deploy spec comparison key, the orchestrator's exact-match check, and -- one layer down -- SQLAlchemy's own attribute history, which decides whether to include the column in an UPDATE at all. The first two left the deploy reporting a noop; the third let the deploy report an update while the old value stayed in the database. Both link write paths now flag the column modified explicitly, and the two value comparisons go through a shared key that carries the type.
shangyian
force-pushed
the
feat/typed-default-value
branch
from
September 24, 2026 07:45
308e3b8 to
902d9b3
Compare
shangyian
marked this pull request as ready for review
September 24, 2026 07:45
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A dimension link's
default_valuehas only ever worked for string columns.The stored value is now JSON and accepts a string, number, or boolean. How it renders is decided by the type of the column it's referring to. That means links with a default value of "0" for a numeric column will emit
COALESCE(col, 0).Pulled the relevant changes from #2406.
Test Plan
default_valueon dimension links doesn't work for non-string columns #2377make checkpassesmake testshows 100% unit test coverage