Skip to content

dialect aware sketch decomposition - #2559

Open
robinld wants to merge 24 commits into
bdealmeida/reaggregate-paramsfrom
robind/dialect-aware-sketch-decomposition
Open

robinld wants to merge 24 commits into
bdealmeida/reaggregate-paramsfrom
robind/dialect-aware-sketch-decomposition

Conversation

@robinld

@robinld robinld commented Sep 21, 2026

Copy link
Copy Markdown
Contributor

No description provided.

betodealmeida and others added 21 commits September 4, 2026 12:31
# Conflicts:
#	datajunction-server/datajunction_server/models/deployment.py
Persist reaggregation parameters on frozen measures and include them in measure identity and pre-aggregation matching. Thread the parameters into decomposition and Druid metric spec construction, with migration and API coverage.
Pass the output dialect and source aggregation call through metric decomposition so dialect-specific combiners can be rendered correctly. Add family-based registration so sketch decompositions can be selected explicitly through a metric's reaggregation spec.

@shangyian shangyian left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add some tests that assert the setup with the full generated SQL end-to-end, at both the measures layer and the final metrics layer? Just to verify that it all looks reasonable

Comment thread datajunction-server/datajunction_server/api/sql.py
Comment thread datajunction-server/datajunction_server/models/decompose.py Outdated
Comment thread datajunction-server/datajunction_server/models/decompose.py Outdated

Args:
node_revision_id: ID of the metric node revision
dialect: Dialect the combiner will be rendered for. Combiners are

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there cases where there is no combiner for Spark but exists one for Druid?

Comment thread datajunction-server/datajunction_server/config.py Outdated
Robin Davis added 2 commits September 22, 2026 10:10
Allow components to serialize accumulated values for specific materialization targets and declare the resulting type. Add multi-argument type inference and fixed merge arguments so t-digest measures retain the correct representation and compression when written to Druid.
@robinld
robinld force-pushed the robind/dialect-aware-sketch-decomposition branch from bfba773 to a2c80ce Compare September 22, 2026 18:22
@betodealmeida

Copy link
Copy Markdown
Member

@robinld do you want to change the base to #2568?

@robinld
robinld changed the base branch from robind/reaggregate-params to bdealmeida/reaggregate-params September 24, 2026 04:02
@robinld

robinld commented Sep 24, 2026

Copy link
Copy Markdown
Contributor Author

@robinld do you want to change the base to #2568?

I just did but I'm not sure it's doing what we want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants