Skip to content

feat(diff-stats): xorq diff analysis klasses — pinned-row stats + styling for keyed-diff frames - #959

Open
paddymul wants to merge 1 commit into
mainfrom
feat/xorq-diff-analysis
Open

feat(diff-stats): xorq diff analysis klasses — pinned-row stats + styling for keyed-diff frames#959
paddymul wants to merge 1 commit into
mainfrom
feat/xorq-diff-analysis

Conversation

@paddymul

@paddymul paddymul commented Aug 3, 2026

Copy link
Copy Markdown
Collaborator

Ports the diff analysis klasses proven as tallyman project artifacts into buckaroo proper. Companion to #958 (which carries the renderer changes: per-bar histogram colors, sparkline rendering); this PR is the analyses themselves and depends only on displayers that already exist.

What

A new buckaroo/customizations/xorq_diff_stats.py with three phase-2 summary stats over keyed-diff frames — tables carrying {col} / {col}_v2 / {col}_pct_delta triples — plus the styling klass that assembles the view:

  • diff_histogram — the per-column change distribution in log2-ratio space, 11 bins with an edge pinned at exactly zero change. A column where everything shrank reads as all mass left of zero at a glance.
  • diff_line — new/old as a percent (100 = unchanged), one point per row in key order.
  • left_right — the before series (lineGray) and after series (lineRed) on one chart per column, raw units, so both share a scale and a collapsed after-series reads as exactly that.
  • All three resample to at most 100 points by averaging position-ordered buckets (floor(row_number × min(100, n) / n)), so they stay readable at any row count; at ≤ 100 rows this degenerates to one point per row. On columns that are not the _v2 side of a triple they return [] without issuing a query.
  • DiffStyling — hides the helper columns (_pct_delta, _abs_delta, _eq, membership, the old-value column), renames _v2 to the bare metric, paints cells via color_from_column from a precomputed {col}_cellcolor column, shows the old value on hover, and pins the three stats. Inert on frames without _cellcolor columns, so it can safely serve as a main override.

Unlike the tallyman prototypes (sandboxed compute(col) stats that had to reach sibling columns through col.op().rel and smuggle results out as array-of-struct scalar subqueries), these use the phase-2 XorqExpr + XorqExecute idiom the built-in histogram already uses — own query, post-processed in Python.

Why

A diff view's cells can only color one number at a time. The summary rows carry the rest of the story: the shape of the change distribution, and the before/after trajectories on a shared scale. Prototyped and validated against real diffs (NYC taxi by-hour, 24 rows, and a 1440-row synthetic) in tallyman.

Not wired into XORQ_STATS_V2 defaults — exported as XORQ_DIFF_STATS for consumers to opt in via extra_klasses, the same path project-authored stats use.

Tests

TDD structure: this PR's first commit carries only the failing tests (red on the missing module); the implementation lands in the follow-up commit.

tests/unit/test_xorq_diff_stats.py — bin labels and zero-edge semantics, all-decrease mass lands left of zero, per-row points under the cap, exact-100-point resampling with bucket-mean values, [] on non-diff columns, log-safety at pct_delta == -1, and DiffStyling hide/rename/color/tooltip behavior plus inertness without _cellcolor.

Column-naming note: compare.col_join_dfs uses the |df2 suffix convention; these stats use the _v2 / _pct_delta convention of the keyed-diff builder they were proven against. Unifying the two shapes is deliberate follow-up.

🤖 Generated with Claude Code

Red per TDD for the diff analysis klasses proven in the tallyman
diff-view prototypes: three pinned-row summary stats over keyed-diff
frames ({col} / {col}_v2 / {col}_pct_delta triples) plus the styling
klass that assembles the diff view.

- diff_histogram: change distribution in log2-ratio space, 11 bins with
  an edge pinned at zero change
- diff_line: new/old as a percent (100 = unchanged), in key order
- left_right: before (lineGray) and after (lineRed) series on one chart
  so both share a scale
- all three resample to <= 100 points by position-bucketed means and
  return [] on non-diff columns
- DiffStyling: hides helper columns, renames _v2 to the bare metric,
  paints via color_from_column from {col}_cellcolor, old value on
  hover; inert on frames without _cellcolor columns

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: aff733059d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +32 to +35
from buckaroo.customizations.xorq_diff_stats import ( # noqa: E402
DIFF_HISTOGRAM_LABELS,
XORQ_DIFF_STATS,
DiffStyling)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Add the diff stats module before importing it

In xorq-enabled test environments this import is executed, but this commit only adds the test file and does not add buckaroo/customizations/xorq_diff_stats.py; git ls-tree for the reviewed commit shows no such module. As a result pytest collection raises ModuleNotFoundError before any of these tests can run, so the xorq unit suite is broken until the implementation is included or the tests are gated differently.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant