You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
fix(analytics): collector undercounts past 100k history rows and stores schedule-dependent units #1074
Two correctness gaps in the analytics savings-snapshot collector (internal/analytics/collector.go) surfaced during CodeRabbit review of PR #1049. Both are "heavy lift" changes deferred from that PR (which made them safe/observable) into this follow-up so they can be designed properly.
1. Fixed 100k history fetch cap silently undercounts
Collect calls GetAllPurchaseHistory(ctx, purchaseHistoryFetchLimit) with a hardcoded purchaseHistoryFetchLimit = 100000. Once purchase_history grows past that size the source set is silently truncated, so snapshots undercount with no error signal. Older-but-still-active 1y/3y commitments are exactly the rows most likely to fall off.
Interim mitigation in PR #1049: the collector now logs a loud WARNING when the result fills the cap, so the undercount is observable. The proper fix is an active-only store query (filter by status/expiry server-side) or a paginated/cursor scan, which requires a new method on the config store interface.
2. Persisted units are schedule-dependent
Collect converts monthly values to hourly rates (EstimatedSavings / HoursPerMonth, UpfrontCost / (Term*HoursPerYear), MonthlyCost / HoursPerMonth), but the query layer later does SUM(total_savings), SUM(total_commitment), SUM(total_usage) as if they were additive monthly totals. With the default daily schedule a $720/month purchase contributes only ~$30 to that month's series, and changing the schedule changes the numbers.
Fix options (design decision needed): store schedule-invariant monthly amounts (amortize upfront by term in months), OR weight by elapsed time during aggregation. Either touches the materialized-view definitions in migration 000067 and the integration suite, so it was not changed blindly under the PR #1049 timeline. No consumer is currently misled: the Trends frontend is deferred (#1048).
Verification sweep against main (101f099fb) on 2026-07-27 finds this already resolved.
The collector now reads active-only rows via SQL with no cap and stores schedule-invariant monthly run-rates.
Evidence: commit 5fc0899 (#1049) plus b0233d3.
Recommending close.
Verified resolved at 3c0f8ac: the collector no longer uses a fixed history cap; it reads active commitments through the uncapped SQL active-only query, and it persists schedule-invariant monthly run-rates (upfront amortized over the term in months) that the trends queries and the monthly materialized view aggregate with a per-day SUM followed by AVG, so the collection cadence no longer changes the reported totals. Evidence: internal/analytics/collector.go:95 and :180-190, internal/analytics/postgres_analytics.go:353 (#1221, #1209, migration 000078). Residual axes checked: Term<=0 rows are skipped and counted rather than dividing by zero; nil MonthlyCost leaves usage unknown instead of $0; no other caller of the old capped GetAllPurchaseHistory remains in the collector. Closing as completed; reopen if the behaviour recurs.
Activity
cristim commentedon Jul 27, 2026
Verification sweep against
main(101f099fb) on 2026-07-27 finds this already resolved.The collector now reads active-only rows via SQL with no cap and stores schedule-invariant monthly run-rates.
Evidence: commit 5fc0899 (#1049) plus b0233d3.
Recommending close.
cristim commentedon Sep 2, 2026
Verified resolved at 3c0f8ac: the collector no longer uses a fixed history cap; it reads active commitments through the uncapped SQL active-only query, and it persists schedule-invariant monthly run-rates (upfront amortized over the term in months) that the trends queries and the monthly materialized view aggregate with a per-day SUM followed by AVG, so the collection cadence no longer changes the reported totals. Evidence: internal/analytics/collector.go:95 and :180-190, internal/analytics/postgres_analytics.go:353 (#1221, #1209, migration 000078). Residual axes checked: Term<=0 rows are skipped and counted rather than dividing by zero; nil MonthlyCost leaves usage unknown instead of $0; no other caller of the old capped GetAllPurchaseHistory remains in the collector. Closing as completed; reopen if the behaviour recurs.