perf: precompute org page contributors leaderboard - #4501
Conversation
Signed-off-by: anilb <epipav@gmail.com>
PR SummaryMedium Risk Overview Adds a nightly replace-mode copy pipe ( Reviewed by Cursor Bugbot for commit 0686845. Bugbot is set up for automated code reviews on this repo. Configure here. |
|
Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability. Example:
Projects:
Please add a Jira issue key to your PR title. |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 0686845. Configure here.
| TYPE COPY | ||
| TARGET_DATASOURCE org_page_contributors_copy_ds | ||
| COPY_MODE replace | ||
| COPY_SCHEDULE 50 3 * * * |
There was a problem hiding this comment.
COPY schedule collides overnight
High Severity
COPY_SCHEDULE 50 3 * * * overlaps activityRelations_by_member_copy (starts 35 3, ~22 min over a full activity scan) and spills into the 4 AM collection-bucket band. This full scan of activityRelations_deduplicated_cleaned_bucket_union (~744M rows) can push concurrent copy jobs over the account quota of 12, leaving pipes permanently queued and emptying downstream datasources.
Triggered by learned rule: Tinybird COPY pipe schedules must not collide — account concurrent quota is 12
Reviewed by Cursor Bugbot for commit 0686845. Configure here.


Optimizes
org_page_contributorsendpoint pipe with a nightly dedicated precompute copy pipe