Skip to content

Use outer join to get data from both tables - #96

Merged
tofu-rocketry merged 4 commits into
apel:devfrom
garaimanoj:gt-1750-sites-missing-from-sync-views
Sep 10, 2026
Merged

tofu-rocketry merged 4 commits into
apel:devfrom
garaimanoj:gt-1750-sites-missing-from-sync-views

Conversation

@garaimanoj

@garaimanoj garaimanoj commented Sep 8, 2026 •

Copy link
Copy Markdown
Contributor

Resolves GT-1750
Resolves #36

@garaimanoj
garaimanoj marked this pull request as ready for review September 9, 2026 10:26
@garaimanoj garaimanoj self-assigned this Sep 9, 2026
@tofu-rocketry tofu-rocketry added this to the v2.1 milestone Sep 9, 2026

@tofu-rocketry tofu-rocketry left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The code looks good, but it's the commit messages again. See notes below. Ensure that they're formatted correctly, i.e. short header line then longer body where necessary.

Use outer join to get data from both tables

  • This isn't a good explanation of an outer join - we're already getting data from both tables - so you really mean we're keeping rows even if they don't fully match between the tables
  • Also need a "why" we're doing this. Something about showing a wider variety of accounting data issues.

Allow null to show the records which is not available in both tables

  • Explain why these records might not be available.

Check for nan to allow display of records not present in both tables

  • "NaN" rather than nan. 👵🏻
  • Need to add something about this being to ensure that NaNs aren't passed onwards.
  • Also explain addition of the helper function.

Check RecordCountInDb to note the omission

  • What does this mean? Do you mean handle it separately so that we can reduce the WARNING to an OK?

Fix indentation

  • Fine.

Use an outer join when combining summary and sync record data so
that rows are retained even when they only exist in one source.

This exposes a wider range of accounting and publication issues,
including records that are missing from either VSuperSummaries or
VSyncRecords.
Some records can exist in VSyncRecords without a corresponding
published summary record. Allow RecordStart and RecordEnd to be
null so these records can be displayed.
Convert Pandas NaN and NaT values to None for dates and zero for counts
before storing records and add a helper function to centralise
the conversion logic.

This prevents invalid values being passed to Django model fields
and ensures unmatched records are displayed correctly.
Avoid flagging records as invalid solely because sync
record is unavailable.

Fix indentation
@garaimanoj
garaimanoj force-pushed the gt-1750-sites-missing-from-sync-views branch from 2f0844a to 9cc8e63 Compare September 10, 2026 09:01

@tofu-rocketry tofu-rocketry left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good. Thanks.

@tofu-rocketry
tofu-rocketry merged commit 49ff5f5 into apel:dev Sep 10, 2026
4 checks passed
@garaimanoj
garaimanoj deleted the gt-1750-sites-missing-from-sync-views branch September 10, 2026 13:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

Issue with fetching records with the Year and the Month for specific sites

2 participants