Weekly harvest stops at PR #22: main and the website serve the 2026-09-09 snapshot #23

What I saw

The catalogue CSV that the website reads (inst/extdata/owd_packages.csv on main) still shows gwinfraassess at version 0.0.0.9000 with no DOI, while the package itself is at 1.0.1 and its CITATION.cff carries 10.5281/zenodo.21839342. On main, 32 of 64 packages have no DOI.

Cause

The weekly harvest works as designed, but its results never reach main. The workflow runs every Monday, passes its gates, and updates pull request #22 (open since 2026-09-14, last updated 2026-10-26, mergeable, no checks, no comments). Nobody has merged it, so main still holds the harvest of 2026-09-09, the last one merged (#21).

The current harvest on the harvest branch has gwinfraassess at 1.0.1 with its DOI and 10 packages without a DOI instead of 32.

Effect on the website

Both the homepage numbers strip (index.qmd) and the data gallery page read the CSV from main by raw URL. The live site therefore shows the September snapshot: 64 packages, 99 datasets, 32 DOIs missing. The website's own refresh workflow re-renders from main on Monday 09:00 UTC, so it also sees nothing until the pull request is merged.

What changed between the two harvests

Checked on 2026-10-31 against the repositories and the harvest code; nothing in the harvest looks wrong, and #22 can merge as it is.

• 60 packages instead of 64: five repositories were archived on 2026-09-23 and 24 (ploswater, whatawaste, whdxdata, wasteburningblantyre, wastecollectionblantyre) and the harvest keeps unarchived repositories only; washdataportalsl went private, which the workflow token cannot see; kalaiwash and waterqualityuganda are new. ploswater carries a DOI (10.5281/zenodo.14616993) and is archived, and no rule says what happens to an archived repository with a deposit.
• Datasets: the homepage figure, the sum of n_datasets, goes from 99 to 93. (An earlier version of this issue said 78; that was the row count of owd_datasets.csv, which holds published packages only, so the two numbers were not comparable.)
• 18 packages flipped to published because their CITATION.cff gained a DOI after 2026-09-09; published means that the CITATION.cff carries a Zenodo DOI.

Related: the website refresh fails

The website's refresh workflow failed on 12, 19 and 26 October: the dataset of the month chunk loads tidyr, and tidyr is not in the workflow's package list. Merging #22 changes nothing on the site until that is fixed in openwashdata/website.

Decision

The workflow comment says the maintainer merges the weekly pull request and the harvest never pushes to main. With a human merge in the loop, the catalogue goes stale whenever the merge is missed, which has been the case for seven weeks. Options:

1. Keep the human merge and add a reminder (a notice when the pull request has waited more than a week).
2. Merge automatically when the gates pass, and keep the human review for the report only.
3. Let the website read the harvest branch, or a published artefact of the harvest, instead of main.

A plan with the trade offs follows.