Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
53 changes: 43 additions & 10 deletions .github/workflows/harvest.yaml
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
# Weekly catalog harvest. Regenerates the catalog from the openwashdata
# GitHub organisation, runs the schema and content gates, and opens or
# updates one pull request with the generated files. The maintainer
# merges it; the harvest never pushes to main.
# GitHub organisation, runs the schema and content gates, opens or
# updates one pull request with the generated files, and merges it in
# the same run once the gates have passed. The pull request stays as the
# report trail, with the maintainer as assignee. Until 2026-10-31 a
# person merged it; seven unmerged weeks showed that step was the weak
# link (#23).
#
# A pull request opened with the built-in GITHUB_TOKEN triggers no
# pull_request workflows, so the gate runs inside this job. R-CMD-check
# and pkgdown run on main after the merge, which is a normal push by a
# person. Do not make R-CMD-check a required status on main, or this
# pull request can never merge.
# The built-in GITHUB_TOKEN creates no workflow runs for the events it
# causes: the pull request triggers no pull_request workflows, so the
# gate runs inside this job, and the merge triggers no push workflows,
# so this job dispatches R-CMD-check, pkgdown and test-coverage on main
# by hand. Do not make R-CMD-check a required status on main, or the
# merge step can never succeed.
#
# A failed run is red in the Actions tab and opens no pull request.
# There is no failure-issue automation.
# A failed harvest or gate is red in the Actions tab, opens no pull
# request and merges nothing. There is no failure-issue automation.

name: harvest

Expand All @@ -22,6 +26,7 @@ on:
permissions:
contents: write
pull-requests: write
actions: write

jobs:
harvest:
Expand Down Expand Up @@ -56,6 +61,7 @@ jobs:
run: Rscript -e 'testthat::test_local(stop_on_failure = TRUE)'

- name: Open or update the harvest pull request
id: pr
uses: peter-evans/create-pull-request@v6
with:
branch: harvest
Expand All @@ -71,4 +77,31 @@ jobs:
commit-message: "chore: weekly catalog harvest"
title: "Weekly catalog harvest"
body-path: data-raw/harvest_report.md
assignees: larnsce
delete-branch: false

# GitHub computes mergeability after the push, so the first attempt
# can meet an unknown state; retry for two minutes.
- name: Merge the pull request
if: steps.pr.outputs.pull-request-operation == 'created' || steps.pr.outputs.pull-request-operation == 'updated'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
PR: ${{ steps.pr.outputs.pull-request-number }}
run: |
for attempt in 1 2 3 4 5 6; do
if gh pr merge "$PR" --squash --repo "$GITHUB_REPOSITORY"; then
exit 0
fi
sleep 20
done
echo "::error::pull request $PR was not merged after six attempts"
exit 1

- name: Run the workflows that a token merge does not trigger
if: steps.pr.outputs.pull-request-operation == 'created' || steps.pr.outputs.pull-request-operation == 'updated'
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
run: |
gh workflow run R-CMD-check.yaml --ref main --repo "$GITHUB_REPOSITORY"
gh workflow run pkgdown.yaml --ref main --repo "$GITHUB_REPOSITORY"
gh workflow run test-coverage.yaml --ref main --repo "$GITHUB_REPOSITORY"
1 change: 1 addition & 0 deletions .github/workflows/test-coverage.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ on:
push:
branches: [main]
pull_request:
workflow_dispatch:

name: test-coverage.yaml

Expand Down
5 changes: 3 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,9 @@ locale; the test gate fails on `<U+xxxx>` escapes in the tables.
## Branches

Branches and pull requests to main. No dev branch. The weekly harvest
bot opens a pull request; it never pushes to main. Do not commit
generated data by hand.
bot opens a pull request and merges it in the same run once the gates
pass (since 2026-10-31, #23); outside that pull request it never
pushes to main. Do not commit generated data by hand.

## Dependencies

Expand Down
17 changes: 17 additions & 0 deletions dev/decisions-2026-09.md
Original file line number Diff line number Diff line change
Expand Up @@ -149,3 +149,20 @@ in `exclude_repos.csv` or the README, and stop counting to 70.

Closed by this file: #2, #7 (as written), #10. Trimmed: #5, #6, #8, #9,
#11, #16. #12 is rewritten to the list above.

## Amendments of 2026-10-31

- Section 5: the harvest job merges its own pull request once the gates
pass, then dispatches R-CMD-check, pkgdown and test-coverage on main,
because a merge by the built-in token triggers no push workflows. The
pull request stays as the report trail, with the maintainer as
assignee. Reason: the maintainer merge was the weak link. #22 waited
from 2026-09-14 to 2026-10-31 while the website served the September
catalog (#23).
- Section 1: archived repositories leave the catalog, also when they
carry a Zenodo DOI (ploswater, archived 2026-09-23). The deposit stays
citable on Zenodo; the catalog lists what the organisation maintains.
- Open question, answered in part: of the six repositories that left
the catalog between the September and October harvests, five were
archived and one (washdataportalsl) went private.

Loading