Contributions, contributor pages, fetch work and the CLI workerP3/r2 engine

[andreimarinescu]

What changes

Article contributions

• A contributing service principal holds the articles:contribute scope.
• The contribution PUT has its own rules: it must be published with no sensitivity class, the expected version is required, a lower tier is refused, and the title and slug are fixed.
• It stamps attribution: origin, contributors and version_contributor. Community records keep 20 versions.
• POST /v1/articles/{id}/revert restores a kept version as a new one.
• Articles list filters: status=pending and tier=.
• /v1/contents reports a document's source, and for an attested one verified_by.

Public article pages

• Contributor attribution, with a generic name for a banned or unresolved contributor.
• The CC BY-SA 4.0 line.
• New pages: /wiki/style (and .md), /wiki/terms and /wiki/wanted.
• A safe Markdown form at /wiki/<slug>.md.
• /llms.txt, with its Caddy route.
• A "My contributions" view in the app, backed by GET /api/contributions.

Fetch work

• New internal/work package: a pool fed by blocked pages, pushed pages and the frontier; leases; attestations grouped by agreement, with the quorum requiring distinct accounts and networks; a server re-fetch after the quorum window; moderation under a daily ceiling; and indexing of the verified text.
• Engine routes: /v1/work/{claim,submit,push,settled} under the work:broker scope, plus /admin/work-purge, /admin/work-void and cosift work purge|void.
• Switches: the config work block, COSIFT_WORK_ENABLED and COSIFT_WORK_FRONTIER.
• The community app brokers the work through POST /api/work/claim and /api/work/submit: bearer token only, a join check, daily fetch credits for new content, counters and the ceiling alert.
• App switches: COSIFT_COMMUNITY_WORK, COSIFT_COMMUNITY_FETCH_DAILY_CAP and COSIFT_ENV.

CLI worker

• cosift contribute -worker [-max-pages] [-duration] [-concurrency].
• It fetches like a browser through the public-only dialer, re-checks every redirect hop, and extracts exactly as the server's fetch does.
• PDFs are parsed in a child process.
• It is bounded and stops cleanly on SIGINT, with progress printed as JSON lines.
• -index-locally now shares the same fetch and extraction.

Community app

• A per-account daily share of the content checks (COSIFT_COMMUNITY_MODERATIONS_PER_ACCOUNT, default 100), so one account cannot use the whole day's allowance.
• A clawback is applied even while the account is suspended or unverified, and is never given up on.
• Dispatch takes accounts in rotation.

Docs

• ENV.md, COMMUNITY.md, HANDOVER.md, API.md, AGENT-SETUP.md, SHARED-ACCOUNTS.md, README.md and the cosift skill now describe the above.

Every new path is off until its switch is set.

Tests

• gofmt and vet are clean.
• go test -race ./... passes, 25 of 25 packages.
• All 5 release targets build.
• The web tests pass (49).
• The Caddyfile validates under Caddy 2.11.3.
• Every commit builds and vets on its own.
• No dependency changes.

[codecov]

Codecov Report

❌ Patch coverage is 89.36479% with 293 lines in your changes missing coverage. Please review.

Files with missing lines	Patch %	Lines
internal/articles/revert.go	85.86%	20 Missing and 6 partials ⚠️
cmd/cosift/serve_v1_work.go	86.50%	17 Missing and 5 partials ⚠️
cmd/cosift/serve_work_deps.go	75.55%	15 Missing and 7 partials ⚠️
internal/crawler/crawler.go	69.11%	17 Missing and 4 partials ⚠️
internal/community/contributions.go	87.68%	12 Missing and 5 partials ⚠️
cmd/cosift/worker.go	94.32%	11 Missing and 5 partials ⚠️
internal/community/work_push.go	76.11%	8 Missing and 8 partials ⚠️
internal/community/work_submit.go	82.41%	10 Missing and 6 partials ⚠️
internal/crawler/fetchone.go	42.30%	9 Missing and 6 partials ⚠️
internal/community/work_routes.go	88.59%	9 Missing and 4 partials ⚠️
... and 22 more

📢 Thoughts on this report? Let us know!