Repository navigation
ci: add coverage report to tests workflow - #119
adityamulik wants to merge 3 commits into
Conversation
Run pytest with term-missing/xml/html coverage, publish a markdown coverage table to the Actions job summary, and upload coverage.xml and htmlcov/ as an artifact. Adds pip caching to speed up installs. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add py-cov-action to the Tests workflow and a companion "Post coverage comment" workflow triggered via workflow_run, so coverage is posted as a PR comment even for fork-originated PRs (whose pull_request runs get a read-only token). Set coverage relative_files = true so paths resolve across the two workflows' checkouts. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add --cov-fail-under=60 to the pytest run so the Tests check fails on regressions below the threshold. Current coverage is ~61%. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
drk-ram
left a comment
There was a problem hiding this comment.
Reviewed all three changed files at f42b64887e10a3a0047e440d6bcdb48b4ccdcbb8. Approving the coverage integration.
The 60% gate is enforced by pytest's exit status, independently of whether a comment can be posted. XML/HTML artifacts and the summary run even after a test failure. The fork path leaves test execution in pull_request and uses a separate workflow_run reporter without checking out or executing the contributor's source. relative_files = true is consistent with coverage data consumed across workflow contexts.
Evidence:
- The exact-head Tests run passed all 182 tests and reported 61% coverage; its log also confirms creation of the deferred-comment artifact.
- Locally,
python -m pytest tests -o addopts= -q --tb=short -k "not test_generate_uses_aiofiles_async and not TestFixtureEndToEnd" --cov=src --cov-report=term --cov-report=xml --cov-report=html --cov-fail-under=60passed: 178 tests, 4 deselected, 61.20% coverage, with both report formats generated. python -m coverage report --format=markdownpassed.
Local scope: three tensor-fixture tests failed in an earlier broader run because this environment lacks torch; those three and one pre-existing test that hardcodes a POSIX output path were excluded from the passing subset. The complete-suite result above comes from the published exact-head run, not that subset. End-to-end posting by the new privileged workflow remains unverified until the workflow is present on the default branch.
Non-blocking hardening: pin the new third-party action to an audited commit rather than a moving v3 tag. Also consider narrowing the reporter's contents grant: editing a PR comment needs pull-requests: write, not repository-content write. Keep any content-write permission required for coverage-data branch publication confined to the path that actually performs that operation.
Add test coverage reporting and a CI coverage gate
Sets up coverage reporting for the test suite and surfaces it in CI: