Repository navigation
ci(security): enforce fail-closed scan evidence and a runtime dependency release gate - #817
Conversation
|
Merge-order note. PR #813 adds |
There was a problem hiding this comment.
🟡 Changes recommended
The dependency audit rejects the unpublished candidate itself, blocking first-time publication even when its third-party dependencies are clean.
1 open issue
What changed in this PR
This PR strengthens security evidence and release checks for azure-functions-openapi, addressing the security-automation portion of #812 separately from the protected-docs changes.
Changes:
- Expands Bandit coverage and adds classified, redacted scan evidence with expiring exceptions.
- Adds candidate-wheel dependency auditing and Swagger UI CDN integrity verification as release gates.
- Adds regression tests and documents enforcement, recovery, and local-only example defaults.
| File | Description |
|---|---|
tools/security_report.py |
Defines evidence statuses and JSON/Markdown reporting. |
tools/security_exceptions.py |
Validates narrow, owned, expiring exceptions. |
tools/security_evidence.py |
Collects Bandit inventory and Semgrep outcome evidence. |
tools/dependency_audit_policy.py |
Runs and classifies runtime dependency audits. |
tools/browser_integrity.py |
Verifies pinned CDN asset hashes. |
tests/test_security_workflows.py |
Checks workflow gates, scope, and artifact retention. |
tests/test_security_tool_execution.py |
Covers tool execution and failure handling. |
tests/test_security_policy_boundaries.py |
Tests input validation and exception boundaries. |
tests/test_security_evidence.py |
Tests scan classification and redacted reporting. |
tests/test_dependency_audit_policy.py |
Tests advisory blocking and exception handling. |
tests/test_browser_integrity_evidence.py |
Tests rendered integrity attributes and hash classification. |
security/dependency-audit-policy.json |
Establishes blocking runtime advisory policy. |
security/bandit-policy.json |
Records reviewed static-analysis exceptions. |
SECURITY.md |
Documents controls, evidence, recovery, and deployment boundaries. |
pyproject.toml |
Adds type checking for security helpers. |
examples/README.md |
Clarifies local-only authentication defaults. |
docs/contributing.md |
Documents expanded CI scan scope. |
demo/README.md |
Clarifies preview tooling and synthetic-input requirements. |
AGENTS.md |
Documents the new release gate. |
.gitignore |
Excludes generated security evidence. |
.github/workflows/security.yml |
Adds expanded scanning and retained evidence. |
.github/workflows/publish-pypi.yml |
Gates publishing on dependency and CDN checks. |
.github/workflows/dependency-audit.yml |
Reuses helpers for scheduled advisory reporting. |
Review effort: Balanced
|
C2/C3 follow-up: 7914927 records successful action-only Semgrep evidence as INCOMPLETE (explicit action outcome only); 38a1a31 tolerates exactly collector exit 3 while preserving the pinned action gate and tool-error failures. Bandit/Semgrep collectors run always. Shared Make targets cover expanded scan/inventory, type checks, runtime blocking audit, development reporting audit, and browser/SRI; workflows and SECURITY.md use them. make security now includes the expanded inventory scan. Tests: test_semgrep_make_collector_tolerates_only_incomplete and test_local_security_includes_expanded_inventory_scan. CI is green at 9350af8. Integration note with #813: retain its verification wrapper/check-commands and maintenance interpreter; replace only its security recipe with security: security-bandit plus the separate new target block. Retain both security-types and verification mypy paths in pyproject; retain setup from #813 and collector/always changes here in security.yml. No action pins, gate names, matrix, permissions, or browser/SRI gate were weakened. Release-only workflow execution and Azure certification were not dispatched. |
Bandit 1.9.4 prints a "Working..." progress bar to stdout when it scans more than 50 files at INFO log level, so the evidence collector could not parse the JSON that follows and reported TOOL_ERROR. The declared scope is 49 files today, so two new Python files would have turned the gate red. Pass -q to keep stdout pure JSON, and add a regression test that runs the real scanner through run_bandit() on either side of the 50-file threshold.
|
R3 authorization for head The maintainer delegated the merge decision to an Oracle review. The Oracle (AI) evaluated this exact head through code and workflow inspection and independent probes (real Bandit red/green at 49 and 51 files, real pip-audit with an unpublished candidate, live candidate-wheel SRI checks on 3.11/3.14, 107 targeted tests) and returned GO after one NO-GO round that is fixed in this head. This is advisory AI evidence, not line-by-line human review or human approval. The Copilot thread on unpublished candidates is addressed and resolved. Release PRs are not merged by this change. |

Summary
Second half of #812 (security automation and release evidence). Scan results can no longer pass vacuously, runtime-dependency advisories block a release, and the pinned Swagger UI CDN assets are verified against their SRI hashes. The runtime key-forwarding half is in #823 (merged).
Changes
tools/security_evidence.pyscanssrc,examples,demo,scriptsandtoolswith an explicit inventory. Zero files scanned, tool errors, a crash or missing JSON, and unlisted suppressions each fail with a status distinct from findings (PASS|FAIL|TOOL_ERROR|INCOMPLETE). Thirteen reviewed LOW findings are recorded as exact id/file/line exceptions with owner and expiry (2027-01-19); inline#nosecis not allowed.if: always()).security/dependency-audit-policy.json(all known runtime advisories block, unknown severity blocks, exceptions need a reason, owner and at most 90 days, expired exceptions fail, stale ones are reported, no exceptions seeded). Newruntime-dependency-auditjob inpublish-pypi.ymlaudits the exact candidate wheel from thebuildartifact in an isolated venv on Python 3.11 to 3.14, andpublishnow needs it.TOOL_ERRORandINCOMPLETEalso block;workflow_dispatchon the tag is the recovery path.dependency-audit.ymlreuses the same helper and stays reporting-only for runtime-plus-development findings.integrity/crossorigin.tools/browser_integrity.pydownloads the pinned assets and compares sha384 in the release gate (mismatch isFAIL, network failure isTOOL_ERROR).security-results/**JSON plus a Job Summary, uploaded withif: always(); only IDs, paths, lines and severities are recorded.SECURITY.mddocuments what blocks merge, release and what only reports, the exception process, and local-only versus production defaults; example/demo READMEs label anonymous/local-only configs.AGENTS.mdgets a one-line pointer.azure-e2echain, existing job names, all action SHA pins,permissions: contents: readon every new job.Type of Change
Checklist
make check-alland all checks pass (including tests and at least 95% coverage)Verification (Python 3.11): full suite passes at 95.11% coverage; 94 new security tests (97.28% helper coverage);
ruffclean; strict mypy clean on the 5 new tool modules; the three workflow lints exit 0. Real runs: Bandit evidence over 48 files / 8,759 LOC (13 reviewed LOW, zero errors);pip-audit==2.10.1against a venv containing only the built wheel: 10 packages, zero advisories; live CDN check: both assets match the pinned SRI.Not verified: behaviour inside GitHub Actions (the new release job has only run locally) and anything on real Azure. The Bandit/mypy tooling lines may need a trivial reconciliation with #813 (
pyproject.tomltypecheck line,tools/__init__.py).Deliberately unchanged (code/config review findings, not demonstrated vulnerabilities):
infra/main.bicepstill uses shared-key storage/deployment authentication, public networking, and lacks explicit Function App/SCM minimum-TLS settings. Changing these needs real-Azure validation of the e2e certification flow, so they are left for a follow-up.Related Issues
Closes #812
#812 acceptance mapping