Skip to content

test: prove candidate validation behavior on a real Functions host - #603

Merged
yeongseon merged 3 commits into
mainfrom
test/candidate-validation-host
Oct 26, 2026
Merged

yeongseon merged 3 commits into
mainfrom
test/candidate-validation-host

Conversation

@yeongseon

Copy link
Copy Markdown
Owner

Summary

Replace the unrelated Cookbook HTTP host-boot invocation with repository-local candidate-wheel validation behavior, retaining the existing release gate job ID.

Changes

  • Install the wheel in an isolated environment outside the checkout and verify version/import origin.
  • Prove indexing, request/context binding, custom trigger name, body injection, valid response filtering, malformed JSON 400, model-error 422 and sanitized response-error 500.
  • Require a deliberately broken-wrapper control to fail worker indexing; retain host logs and HTTP evidence.
  • Run the proof in required host CI and release, uploading evidence with always(). Keep downstream import smoke and Azure certification unchanged.

Validation

  • A workflow regression test failed before implementation.
  • Python 3.11 make build and make host-candidate passed with local Core Tools 4.8.0. Positive host executed all HTTP probes; no-op wrapper produced FunctionLoadError on return-annotation mismatch.
  • make check-all passed: 763 passed, 27 deselected, 4 warnings in 18.16s; coverage 97.15%.
  • bash -n and git diff --check passed.
  • GitHub CI pending. No cloud commands run; local evidence is not Azure certification.

Related Issues

Closes #574

Merge authorized by the maintainer’s explicit instruction.

@yeongseon

Copy link
Copy Markdown
Owner Author

Reviewed candidate isolation, local-process ownership, positive probes and indexing-specific negative control. Merge authorized by the maintainer’s explicit instruction. Exact head 92ad377 has success for bandit, semgrep, Analyze, PR title and ci-required; no unresolved review threads. Required host matrix ran the candidate proof on Python 3.11 and 3.12. Local summary: 763 passed, 27 deselected, 4 warnings in 18.16s; coverage 97.15%. No Azure certification claimed.

@yeongseon
yeongseon merged commit 92d0406 into main Oct 26, 2026
26 checks passed
@yeongseon
yeongseon deleted the test/candidate-validation-host branch October 26, 2026 14:12

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The response-error probe accepts generic HTTP 500 failures without proving that the library returned its sanitized error envelope.

1 open issue
What changed in this PR

This PR replaces the unrelated Cookbook host-boot check with repository-local validation tests against the built wheel on a real Azure Functions host, addressing #574.

Changes:

  • Adds isolated wheel installation, identity checks, HTTP behavior probes, and a broken-wrapper negative control.
  • Integrates the proof into host CI and the existing release gate, with evidence uploads.
  • Documents local verification while preserving downstream import smoke and real-Azure certification.
File Description
tests/​test_host_release_contract.py Checks release-gate wiring and evidence-upload configuration.
tests/​host_smoke/​function_app.py Adds validation routes and a broken-wrapper mode.
scripts/​check_candidate_host.sh Installs the candidate and runs host behavior checks.
Makefile Adds the host-candidate entry point.
docs/​candidate-host.md Documents verification scope, execution, and evidence.
AGENTS.md Updates release-tier descriptions and local instructions.
.github/​workflows/​publish-pypi.yml Replaces Cookbook host boot with candidate validation.
.github/​workflows/​ci-test.yml Adds candidate verification and evidence uploads to host CI.

Review effort: Balanced

Comment on lines +59 to +60
[ "$status" = 500 ] || return 1
! grep -q 'must not leak' "$output/response.json"
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

test(release): replace unrelated Cookbook host-boot gate with Validation host behavior proof

2 participants