Skip to content

docs: what a domain's tests owe - #244

Merged
retr0h merged 1 commit into
mainfrom
docs/domain-test-obligations
Oct 2, 2026
Merged

retr0h merged 1 commit into
mainfrom
docs/domain-test-obligations

Conversation

@retr0h

@retr0h retr0h commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

One section in domains.md, one routing row in the skill.

This is the piece of "go implement X and have it wired soup to nuts, fully tested" that was never written down.

The gap

The corpus said where validation is declared and nothing about proving it runs. A tag can be dropped from the specification, the generator quietly stops emitting it, validation.Struct() quietly stops checking it, and nothing fails until somebody sends bad input to production.

The four obligations

Validation is proved to fire, not proved to exist. The test asserts 400 and that the body names the rule that rejected it:

name: "when missing key returns 400",
body: `{"value":"1"}`,
    s.Contains(rec.Body.String(), "Key")

name: "when target agent not found",
    s.Contains(rec.Body.String(), "valid_target")

A test checking only the status code passes just as happily when the handler rejects for an unrelated reason. That is the failure it exists to catch.

The RBAC trio per endpoint. No token is 401, a token without the permission is 403, a token with it succeeds. The third case catches a permission spelled one way in the spec and another in the table.

Every declared status has a path. A documented 404 nobody produces is either undocumented behaviour or a lie in the specification.

Broadcast asserted both directions. One host and forty, because the single-host case is what gets special-cased.

What the audit found

log        0 validation suites   3 RBAC    <- gap
hostname   1 validation suite    2 RBAC    <- gap
user       9 validation suites  14 RBAC    <- gap
schedule   no integration test             <- osapi-io/osapi#563
everything else                            matched

75 validation suites and 80 RBAC suites across the domains, 21 of 22 with an integration test. The obligation was already being met almost everywhere, by habit rather than by rule, which is exactly how three domains drifted without anyone noticing.

Filing the specific gaps separately rather than widening this.

just test passes. All skill links and anchors resolve.

🤖 Generated with Claude Code

https://claude.ai/code/session_01FuKUsHFG1EqZXamffh9M2c

The corpus said where validation is declared and nothing about proving it
runs. A tag can be dropped from the specification and the generator will
quietly stop emitting it, validation.Struct will quietly stop checking it,
and nothing fails until somebody sends bad input to production.

States the four obligations: validation proved to fire rather than proved to
exist, the RBAC trio per endpoint, a path through the tests for every status
the specification declares, and the collection shape asserted for one host
and for many.

The part worth having written down is that a validation test asserts the
body names the rule that rejected the request, not only the status code. A
test checking for 400 alone passes when the handler rejects for an unrelated
reason, which is the failure it exists to catch.

Measured counts included so the next reading can tell drift from a figure
somebody guessed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01FuKUsHFG1EqZXamffh9M2c
@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown

Thank you for contributing to this project! 😊🕹️

@retr0h
retr0h merged commit a37639b into main Oct 2, 2026
5 checks passed
@retr0h
retr0h deleted the docs/domain-test-obligations branch October 2, 2026 04:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant