Add generated SDK contract tests for spec 3 sections 14.2, 14.6, and 18.5

[pevans]

Summary

This PR adds hand-written contract tests for the generated v4 Python SDK. The tests check spec 3 section 18.5 and the generated-code requirements of sections 14.2 and 14.6 (ZEPAI-3706). .fernignore lists tests/contract/, so Fern does not overwrite the tests.

tests/contract/test_generated_contract.py holds a table of the spec 3 section 4.2 endpoint map (SDK method, HTTP method, path, P, POST read). The tests check these items on Zep and AsyncZep:

• Each section 4.2 method exists, and the client has no other public method. The 20 /abac and docs-audience operations are absent.
• Each P method returns SyncPager or AsyncPager. With httpx.MockTransport, batch.list (GET) and user.list (POST read) iterate two pages, send cursor=c1 on the second request, stop on the page without next_cursor, and return each item one time.
• Only a state-changing method has an idempotency_key parameter. A caller key is sent unchanged and is sent again on a retry after a 503. project.get and user.list send no Idempotency-Key.
• The client sends Authorization: Api-Key <key>.
• graph.get_context recency_bias is the string enum off, mild, strong. thread.get_context does not type it as an object.
• No public class or method name outside zep_cloud.core and the ontology DSL contains lastn, uuid_cursor, group_id, or scope.

The table does not mark agent.learning.get as P. https://github.com/getzep/zep-proprietary/pull/6715 removes that mark from spec 3 section 4.2, because the endpoint returns one AgentLearningState and has no cursor.

The existing tests/ontology/ tests run against the v4 client without a change.

Local result: pytest tests/contract tests/ontology: 431 passed, 353 expected failures. mypy passed. ruff check and ruff format --check passed on the new file.

Expected failures

Each expected failure is strict. When a later generation fixes the gap, the test fails and tells the developer to remove the entry. The v4 branch holds the 4.0.0-alpha.5 code, which is older than the current spec 3 contract on zep-proprietary main. These are the open gaps:

ID	Gap	Owner of the fix
Alpha.5	68 section 4.2 operations are absent from the alpha.5 code. 24 POST-read methods expose an idempotency option. The table also marks agent.split.plan as a POST read, because spec 3 section 4.2 will classify it as a read (https://github.com/getzep/zep-proprietary/pull/6715). The operation is absent from alpha.5, so its test is a strict expected failure on that version.	The next regeneration from the current contract. The tests are version-gated to 4.0.0-alpha.5.
D1	A state-changing call without a caller key sends no Idempotency-Key. The generator configuration does not enable automatic key generation, and the pinned Fern toolchain cannot enable it. https://github.com/getzep/zep-proprietary/pull/6716 gives the documentation sources. This gap is a post-GA follow-up and does not block GA: the server accepts a write without a key, and the fix does not change a method signature.	https://linear.app/zep/issue/ZEPAI-3750

CI and release gate

The ci.yml workflow of this repository runs the full test command on every pull request, so it runs the new tests. The zep-proprietary SDK Release workflow runs gh pr checks on each generated SDK pull request and stops the release on a failed or pending check. Thus a regeneration that breaks these tests stops the release.

https://github.com/getzep/zep-proprietary/pull/6716 maps these tests in docs/specs/api/test-coverage.yaml under sections 14.2, 14.6, and 18.5.

Changes after the adversarial review

• The D3 administrative-auth test and its reason are removed. Spec 3 section 14.1 now states that the SDK document declares only the project API-key scheme, and that the bearer-only ABAC operations are outside the SDK audience (https://github.com/getzep/zep-proprietary/pull/6715). The test checked a boundary that the SDK does not promise. We did not add a bearer scheme to the generated clients, because no SDK customer needs it.
• The D1 expected failures now name ZEPAI-3750 as the follow-up. Classification: deferred follow-up.

Invariants

None

Required configs

None

[devin-ai-integration]

I'll fix CI failures and address comments from users with write access that start with 'Devin'.

• Disable automatic comment, CI, and merge conflict monitoring

[devin-ai-integration]

👀 2 findings need your review

Devin reviewed all 2 findings on 521f302 and left them for you. Click a finding below to jump to its comment.

For your review (2)

• Most endpoint routes remain untested
• Missing thread recency bias passes

View all findings in Devin Review

[devin-ai-integration]

Devin Review found 2 potential issues.

[devin-ai-integration]

🔍 Most endpoint routes remain untested

test_client_methods_match_section_4_2 checks method names but never checks its HTTP method or path columns. A regenerated operation can target the wrong route while this contract check passes.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

I did not change the test. Spec 3 section 18.5 requires that the generated namespaces and method names match section 4.2, and this test checks that requirement. The routes come from the v4 OpenAPI document, and Fern copies them to the generated code. The zep-proprietary test sdk-compat/tests/test_coverage.py::test_v4_chi_and_openapi_paths_match (section 14.3) checks that the OpenAPI paths match the server routes. A route check for each of the 167 operations in this file would duplicate that check. The HTTP method and path columns stay in the table as reference data for the mock-server tests.

[devin-ai-integration]

🔍 Missing thread recency bias passes

test_thread_context_recency_bias_is_not_an_object passes when recency_bias disappears entirely. If the contract requires this option, assert its presence before checking its type.

---

Was this helpful? React with 👍 or 👎 to provide feedback.

[devin-ai-integration]

I did not change the test. Spec 3 section 7.4 makes recency_bias on thread context conditional: the operation accepts the preset only when recency-aware auto-search ships in v3. The current v4 OpenAPI document gives GET /threads/{thread_uuid}/context only the template_uuid query parameter. Thus an absent parameter is correct for the current contract. When the parameter is present, the test requires the off, mild, strong string enum, which is the requirement of section 18.5.