Skip to content

fix: keep transient idempotency responses retryable - #1537

Draft
stmr wants to merge 10 commits into
CalloraOrg:mainfrom
stmr:fix/assigned-1499-idempotency-cache-policy
Draft

stmr wants to merge 10 commits into
CalloraOrg:mainfrom
stmr:fix/assigned-1499-idempotency-cache-policy

Conversation

@stmr

@stmr stmr commented Oct 20, 2026 •

Copy link
Copy Markdown

Scope

Closes #1499. Integrated accepted upstream main 5b7a197214b976dfa0cf90d014220962a4e378bc without force-pushing. No main ref changed in this update.

  • Preserve accepted lease fencing, transient-response retries, deterministic replay and safe response headers.
  • Record refund execution receipts atomically with the SQLite credit mutation, independently of the bounded HTTP response cache. Same-key retries after an oversized response no longer grant twice.
  • Verify quote-bound app-token payment and consume quote/payment atomically before returning a fee-bump. Both Horizon fetches observe request cancellation. Cancellation observed before signing rolls the transaction back; the same quote/payment can retry. Verification budgets must be below an enabled billing deadline. No atomic HTTP-delivery guarantee after COMMIT is claimed.
  • Preserve payer isolation and fail closed on incoherent recorded SQLite migration history. Paired additions are PostgreSQL0033/0034 and SQLite0035; populated quote/receipt rollback refuses to erase single-use evidence.

API contract: fee execution requires quoteId, unchanged inner XDR and a64-hex payment transaction hash carrying the quote MEMO_HASH. See docs/fee-abstraction.md.

Published source and exercised proof

Head 51b33d72eb6d1913da120ade0c951ebf7e8cd987; tree c537ae3c05f40e4c2845ca827d51f2c5f9ff0e21.

  • Local actual Node20: 202 suites /2,970 tests /2 snapshots passed, existing skipped suite/two skipped tests unchanged; full types/lint/build/error catalog/OpenAPI validation/migration policy passed.
  • Actual PostgreSQL17.11: 8/8 multi-backend race/rollback/cancellation scenarios passed. Runnable: tests/integration/feeAbstraction.postgres.mjs. Its signer callback is synthetic.
  • Separate real HTTP/native-PG/file-SQLite refund proof: cap128,response261,two same-key requests,balance5.00,one durable receipt after cache-lease deletion; changed-body retry409 without another credit.
  • Actual HTTP/SDK deadline proof: a row-lock wait crosses500ms; HTTP504 is followed by backendROLLBACK and pending quote/payment. Retrying the same quote/payment returns one cryptographically valid envelope. Both real Horizon fetch phases cancel on request abort.
  • Full SQLite migration initialization, populated pre0012 upgrade preservation, missing0035-history rejection and empty/nonempty receipt rollback passed. Current Vercel demo-bundle HTTP smoke passed.
  • SDK chain transport is controlled loopback fixture data, not live-chain inclusion, real funds or payout.

Current exact-tree author CI — NOT all green

Run40230593334 uses the approved CI-only develop baseline at accepted upstream5b7a197. The author-fork main remains frozen at22fd831. The unchanged workflow already supports develop; no checker rule, skip or permission was changed.

CI synthetic merge 542c4e188d234a6854609104fbd99af2ff4c2f91 has tree c537ae3c05f40e4c2845ca827d51f2c5f9ff0e21, exactly the reviewed source tree.

  • build-and-test: SUCCESS (full lint/types/build/tests and Vercel smoke).
  • migration-policy: SUCCESS.
  • openapi-compat: FAILURE,537/626 checks pass. The89 remaining reports all concern adding the same IDEMPOTENCY_KEY_REUSE_MISMATCH response-enum option. Incoming-main DEMO/billing-schema differences from the older22fd831 baseline are no longer the issue.

The author CI PR is technical validation only, OPEN/DRAFT and must not be merged.

Maintainer catalog decision needed

Accepted main already emits IDEMPOTENCY_KEY_REUSE_MISMATCH directly from idempotency middleware, and its tests/docs rely on that value, but omitted it from the authoritative generated catalog. This PR catalogs that existing code so the new durable-receipt mismatch path remains consistent after HTTP-cache eviction. I have kept the documented code rather than introducing a receipt-only generic CONFLICT split.

Please confirm acceptance of cataloging this already-emitted code and the project's proper canonical catalog/compatibility process needed to clear the gate. No ignore rules, skipped checks or fabricated approval have been added; the compatibility failure is intentionally reported, not waived.

Review / deployment gate

Keeping this PR draft while that contract gate is unresolved. Accepted numbering does not prove the real deployed PostgreSQL filenames, versions, checksums or financial-backfill execution history. The existing deployment-history question remains unanswered. No aliases, historical ledger rewrites, blind replay, deployment-readiness, merge approval, points or payment are claimed.

Cache deterministic outcomes with safe response headers and a bounded
UTF-8 body. Keep reconciliation responses retryable.

Repair full lint/type contracts and preserve SQLite quota rollback.
Published 0026 collisions remain blocked on deployment history; do not
rename applied artifacts or bypass migration policy.

Refs CalloraOrg#1499
@drips-wave

drips-wave Bot commented Oct 20, 2026

Copy link
Copy Markdown

@stmr Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

Keep the cache retry transition regression aligned with the current
public ErrorEnvelope contract. Preserve all retry, replay and
deduction-count assertions.
@stmr

stmr commented Oct 20, 2026

Copy link
Copy Markdown
Author

Fixed the reachable author-side full-suite regression in eb3fd3f: the test now reads the current documented error.code envelope, preserving the same-key reconciliation→success→cached-replay and exactly-two-deductions assertions.

Full original Linux workflow on the exact source/tree: build-and-test passed (full lint/type/build,175 suites/2629 tests/1 snapshot,2 existing skipped tests, Vercel bundle), and OpenAPI compatibility passed. migration-policy still fails only on the existing duplicate0026 prefixes (twice) and missing payout rollback. Overall CI is therefore not all green.

The only remaining repair prerequisite is the published-migration deployment-history clarification, needed to avoid replaying a financial backfill or invalidating applied artifacts. No CI-setting change or weakened check is requested. Keeping this draft until the data-safe migration resolution is possible. No merge, deployment, points or payment claimed.

stmr and others added 5 commits October 21, 2026 14:52
Merge accepted upstream main while preserving lease ownership, fencing,
renewal and response completion. Apply transient-response cache policy,
bounded encoded-body inspection and safe header replay.

Enforce billing payer ownership and duplicate-request isolation, restore
canonical error envelopes, repair source contracts and stable history
ETags, and restore seven repaired suites to default CI.

Keep every accepted historical SQL file byte-identical. Only the
unapplied response-header migration moves to the unused 0028 prefix.
Historical migration-policy violations remain explicit and unresolved.

Refs CalloraOrg#1499
Merge accepted upstream main while preserving lease ownership, fencing,
renewal and response completion. Apply transient-response cache policy,
bounded encoded-body inspection and safe header replay.

Enforce billing payer ownership and duplicate-request isolation, restore
canonical error envelopes, repair source contracts and stable history
ETags, and restore seven repaired suites to default CI.

Keep every accepted historical SQL file byte-identical. Only the
unapplied response-header migration moves to the unused 0028 prefix.
Historical migration-policy violations remain explicit and unresolved.

Refs CalloraOrg#1499
Preserve accepted lease fencing while integrating current main.
Lock quotes before expiry checks; verify on-chain payment and match
the actual signed envelope fee before committing consumption.

Keep published migration history unchanged. Migration policy remains
blocked pending deployed-history confirmation.

BREAKING CHANGE: fee execution requires quoteId and a quote-bound memo.
Refs CalloraOrg#1499
The published API-created head has the same complete tree as an
already integrated local ancestor. Join its ancestry without changing
the verified integration tree; no published changes are discarded.
@stmr

stmr commented Oct 21, 2026

Copy link
Copy Markdown
Author

Integrated accepted main 22fd831 and pushed 5da7305c51371d35767218b8680f4abcf26afa77 without force; verified tree ee43bd3754d5acb556586133ebd77dfbac57b908.

Exact-tree Linux run: build-and-test passed (lint, full types, 191 suites / 2,844 tests, Vercel HTTP smoke); OpenAPI compatibility passed. Two pre-existing skips remain. Overall CI is not green: migration-policy still reports ten published-history/layout violations. No historical SQL was edited or renumbered; the existing deployed-history question remains the prerequisite for a data-safe repair.

Payment signing now verifies quote-bound on-chain payment and consumes it atomically. Native PostgreSQL reproduced an expiry-after-lock-wait bug; explicit row locking fixes it. Four deterministic multi-backend races passed, plus real compiled HTTP/SDK signing with two blocked native backends and exactly one cryptographically verified envelope. Synthetic chain transport only; no real funds. The API requires quoteId and the quote MEMO_HASH, documented in docs/fee-abstraction.md.

The temporary author-side validation PR was closed unmerged. Keeping this PR draft; not claiming deployment readiness, points or payment.

@greatest0fallt1me

Copy link
Copy Markdown
Contributor

This PR is still marked as a draft, so it can't be reviewed or merged yet. When it's ready, please rebase onto the latest default branch, make sure CI passes, and click Ready for review. Note: PRs that change .vscode/, .devcontainer/, dist/, api.js, font or binary files, or package lifecycle scripts are rejected by our security scan. It also currently conflicts with main (several migrations were renumbered to 0027–0032 in #1550), so please rebase first.

stmr added 3 commits October 24, 2026 21:51
Preserve atomic fee payment verification and idempotency behavior.
Accepted main layout changes do not establish deployed filename or
checksum history; deployment remains gated on maintainer confirmation.
Record each refund grant atomically with its balance change.
Ledger SQLite migrations without rewriting existing records.

Integrate accepted main without replaying published backfills.
Keep populated quote and receipt rollbacks fail-closed.

Refs CalloraOrg#1499
Propagate billing cancellation through payment verification and quote consumption. Check after database waits before signing so cancelled requests roll back and can retry with the same payment. Keep verification budgets below enabled billing deadlines.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Stop idempotencyMiddleware from caching transient 4xx responses such as 429, 408 and 409 for the full retention window

2 participants