Skip to content

fix(s3): 🐛 accept a refused write when the object is the call's own - #370

Merged
justapithecus merged 2 commits into
mainfrom
andrew/fix/s3/idempotent-put
Nov 8, 2026
Merged

justapithecus merged 2 commits into
mainfrom
andrew/fix/s3/idempotent-put

Conversation

@justapithecus

@justapithecus justapithecus commented Nov 8, 2026 •

Copy link
Copy Markdown
Member

Summary

A Quarry run against R2 stalled for ninety minutes on one write and then failed with path already exists. The SDK had retried a PutObject whose response the network lost, and the retry met the object the first attempt had stored. The S3 adapter read the refusal as another writer's object, so a Put that had succeeded returned ErrPathExists, and the commit on top of it failed. The adapter now recognises its own upload.

Highlights

  • The contract states the exception. CONTRACT_STORAGE.md § Put: where the backend reports a content digest the adapter can compare with its payload's, a refused conditional create is a success when the stored object's length and digest equal the payload's. When either differs, or the comparison fails, Put returns ErrPathExists as before.
  • The S3 adapter compares length and ETag. Put hashes the payload as it spools it. On a precondition failure the atomic path reads the object's HEAD and returns nil when its length and ETag equal the payload's length and MD5, which an S3 store gives a single-part object. A multipart ETag carries a part count and never matches. A HEAD that fails leaves the refusal in place.
  • The mock gives S3's ETag. The test client's ETag was a SHA-256. It is now the quoted MD5 that S3 gives, and HeadObject reports length and ETag as S3 does.
  • Tests. A client that stores the object and then refuses, as a backend does after a lost response, makes Put succeed with the bytes in place. Another writer's object, a HEAD that fails, and a multipart ETag each keep ErrPathExists. The existing duplicate-write test writes other bytes for the refusal and the same bytes for the success.

Test Plan

  • task lint and task test pass, with the four new cases and the adjusted duplicate-write case.
  • The prose gate on the contract text, the changelog entry, the new comments and this body.
  • Codex round 1: no findings.
  • Bot review of fd46186a: no findings.
  • Bosun's changelog check failed on the first head, since the entry ran to 103 words against the limit of 90. Commit d142e63d shortens it to 81.

🤖 Generated with Claude Code

The SDK retries a PutObject whose response the network lost, and the
retry meets the object the first attempt stored. The backend refuses the
conditional write, and the adapter returned ErrPathExists for a Put that
had succeeded, so the commit on top of it failed. A Quarry run against R2 met this after a
long stall and lost ninety minutes of work.

The adapter now answers a refusal by reading the object's length and
ETag, which an S3 store gives a single-part object as the MD5 of its
bytes. When both equal the payload's, Put returns success. A multipart
ETag never matches, and a refusal the HEAD cannot confirm stays
ErrPathExists. The storage contract states the rule, and the mock gives
the ETag S3 gives.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-08T15:25:32.705339Z fd46186 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@justapithecus justapithecus changed the title andrew/fix/s3/idempotent put fix(s3): 🐛 accept a refused write when the object is the call's own Nov 8, 2026
Bosun's changelog rule C6 holds an entry to ninety words, and the entry
ran to a hundred and three. The shorter entry says the same.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@justapithecus
justapithecus merged commit 8f60064 into main Nov 8, 2026
12 checks passed
@justapithecus
justapithecus deleted the andrew/fix/s3/idempotent-put branch November 8, 2026 15:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant