Skip to content

fix: upload local MinerU shards when HeadObject returns 403 - #327

Merged
suguanYang merged 2 commits into
mainfrom
fix/wangbinqi/mineru-shard-s3-head-forbidden
Aug 25, 2026
Merged

fix: upload local MinerU shards when HeadObject returns 403#327
suguanYang merged 2 commits into
mainfrom
fix/wangbinqi/mineru-shard-s3-head-forbidden

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

  • Fresh tmp/mineru-shards/... keys are not in S3 yet. Staging HeadObject returned 403 (not 404), which aborted URL mode and forced a slow direct upload to MinerU.
  • Treat 403/Forbidden on the existence probe as “not verified”, then upload the local shard and continue S3 URL mode. Other verify errors still fall back to direct upload.
  • Reuse of existing uploads/ objects is unchanged.

Test plan

  • uv run pytest apps/worker/tests/unit/test_mineru_pdf_service.py
  • After deploy, re-run Exercise_Prescription_in_Cardiac_Rehabilitation.pdf on staging and confirm Logfire shows url_mode_source_unverified then url_mode_source_uploaded / Using S3 URL mode, not an immediate direct-upload fallback from verify_source_object

Made with Cursor

suguanYang and others added 2 commits August 25, 2026 10:38
Temp shard keys are not in S3 yet. A forbidden HeadObject must not abort URL mode; upload the local shard and continue with a presigned URL.

Co-authored-by: Cursor <cursoragent@cursor.com>
HeadObject 403 happened because split shards were never uploaded. Put them in S3 and confirm they exist after split, then start MinerU, instead of treating 403 as a later upload fallback.

Co-authored-by: Cursor <cursoragent@cursor.com>
@suguanYang
suguanYang merged commit 9e00edd into main Aug 25, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant