Skip to content

fix: keep PyMuPDF asset probe from hanging or dropping results - #323

Merged
suguanYang merged 3 commits into
mainfrom
fix/wangbinqi/pymupdf-probe-assets
Aug 24, 2026
Merged

fix: keep PyMuPDF asset probe from hanging or dropping results#323
suguanYang merged 3 commits into
mainfrom
fix/wangbinqi/pymupdf-probe-assets

Conversation

@suguanYang

Copy link
Copy Markdown
Contributor

Summary

  • Skip find_tables and drawing clustering on image/vector-dense pages so _probe_assets_worker stays under the 300s child timeout (production lipoprotein PDF page 2 has ~114k drawings).
  • Flush the multiprocessing result queue in the child after put(), and retry once on exitcode=0 with an empty queue so a healthy probe is not reported as SUBPROCESS_CRASH.

Test plan

  • uv run pytest tests/unit/test_probe_dense_visual_assets.py tests/unit/test_pymupdf_subprocess_queue.py tests/unit/test_probe_visible_text.py
  • Local production path: lipoprotein PDF probe finished in 21.8s (previously timed out at 300s)
  • Local production path: nucleic PDF probe succeeded 3/3
  • Staging re-parse of job_6265564f67e8 and job_83eb3d464064 after merge

Made with Cursor

Dense vector pages skip table detection and drawing clustering so probe stays under the 300s child timeout, and workers flush the result queue before exit so an empty-queue crash is retried instead of failing the job.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment thread apps/worker/tests/unit/test_pymupdf_subprocess_queue.py Fixed
suguanYang and others added 2 commits August 24, 2026 22:20
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
…ith 'import' and 'import from''

Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
@suguanYang
suguanYang merged commit 10658cb into main Aug 24, 2026
6 of 7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants