Skip to content

fix: 大 PDF 分批拆页写盘,拆页崩溃不再无限循环 (#88) - #94

Merged
haywolf merged 2 commits into
masterfrom
fix/88-split-memory
Nov 5, 2026
Merged

haywolf merged 2 commits into
masterfrom
fix/88-split-memory

Conversation

@haywolf

@haywolf haywolf commented Nov 4, 2026

Copy link
Copy Markdown
Contributor

Fixes #88

改动

  • PDFSplitter 改为分批渲染(每批 10 页)并写盘,循环外只保留页面元数据;重试只重试单批
  • 不带页码范围时,连续往后取批,直到 pdf.js 返回的页数不足一批,避免页面树缺 /Type 的 PDF 被 pdf-lib 少算页数而丢页
  • WorkerOrchestrator 新增 recoverInterruptedSplits:每次拆页中断都计数(Task.split_interrupt_count,新迁移),累计 3 次就标记为 FAILED 并清理残留图片,不再无限循环;task:retry 会把计数清零

测试

  • 补充 PDFSplitter 分批测试(批次切分、页序、跨批范围、最多只持有一批、单批重试、页数算少或算多)、真实库集成测试、WorkerOrchestrator 计数测试、task.handler 测试
  • npm run test:unit:49 个文件,1051 个用例全部通过;npm run test:renderer:283 个用例通过
  • npm run typecheck、eslint:通过
  • 内存实测(Node 下):600 页扫描式 PDF 峰值从 3729MB 降到 1037MB;同样文件大小时,300 页和 1200 页峰值分别是 514MB 和 568MB。新旧拆页结果的摘要一致
  • 崩溃循环实测:真实 SQLite,拆页中途被 SIGKILL,第 4 次启动时任务转为 FAILED

已知限制

  • 峰值内存仍会随 PDF 文件大小增长(大约降到原来的 1/5),所以验收标准第 1 条只算部分达成
  • 拆页中途正常退出应用也会计数

未验证项

  • 没有在真实 Electron 或打包版里测,没有用真实扫描件,也没有在 Windows、macOS 上测

pdfToPng holds every page it renders until it returns, so a large PDF kept
all its PNGs in the main process. Pages are now rendered and written in
batches of WORKER_CONFIG.splitter.renderBatchSize, and only their metadata
is kept. Without a page range, pdf.js still decides where the document ends,
because pdf-lib misses page tree nodes that lack a /Type.

A split that brought the app down was re-queued on every start. Interrupted
splits are now counted in Task.split_interrupt_count, and a task fails once
it reaches maxInterruptedSplits. Retrying the task resets the count.
Conflicts in PDFSplitter.ts and its unit test. Batched rendering now goes
through renderPdfPages (#92), which also decides the pdf.js log level, so the
splitter's verbosityLevel: 0 is gone. PDFPageRenderer needed no change:
pagesToProcess already passes through.

strictPagesToProcess, which master still passed, is left out: PdfToPngOptions
of the pinned converter has no such option and nothing reads it.
@haywolf
haywolf merged commit fbdf636 into master Nov 5, 2026
2 checks passed
@haywolf
haywolf deleted the fix/88-split-memory branch November 5, 2026 03:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

大 PDF 拆页时全部 PNG 堆在主进程内存,可能 OOM 并在重启后循环崩溃

1 participant