fix(docker): remove build artifacts from image layers

[nightcityblade]

Summary

Fixes #2092.

Keep Playwright installation, runtime-cache copying, and root-cache cleanup in one Docker layer so the superseded root-owned browser cache is not committed to the image. Remove QEMU core dumps immediately after crawl4ai-doctor, before that layer is committed.

List of files changed and why

• Dockerfile - clean duplicate Playwright data and doctor core dumps in the same layers that create them.
• deploy/docker/tests/test_security_container_posture.py - enforce both same-layer cleanup guarantees.

How Has This Been Tested?

• docker buildx build --check --progress plain . — check complete, no warnings found.
• .venv/bin/python -m pytest deploy/docker/tests/test_security_*.py -q — 317 passed, 1 xfailed.
• .venv/bin/black --check --target-version py312 deploy/docker/tests/test_security_container_posture.py — passed.
• .venv/bin/ruff check deploy/docker/tests/test_security_container_posture.py — passed.
• git diff --check — passed.

A full multi-architecture image build was not run locally; the focused posture test verifies that each cleanup remains in the Docker layer that creates its artifact.

Checklist:

• My code follows the style guidelines of this project
• I have performed a self-review of my own code
• I have commented my code, particularly in hard-to-understand areas — N/A; the cleanup commands and focused assertions are self-explanatory.
• I have made corresponding changes to the documentation — N/A; this changes image contents without changing user-facing behavior.
• I have added/updated unit tests that prove my fix is effective or that my feature works
• New and existing unit tests pass locally with my changes

[nightcityblade]

CI follow-up: both Security jobs pass. The only failure is the Discord notification job, whose log reports that DISCORD_WEBHOOK is not defined; this is a repository workflow-secret issue rather than a failure in the Dockerfile change.

[ntohidi]

Thanks for turning this around so fast. The core-dump part is right — moving that cleanup into the crawl4ai-doctor layer is exactly what's needed.

Two problems though, and they're both about layer placement.

The rm -rf /root/.cache/ms-playwright is one layer too late. Docker only gives the space back if you delete in the same layer that made the file, and the browsers don't come from playwright install --with-deps — they come from RUN crawl4ai-setup on the line above, which calls playwright install --force chromium internally. I measured it on develop: that setup layer is 1.26GB. So after this change du inside the container looks clean, but docker images stays about where it was.

The other one is subtler. crawl4ai-doctor runs as root and needs /root/.cache/ms-playwright, which the previous layer just deleted. The build still passes, which is what makes it easy to miss — run_doctor() catches everything and doctor() exits 0 no matter what (crawl4ai/install.py:167-211). So it quietly stops checking anything.

While you're in there, one more: playwright install without naming a browser also pulls Firefox and WebKit. Neither gets copied to appuser, so they just sit in the image — 909MB on arm64.

Putting it all in one layer, with the doctor running before the delete:

RUN crawl4ai-setup \
    && playwright install --with-deps chromium \
    && crawl4ai-doctor \
    && mkdir -p /home/appuser/.cache/ms-playwright \
    && cp -r /root/.cache/ms-playwright/chromium-* \
        /root/.cache/ms-playwright/chromium_headless_shell-* \
        /home/appuser/.cache/ms-playwright/ \
    && chown -R appuser:appuser /home/appuser/.cache/ms-playwright \
    && rm -rf /root/.cache/ms-playwright \
    && find "${APP_HOME}" -maxdepth 1 -type f -name '*.core' -delete

I built that on arm64 and got 9.03GB → 6.46GB. Checked the result too: root cache gone, no Firefox or WebKit, doctor passes during the build, and crawls still work as appuser.

Last thing — the new test asserts the cleanup lives in the playwright install layer, which would lock in the wrong spot. Asserting that the install, the copy, the doctor and both cleanups all share one RUN would catch the real thing.

[nightcityblade]

Thanks for the detailed layer analysis — addressed in 14291fa.

• crawl4ai-setup, Chromium-only dependency installation, the doctor, runtime-cache copy, and both cleanups now run in one RUN.
• crawl4ai-doctor runs before the root Playwright cache is deleted.
• The posture test now requires all of those steps to share the setup layer and preserves their required order.

Validation:

• python -m pytest deploy/docker/tests/test_security_*.py -q — 317 passed, 1 xfailed
• python -m black --check --target-version py312 deploy/docker/tests/test_security_container_posture.py — passed
• python -m ruff check deploy/docker/tests/test_security_container_posture.py — passed
• git diff --check — passed

I could not rerun the full image build on this machine because its Docker daemon is unavailable.