Trim the Java runtime with jlink, on a shared base image - #2540
francoisferrand wants to merge 1 commit into
Conversation
|
CI note: the That is Maven Central refusing the runner, not a problem with this change — Verified the published image afterwards: Worth knowing for whoever picks this up: the |
75577fc to
4f39a4f
Compare
Hello francoisferrand,My role is to assist you with the merge of this Available options
Available commands
Status report is not available. |
Incorrect fix versionThe
Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:
Please check the |
4a603c2 to
f426b11
Compare
b78a950 to
8413af5
Compare
The images ship a full Temurin JRE, most of which these services never load. Build a jlink runtime instead and put it on a Debian base. Both the runtime and the patched OS live in a shared java-base image rather than being repeated per image. Inlined, only the jlink output deduplicates and each apt layer is written separately, which cuts the saving across the three images from 135MB to 21MB. ALL-MODULE-PATH keeps every platform module rather than a jdeps-derived list. Netty, the JMX agent, SASL and TLS reach modules through reflection and ServiceLoader, so trimming would trade ~25MB for failures that only appear at runtime, on paths tests may never exercise. Pinning our own base means nothing else patches it, so it upgrades its packages at build time; without that the images inherit whatever was current when Debian last rebuilt the base, which scanned three critical and ten high CVEs worse than the images they replace. The base feeds the build tree hash of every image built on it, so bumping it republishes them all rather than silently changing the content behind an already published tag. Moving off Temurin's Ubuntu images frees uid 1000, so zookeeper no longer needs to delete the default ubuntu user to claim it. Issue: ZENKO-5361
8413af5 to
0868a8f
Compare
Replaces the full Temurin JRE with a
jlink-trimmed runtime, carried in a sharedjavaimage that kafka, kafka-connect, cruise-control and zookeeper all buildFROM.Follow-up to #2539 (ZENKO-5188), now merged.
Size
Measured deduplicated — what a registry or the ISO actually stores for the three service images:
The shared base is what makes this worthwhile. Inlining the same stages in each Dockerfile saved only 21.5 MB (3.5%): the
jlinkoutput deduplicates because it is aCOPYfrom an identical stage, but each image'saptlayer is built separately and does not, so three copies of the OS upgrade ate the saving. Moving both into one base image they genuinely share recovers it.It also reduces Dockerfile complexity rather than adding it — each service image drops its
jlinkstage and itsaptblock, and the JDK/base decisions now live in one file instead of three.CVEs
Scanned with trivy. Java/application CVEs are unchanged (34 vs 34) — jlink does not touch app dependencies.
−9 medium, no change to critical or high. Marginal, and not the reason to do this.
bsdutils2.41.5 vs 2.39.3,libtinfo66.5 vs 6.4). Debian's tracker reports what Ubuntu's marks not-affected.The
apt-get upgradein the base is load-bearing. Pinning our own base transfers patching responsibility from Temurin (rebuilt continuously) to us; without it the images scanned 3 critical and 10 high worse than what they replace, becausedebian:trixie-slimwas three weeks stale.Class data sharing
jlink does not generate a CDS archive, and the stock Temurin images ship one. Measured JVM startup, best of 5:
-Xshare:dumpWithout it every JVM start re-parses class metadata that CDS would otherwise map straight into memory. The base therefore runs
java -Xshare:dump, as the pre-jlink Dockerfile did. This costs 13.8 MB in the shared layer — the difference between the 479.3 MB an archive-less build reaches and the 492.5 MB above — and is why the saving is 20% rather than 22%.Without this, two e2e scenarios timed out (
Deletion of an archived object,PRA (nominal case)); both pass with it.Notes for review
ghcr.io/scality/zenko/javais built and pushed but never deployed, so it is markedbuildOnly: trueindeps.yamland excluded from the ISO image list. The three service builds nowneeds: java, so CI wall-clock grows by one build.solution/javafeeds the build tree hash of every image built on it. Without that, changing the base would republish different content under an already-published immutable tag.ALL-MODULE-PATHrather than ajdeps-derived module list: Netty, the JMX agent, SASL and TLS resolve modules via reflection and ServiceLoader, which jdeps cannot see, so trimming saves ~21 MB in exchange for failures that only appear at runtime. Trivy does not detect the jlink runtime as a package at all, so trimming would not change CVE counts either. Dropping the dev-tooling modules to removejavac/jshellwas investigated and rejected separately: it is all-or-nothing (jdk.compileris pulled back byjdk.javadoc,jdk.jdepsandjdk.jshell), and the cluster includesjdk.jdwp.agentandjdk.hotspot.agent, so it would costjstack -F/jmap -Fon a hung JVM.--release-infois required: jlink otherwise drops all vendor metadata, including theIMPLEMENTOR="Eclipse Adoptium"marker that the ZENKO-5188 work relies on to prove we are not shipping Oracle JDK.ubuntuuser to claim it.dependencies_versions_env()now requires anenvsubstkey before emitting a variable.javahas none, and without the guard it emitted nameless=java/=21.0.12_8lines into the generated env.buildjob inbuild-kafka.yamlis renamedkafka, now that the workflow has four jobs.Verified
All four images:
IMPLEMENTOR="Eclipse Adoptium", 69 GPL legal notices, no NFTC text, andjava -Xshare:on -versionsucceeds.Beyond building — a real Kafka 3.9 KRaft broker starts and completes a produce/consume roundtrip; kafka-connect's jmx_prometheus javaagent loads (exercising
java.instrument); ZooKeeper reachesMode: standaloneat uid 1000 withsocat/hostname/ps/gosupresent for the operator probes; CruiseControl runs clean with no classpath errors.CI is green: the four build jobs,
build-iso, and all four e2e suites includingctst-end2end-sharded(4474 scenarios).Issue: ZENKO-5361