Skip to content

Trim the Java runtime with jlink, on a shared base image - #2540

Open
francoisferrand wants to merge 1 commit into
development/2.16from
improvement/ZENKO-5361-jlink
Open

francoisferrand wants to merge 1 commit into
development/2.16from
improvement/ZENKO-5361-jlink

Conversation

@francoisferrand

@francoisferrand francoisferrand commented Sep 17, 2026

Copy link
Copy Markdown
Contributor

Replaces the full Temurin JRE with a jlink-trimmed runtime, carried in a shared java image that kafka, kafka-connect, cruise-control and zookeeper all build FROM.

Follow-up to #2539 (ZENKO-5188), now merged.

Size

Measured deduplicated — what a registry or the ISO actually stores for the three service images:

total
before (Temurin JRE) 614.6 MB
this PR 492.5 MB — −122.1 MB (−20%)

The shared base is what makes this worthwhile. Inlining the same stages in each Dockerfile saved only 21.5 MB (3.5%): the jlink output deduplicates because it is a COPY from an identical stage, but each image's apt layer is built separately and does not, so three copies of the OS upgrade ate the saving. Moving both into one base image they genuinely share recovers it.

It also reduces Dockerfile complexity rather than adding it — each service image drops its jlink stage and its apt block, and the JDK/base decisions now live in one file instead of three.

CVEs

Scanned with trivy. Java/application CVEs are unchanged (34 vs 34) — jlink does not touch app dependencies.

fixable (a patch exists — the actionable number) CRIT HIGH MED LOW total
before 4 72 81 10 167
after 4 72 72 10 158

−9 medium, no change to critical or high. Marginal, and not the reason to do this.

⚠️ Scanner dashboards will show a large jump in unfixable findings (47 → 226 per image). That is a Debian-vs-Ubuntu reporting-policy difference, not new exposure: the flagged packages are present in both images and Debian's versions are newer (bsdutils 2.41.5 vs 2.39.3, libtinfo6 6.5 vs 6.4). Debian's tracker reports what Ubuntu's marks not-affected.

The apt-get upgrade in the base is load-bearing. Pinning our own base transfers patching responsibility from Temurin (rebuilt continuously) to us; without it the images scanned 3 critical and 10 high worse than what they replace, because debian:trixie-slim was three weeks stale.

Class data sharing

jlink does not generate a CDS archive, and the stock Temurin images ship one. Measured JVM startup, best of 5:

runtime startup
stock Temurin 13 ms
jlink, no CDS 32 ms
jlink + -Xshare:dump 13 ms

Without it every JVM start re-parses class metadata that CDS would otherwise map straight into memory. The base therefore runs java -Xshare:dump, as the pre-jlink Dockerfile did. This costs 13.8 MB in the shared layer — the difference between the 479.3 MB an archive-less build reaches and the 492.5 MB above — and is why the saving is 20% rather than 22%.

Without this, two e2e scenarios timed out (Deletion of an archived object, PRA (nominal case)); both pass with it.

Notes for review

  • A fourth published image. ghcr.io/scality/zenko/java is built and pushed but never deployed, so it is marked buildOnly: true in deps.yaml and excluded from the ISO image list. The three service builds now needs: java, so CI wall-clock grows by one build.
  • Tree hashes. solution/java feeds the build tree hash of every image built on it. Without that, changing the base would republish different content under an already-published immutable tag.
  • ALL-MODULE-PATH rather than a jdeps-derived module list: Netty, the JMX agent, SASL and TLS resolve modules via reflection and ServiceLoader, which jdeps cannot see, so trimming saves ~21 MB in exchange for failures that only appear at runtime. Trivy does not detect the jlink runtime as a package at all, so trimming would not change CVE counts either. Dropping the dev-tooling modules to remove javac/jshell was investigated and rejected separately: it is all-or-nothing (jdk.compiler is pulled back by jdk.javadoc, jdk.jdeps and jdk.jshell), and the cluster includes jdk.jdwp.agent and jdk.hotspot.agent, so it would cost jstack -F/jmap -F on a hung JVM.
  • --release-info is required: jlink otherwise drops all vendor metadata, including the IMPLEMENTOR="Eclipse Adoptium" marker that the ZENKO-5188 work relies on to prove we are not shipping Oracle JDK.
  • Moving off Temurin's Ubuntu base frees uid 1000, so zookeeper no longer deletes the default ubuntu user to claim it.
  • dependencies_versions_env() now requires an envsubst key before emitting a variable. java has none, and without the guard it emitted nameless =java / =21.0.12_8 lines into the generated env.
  • The build job in build-kafka.yaml is renamed kafka, now that the workflow has four jobs.

Verified

All four images: IMPLEMENTOR="Eclipse Adoptium", 69 GPL legal notices, no NFTC text, and java -Xshare:on -version succeeds.

Beyond building — a real Kafka 3.9 KRaft broker starts and completes a produce/consume roundtrip; kafka-connect's jmx_prometheus javaagent loads (exercising java.instrument); ZooKeeper reaches Mode: standalone at uid 1000 with socat/hostname/ps/gosu present for the operator probes; CruiseControl runs clean with no classpath errors.

CI is green: the four build jobs, build-iso, and all four e2e suites including ctst-end2end-sharded (4474 scenarios).

Issue: ZENKO-5361

@francoisferrand
francoisferrand changed the base branch from improvement/ZENKO-5188-adobe-fork to improvement/ZENKO-5188-vendor-images September 17, 2026 21:03
Comment thread solution/cruise-control/Dockerfile Outdated
Comment thread solution/kafka/Dockerfile Outdated
@francoisferrand

Copy link
Copy Markdown
Contributor Author

CI note: the build-kafka / zookeeper job failed on the first run with

Could not GET 'https://repo.maven.apache.org/maven2/org/apache/logging/log4j/log4j-core/2.20.0/log4j-core-2.20.0.pom'.
Received status code 403 from server: Forbidden

That is Maven Central refusing the runner, not a problem with this change — build-kafka / cruise-control runs the same gradle-against-Maven-Central pattern in the same workflow and passed on that run. Re-ran the failed job and all three now pass.

Verified the published image afterwards: JAVA_HOME=/opt/java/openjdk, the jlink runtime is copied in, and there is no download.oracle.com anywhere in the build history.

Worth knowing for whoever picks this up: the zu gradle build makes this job dependent on Maven Central availability at build time, so the occasional 403 will need a retry. That is inherited from the ZENKO-5188 vendoring, not introduced here.

@francoisferrand
francoisferrand force-pushed the improvement/ZENKO-5188-vendor-images branch 3 times, most recently from 75577fc to 4f39a4f Compare September 18, 2026 09:31
Base automatically changed from improvement/ZENKO-5188-vendor-images to development/2.16 September 18, 2026 13:59
@bert-e

bert-e commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Hello francoisferrand,

My role is to assist you with the merge of this
pull request. Please type @bert-e help to get information
on this process, or consult the user documentation.

Available options
name description privileged authored
/after_pull_request Wait for the given pull request id to be merged before continuing with the current one.
/bypass_author_approval Bypass the pull request author's approval
/bypass_build_status Bypass the build and test status
/bypass_commit_size Bypass the check on the size of the changeset TBA
/bypass_incompatible_branch Bypass the check on the source branch prefix
/bypass_jira_check Bypass the Jira issue check
/bypass_peer_approval Bypass the pull request peers' approval
/bypass_leader_approval Bypass the pull request leaders' approval
/bypass_source_branch_lineage Bypass the cross-branch contamination check
/approve Instruct Bert-E that the author has approved the pull request. ✍️
/create_pull_requests Allow the creation of integration pull requests.
/create_integration_branches Allow the creation of integration branches.
/no_octopus Prevent Wall-E from doing any octopus merge and use multiple consecutive merge instead
/unanimity Change review acceptance criteria from one reviewer at least to all reviewers
/wait Instruct Bert-E not to run until further notice.
Available commands
name description privileged
/help Print Bert-E's manual in the pull request.
/status Print Bert-E's current status in the pull request.
/clear Remove all comments from Bert-E from the history TBA
/retry Re-start a fresh build TBA
/build Re-start a fresh build TBA
/force_reset Delete integration branches & pull requests, and restart merge process from the beginning.
/reset Try to remove integration branches unless there are commits on them which do not appear on the source branch.

Status report is not available.

@bert-e

bert-e commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

Incorrect fix version

The Fix Version/s in issue ZENKO-5361 contains:

  • None

Considering where you are trying to merge, I ignored possible hotfix versions and I expected to find:

  • 2.16.0

Please check the Fix Version/s of ZENKO-5361, or the target
branch of this pull request.

@francoisferrand
francoisferrand force-pushed the improvement/ZENKO-5361-jlink branch from 4a603c2 to f426b11 Compare September 18, 2026 14:00
@francoisferrand francoisferrand changed the title Trim the Java runtime with jlink Trim the Java runtime with jlink, on a shared base image Sep 18, 2026
Comment thread solution/java/Dockerfile Outdated
Comment thread solution/deps.yaml Outdated
@francoisferrand
francoisferrand force-pushed the improvement/ZENKO-5361-jlink branch 5 times, most recently from b78a950 to 8413af5 Compare September 21, 2026 08:17
The images ship a full Temurin JRE, most of which these services never
load. Build a jlink runtime instead and put it on a Debian base.

Both the runtime and the patched OS live in a shared java-base image
rather than being repeated per image. Inlined, only the jlink output
deduplicates and each apt layer is written separately, which cuts the
saving across the three images from 135MB to 21MB.

ALL-MODULE-PATH keeps every platform module rather than a jdeps-derived
list. Netty, the JMX agent, SASL and TLS reach modules through reflection
and ServiceLoader, so trimming would trade ~25MB for failures that only
appear at runtime, on paths tests may never exercise.

Pinning our own base means nothing else patches it, so it upgrades its
packages at build time; without that the images inherit whatever was
current when Debian last rebuilt the base, which scanned three critical
and ten high CVEs worse than the images they replace.

The base feeds the build tree hash of every image built on it, so bumping
it republishes them all rather than silently changing the content behind
an already published tag.

Moving off Temurin's Ubuntu images frees uid 1000, so zookeeper no longer
needs to delete the default ubuntu user to claim it.

Issue: ZENKO-5361
@francoisferrand
francoisferrand force-pushed the improvement/ZENKO-5361-jlink branch from 8413af5 to 0868a8f Compare September 21, 2026 08:33

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants