Skip to content

ci: log the URL and status when engine image downloads fail - #4612

Merged
bpamiri merged 1 commit into
developfrom
peter/engine-wget-nv
Oct 16, 2026
Merged

bpamiri merged 1 commit into
developfrom
peter/engine-wget-nv

Conversation

@bpamiri

@bpamiri bpamiri commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator

Summary

The Lucee 5, 6 and 7 test images download two Lucee extensions from ext.lucee.org and the SQLite JDBC jar from Maven Central with wget -q. When one of those failed, the build log showed only wget's exit code 8 ("server issued an error response"), so it couldn't tell which download failed or what status came back.

Logging: -q becomes -nv -S. Each download now logs its URL and the response status and headers on every attempt.

  • -nv alone isn't enough: wget 1.21 prints no status for an attempt it's going to retry, only the URL. -S prints HTTP/1.1 <status> for every attempt.
  • The headers also show CloudFront's x-cache / x-amz-cf-pop, i.e. which edge answered.

Retries: 403 added to --retry-on-http-error; 404 not added.

  • Why the old list didn't retry this failure. In the failing builds, each of the three attempts at that RUN step failed about 0.7 s after it started. The retried statuses (429/5xx) would have backed off for several seconds, so the server answered with a status outside that list and wget gave up at once.
  • Why 403. ext.lucee.org is S3 behind CloudFront (server: AmazonS3, via: ... (CloudFront)). It answers 403, not 404, for a missing object (x-cache: Error from cloudfront), and CloudFront edge errors can also surface as 403. The URLs here are pinned and exist, so a 403 from them is transient, and retrying it costs about 10 s on a real miss.
  • Why not 404. Only Maven Central returns 404, and there it means a real miss that retrying won't fix.

The SQLite JDBC SHA-256 check from #4598 is unchanged. The SQL Server image's wget already logs (no -q) and is untouched.

Verification

1. Built tools/docker/lucee7 locally with --no-cache; build exit 0. The download step now logs:

  HTTP/1.1 200 OK
2026-10-16 07:28:20 URL:https://ext.lucee.org/org.lucee.h2-2.1.214.0001L.lex [2440188/2440188] -> "/usr/local/lib/serverHome/WEB-INF/lucee-server/deploy/org.lucee.h2-2.1.214.0001L.lex" [1]
  HTTP/1.1 200 OK
2026-10-16 07:28:20 URL:https://ext.lucee.org/org.lucee.oracle-21.8.0.0-ojdbc11.lex [4851095/4851095] -> "/usr/local/lib/serverHome/WEB-INF/lucee-server/deploy/org.lucee.oracle-21.8.0.0-ojdbc11.lex" [1]
  HTTP/1.1 200 OK
2026-10-16 07:28:20 URL:https://repo1.maven.org/maven2/org/xerial/sqlite-jdbc/3.50.3.0/sqlite-jdbc-3.50.3.0.jar [14346720/14346720] -> "/opt/wheels-lib/sqlite-jdbc-3.50.3.0.jar" [1]
/opt/wheels-lib/sqlite-jdbc-3.50.3.0.jar: OK

2. Ran a failing download, using the same flags against a missing ext.lucee.org object, in the pinned base image. Every attempt logs its status, then wget gives up after 5 tries (exit 4):

  HTTP/1.1 403 Forbidden
https://ext.lucee.org/org.lucee.does-not-exist-0.0.0.lex:
  ... (5 attempts)

3. For comparison, without -S. A non-retried status logs ERROR 403: Forbidden. and exits 8. A retried status logs only the URL on each attempt.

🤖 Generated with Claude Code

The Lucee 5, 6 and 7 test images downloaded their extensions and the
SQLite JDBC jar with `wget -q`, so a failed build reported only wget's
exit code 8 and not which download failed or what the server answered.

The downloads now use `wget -nv -S`, which logs each URL with the
response status and headers on every attempt. -S is needed because wget
1.21 prints no status for an attempt it is going to retry.

403 joins --retry-on-http-error. ext.lucee.org is S3 behind CloudFront,
which answers 403 rather than 404 for a missing object and on edge
errors; these pinned URLs exist, so a 403 there is transient. 404 is
not retried: only Maven Central returns it, for a real miss.

The SQLite JDBC SHA-256 check is unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: Peter Amiri <peter@alurium.com>
@github-actions

Copy link
Copy Markdown
Contributor

Wheels Test Results

44 786 tests   - 216 489   43 931 ✅  - 211 607   15m 0s ⏱️ - 1h 42m 6s
 3 600 suites  -  17 400      855 💤  -   4 882 
     6 files    -      29        0 ❌ ±      0 

Results for commit f1b6a17. ± Comparison against base commit a149b85.

This pull request removes 216489 tests.
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S1: authorize() throws Wheels.Policy.UnknownAction for a missing policy method
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S4: authorize() denies the CFML string true
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S4: authorize() denies the CFML string yes
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S4: authorize() returns the record when the policy returns boolean true
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S8: authorize() denial in production is HTTP 403, not a silent allow
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S9: authorize() with action=init throws Wheels.NotAuthorized and does not Invoke init
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ S9: authorize() with action=scope throws Wheels.NotAuthorized and does not Invoke scope
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ defaults the action from params.action at call time
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ denies a guest (no user)
adobe2023/cockroachdb :: wheels.tests.specs.Authorization.AuthorizationSpec :: Authorization policy layer (wheels.Policy + authorize()/can()/policyScope()) > authorize() ‑ denies the boolean false a missed finder returns
…

@bpamiri

bpamiri commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

Review: approve @ f1b6a17

The change is the same in all three Lucee Dockerfiles: -q becomes -nv -S, and 403 is added to --retry-on-http-error. The SHA-256 check from #4598 is untouched in each.

Retrying 403 makes sense. I checked the live servers:

  • A missing ext.lucee.org object answers 403 with server: AmazonS3 and x-cache: Error from cloudfront.
  • The pinned objects answer 200.

So a 403 on these URLs is an edge or transient error, and retrying it costs only a few seconds when the object really is missing. Leaving 404 out is right, because only Maven Central uses 404, and there it means the file is really gone.

Nothing sensitive is logged.

  • ext.lucee.org returns only S3 and CloudFront metadata.
  • Maven Central sets an anonymous, short-lived Cloudflare __cf_bm bot-management cookie. That isn't a credential.

CI is green. "Cold start: lucee7" builds the image from this branch, and its log shows the new lines:

  HTTP/1.1 200 OK
2026-10-16 07:30:31 URL:https://ext.lucee.org/org.lucee.h2-2.1.214.0001L.lex [2440188/2440188] -> ...
...
/opt/wheels-lib/sqlite-jdbc-3.50.3.0.jar: OK

The Lucee 5 and 6 images are built only by the compat matrix. Their edits are textually identical to Lucee 7, so the risk is low.

LOW (wording, optional): with -nv, -S prints only the status line. The log above shows HTTP/1.1 200 OK and no other headers. The new Dockerfile comment says each attempt logs "the response status and headers", and the PR description mentions seeing x-cache / x-amz-cf-pop, but those headers don't appear in the output. The status line is what the fix needs, so the behaviour is fine. To match what's logged, the comment could say "the response status".

@bpamiri

bpamiri commented Oct 16, 2026

Copy link
Copy Markdown
Collaborator Author

Correction: I'm withdrawing the LOW wording note in my review above. The unfiltered "Cold start: lucee7" log shows that -nv -S prints the full response headers after each status line, for example X-Cache: Hit from cloudfront, X-Amz-Cf-Pop: ORD58-P7 and X-Amz-Cf-Id. The excerpt I quoted had been filtered to the HTTP/ lines. The Dockerfile comment and the PR description are accurate. The approval stands.

@bpamiri
bpamiri merged commit ba1ff86 into develop Oct 16, 2026
32 checks passed
@bpamiri
bpamiri deleted the peter/engine-wget-nv branch October 16, 2026 07:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant