Skip to content

Upgrade to AsyncHttpClient 3.0.14 - #1481

Merged
mkurz merged 13 commits into
playframework:mainfrom
mkurz:feature/ahc-3-clean-upgrade
Oct 6, 2026
Merged

mkurz merged 13 commits into
playframework:mainfrom
mkurz:feature/ahc-3-clean-upgrade

Conversation

@mkurz

@mkurz mkurz commented Oct 6, 2026

Copy link
Copy Markdown
Member

Upgrades Play WS from AsyncHttpClient 2.16.1 to AsyncHttpClient 3.0.14 with Netty 4.2.18, both still shaded into play.shaded.ahc.

Summary

  • Streaming keeps real backpressure. AHC 3 removed StreamedAsyncHandler. Play WS now bridges AHC 3's ResponseBodyControl to a Reactive Streams publisher that suspends socket reads while there is no downstream demand.
  • OAuth 1 signing keeps working. AHC 3 removed OAuthSignatureCalculator. Play WS now signs with its shaded Signpost dependency, and a test-only differential test checks the signatures against AHC 2.16.1's.
  • Existing configuration code stays compatible. AhcWSClientConfig gains new AHC 3 settings but keeps its 3.0 constructor, apply, copy and 12-field extractor on Scala 2 and Scala 3. Only the generated tupled and curried helpers are gone (see "Binary and API changes").
  • Behavior differences are documented. The README's new "AHC 3 migration notes" section covers redirects, cookies, connection pooling and streamed request bodies.
  • AHC 3.0.14 includes fixes for five security advisories. They are listed in the README; whether an older Play WS application is affected depends on its configuration and requests.

Commits

The history is split for review. Only the tip was built and tested; the individual commits were not.

  1. Upgrade Play WS to AsyncHttpClient 3.0.14.
    • Dependencies and shading. The shaded jar build now fails if any class outside play/shaded/ ends up in it, and JetBrains annotation classes are excluded.
    • The core adapters, and the README code examples updated for AHC 3's API.
    • The response publisher on ResponseBodyControl. It suspends reads only on the channel event loop, where AHC applies them inline, so a suspension can never override a newer resume.
  2. Test AHC 3 response streaming for races and backpressure.
    • Deterministic lost-resume and suspend-on-loop specs, plus the Reactive Streams TCK.
    • Live HTTP/1.1 and h2c backpressure tests.
    • Paranoid Netty leak detection in the integration tests, for both the test servers' Netty and the shaded client Netty.
  3. Re-check off-loop resumes on the event loop. AHC 3.0.14 queues a control call made off the event loop and applies it later without checking again. A stale resume can let one more socket read through with no demand left, and for a compressed body that read can inflate to tens of MiB. Play WS therefore runs the resume decision as a drain() pass on the event loop. It finds that loop with the shaded Netty's internal ThreadExecutorMap.
  4. Guard compressed streaming against an extra stale-resume read. An end-to-end regression test streams 64 MiB of gzipped zeros. With the re-check about 1.9 MB is buffered beyond demand; without it, about 35.6 MB. The test fails above 10 MiB.
  5. Reject replay of streamed request bodies. Source, File and input-stream-supplier bodies are one-shot. A redirect or retry that would send one again now fails explicitly instead of subscribing twice.
  6. Preserve OAuth 1 signing and Digest authentication. The Signpost calculator, the differential test, and SHA-256 Digest coverage.
  7. Preserve request construction and cache behavior. Charset normalization, Scala and Java request construction, and cache dispatch work as they did with AHC 2.
  8. Expose AHC 3 client controls without breaking existing config callers. The new settings below, their validation, the compatibility overloads and extractors, and MiMa filters.
  9. Keep HTTP/2 off while Play WS cannot negotiate it over HTTPS. Tests that HTTPS stays on HTTP/1.1 even with http2Enabled = true, and that h2c works when enabled explicitly.
  10. Expose opt-in redirect refusals. AHC 3.0.14's scheme-downgrade and cross-origin-body refusals, unset by default.
  11. Document the AHC 3 migration.
  12. Document the AHC 3.0.14 raw-Cookie limitation with a cookie store. Only relevant with useCookieStore = true.
  13. Document per-read decompression buffering.

Commits 3, 12 and 13 exist only because of AHC 3.0.14's current behavior. Each is self-contained, so it can be changed on its own once AHC releases a fix (see "Known limitations"):

  • 3 needs a replacement, not a plain revert;
  • 12 can be removed;
  • 13 needs revised wording.

Binary and API changes

MiMa (against 3.0.0) needs six filters. Three cover APIs that exposed AHC 2 types; the other three cover helpers that Scala generated for AhcWSClientConfig's companion:

  • DefaultStreamedAsyncHandler no longer extends the shaded AHC 2 StreamedAsyncHandler, and its onStream method is gone. The class is public, but Play WS creates it internally for streamed requests, and AHC 2 called onStream itself. Use StandaloneWSRequest.stream() and bodyAsSource, which are unchanged. The old interface no longer exists in AHC 3.
  • play.libs.oauth.OAuth.OAuthCalculator.getCalculator() now returns the shaded SignatureCalculator interface instead of the removed OAuthSignatureCalculator. Signing with request.sign(new OAuthCalculator(key, token)) is unaffected. Only code that calls getCalculator() directly is affected: it must be recompiled, and adapted if it refers to the removed OAuthSignatureCalculator type.
  • AhcWSClientConfig's companion no longer extends Function12 on Scala 2.13, so AhcWSClientConfig.tupled and .curried are gone. The constructor, apply, copy, 12-field pattern matching and AhcWSClientConfigFactory keep working. Scala 3 companions never extended function types.

Code that uses shaded AHC types directly, for example through underlying, must be updated for AHC 3.

Behavior changes against AHC 2

The README's "AHC 3 migration notes" cover these in detail:

  • Redirects:
    • 301 and 302 keep the method and body of non-POST requests; only POST is rewritten to a bodyless GET.
    • A 303 keeps HEAD and OPTIONS.
    • 307 and 308 keep method and body.
    • Combined with one-shot streamed bodies, a redirect that previously worked as a bodyless GET can now fail explicitly.
  • Cookies:
    • Cookies set through the Play WS cookie API now follow same-origin redirects; they are still stripped on cross-origin redirects.
    • A raw Cookie header is merged with API and cookie-store cookies on every hop.
    • Multiple raw Cookie headers are folded into one.
  • Connection pooling: connections authenticated as a whole (NTLM, SPNEGO/Kerberos, connection-wide proxy authentication) are pooled per principal.
  • Decompression: AHC decodes any Content-Encoding a server sends, even when Play WS did not ask for compression.
  • Clients you pass in yourself: an AsyncHttpClient handed to Play WS uses its own defaults, not all of Play WS's.

New configuration (play.ws.ahc.*)

Setting Default Notes
http2Enabled false HTTPS still uses HTTP/1.1, because Play WS's SSL engine factory offers no ALPN. h2c also needs AHC's http2CleartextEnabled, set through modifyUnderlying.
http2InitialWindowSize AHC default (16 MiB) Must be > 0. Counts wire bytes, not decoded bytes.
http2MaxConcurrentStreams AHC default (server-controlled)
maxDecompressedResponseSize AHC default One limit for HTTP/1.1 and HTTP/2.
refuseSchemeDowngradeOnRedirect AHC default (false) Refuses HTTPS → HTTP redirects.
refuseCrossOriginBodyOnRedirect AHC default (false) Refuses redirects that would resend a body to another origin. A same-host move from http to https is not refused.
shutdownQuietPeriod, shutdownTimeout 0 milliseconds Keeps the existing immediate-shutdown behavior.

Because AHC is shaded, AHC system properties keep the play.shaded.ahc. prefix, for example play.shaded.ahc.org.asynchttpclient.*. Netty's leak detector inside the shaded jar reads play.shaded.ahc.io.netty.leakDetection.level.

Known limitations and follow-ups

These come from AHC 3.0.14's current behavior. Play WS works around the first and documents the other two:

  1. Off-loop control calls are applied late (commit 3). Play WS's event-loop re-check depends on shaded Netty internals. It can be replaced once AHC offers a resume that is evaluated on the event loop.
  2. The raw Cookie header is re-encoded with a cookie store (commit 12). With useCookieStore = true, redirects and authentication retries re-encode the raw header: pairs can be reordered or dropped, and values that AHC's strict encoder rejects make the redirect or retry fail. An upstream fix is being prepared.
  3. Decompression works per socket read (commit 13). Each socket read of a compressed body, up to 64 KiB, is inflated in full before downstream demand is checked, exactly as with AHC 2.

Not in this PR: HTTP/2 over HTTPS (ALPN). It needs an ALPN-capable SSL engine factory that offers h2 only when both http2Enabled and AHC's per-connection http2Allowed are set. Turning HTTP/2 on by default would be a separate decision.

Testing

Local runs on the tip commit, all passing:

  • JDK 17, Scala 2.13: validateCode, mimaReportBinaryIssues and doc, plus all play-ahc-ws-standalone tests (18 suites) and integration-tests (15 suites);
  • Scala 3.3: Test/compile and mimaReportBinaryIssues.

The CI matrix (JDK 17/21/25 × Scala 2.13/3.3/3.9/3.next) runs on this PR.

New tests include:

  • the Reactive Streams TCK for the response publisher, and deterministic specs for the lost-resume, suspend-on-loop and off-loop-resume races;
  • live HTTP/1.1 and h2c backpressure tests, and the gzip buffering regression test;
  • the OAuth 1 differential test against AHC 2.16.1 (test scope only, not published), and SHA-256 Digest;
  • one-shot body replay on redirects and retries, and redirect refusals through the Scala and Java APIs;
  • AhcWSClientConfig source and binary compatibility;
  • HTTPS protocol selection against an h2-capable ALPN server.

mkurz added 13 commits October 6, 2026 10:30
Shade AHC 3.0.14 with Netty 4.2, keep annotation classes and any other unrelocated class out of the shaded jar, and adapt Play WS and the README examples to AHC 3's API.

AHC 3 removed the StreamedAsyncHandler that DefaultStreamedAsyncHandler.onStream belonged to, so remove that method and record the change for MiMa; use StandaloneWSRequest.stream() instead.

Bridge AHC 3's ResponseBodyControl to a Reactive Streams publisher with real backpressure. Suspend transport reads only on the channel event loop, where AHC applies them inline, so a suspension can never override a newer resume.
Cover the response publisher with deterministic lost-resume and suspend-on-loop specs, the Reactive Streams TCK, and live HTTP/1.1 and h2c streaming tests against backpressure-aware test servers.

Consume filtered streamed responses without blocking, so the filter spec does not depend on a short blocking timeout under load.

Run the integration tests with paranoid leak detection for both the test servers' Netty and the shaded Netty inside AHC, which reads its own relocated property.
AHC 3.0.14 queues a control call made off the event loop and applies it later without checking again. A stale resume can then let one more socket read through with no demand left, which inflates to tens of MiB for a compressed body.

Schedule the resume decision as a drain() pass on the event loop instead, and pin the drain race that a direct demand check would reintroduce in a deterministic test. Replace this once AHC offers a resume that is evaluated on the event loop.
Stream 64 MiB of gzipped zeros with early demand, three times, and require each run to buffer less than 10 MiB beyond demand. Measured: 1,875,916 bytes with the event-loop re-check, about 35.6 MB without it.
Play WS adapts Source, File and input-stream-supplier bodies to a one-shot stream. A redirect or retry that would send it again now fails explicitly instead of subscribing twice.
Keep the Signpost-based OAuth 1 calculator, verified against AHC 2.16.1's signatures in a test-only differential test, and cover SHA-256 Digest challenges.
Apply the AHC 2 text charset rule once at final request construction so filters see the original Content-Type, and share the normalization across Scala and Java builders.

Bypass the Play WS cache only for the streamed handler, retain failures for unsupported handlers, and document cache ByteBuf ownership.
Add HTTP/2 settings (HTTP/2 off by default), the decompressed-response limit and event-loop shutdown durations, with validation and documentation.

Keep AhcWSClientConfig's 12-argument constructor, apply, copy and 12-field extractor on Scala 2 and Scala 3, and record the remaining binary changes for MiMa.
Play WS's TLS engine factory does not offer ALPN, so HTTPS stays on HTTP/1.1 even with http2Enabled = true. Test that, test h2c when enabled explicitly, and document that the HTTP/2 window counts wire bytes rather than decoded data.
Add unset-by-default settings for AHC 3.0.14's scheme-downgrade and cross-origin-body redirect refusals, and set them on AHC only when configured. Document that AHC resends the body by default, that a same-host move to https is exempt, and how the refusals differ from credential stripping.

Cover unset, enabled and explicitly disabled values through the Scala and Java APIs, including same-origin replay, and assert that refused redirects never contact their target.
Describe API and binary changes, redirect semantics, cookie and connection-pooling changes against AHC 2, and the security fixes included in AHC 3.0.14.
With a cookie store, AHC 3.0.14 re-encodes a raw Cookie header on redirects and authentication retries, which can reorder or drop pairs and fails on values its strict encoder rejects. Remove this note when AHC keeps the caller's text.
AHC 3.0.14 inflates a whole socket read before demand is checked, as AHC 2 did. Adjust this note when AHC bounds decompression for suspended HTTP/1.1 responses.
@mkurz
mkurz merged commit a2e662d into playframework:main Oct 6, 2026
21 checks passed
@mkurz
mkurz deleted the feature/ahc-3-clean-upgrade branch October 6, 2026 09:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant