fix: release buffered data accounting when an incoming HTTP/2 stream is shut down - #1281
Open
pjfanning wants to merge 1 commit into
Open
fix: release buffered data accounting when an incoming HTTP/2 stream is shut down#1281pjfanning wants to merge 1 commit into
pjfanning wants to merge 1 commit into
Conversation
…is shut down
Motivation:
`IncomingStreamBuffer.shutdown()` fails the entity outlet and drops
whatever is still buffered, but unlike the other discard paths
(`onRstStreamFrame`, `onDownstreamFinish`) it never subtracted those
bytes from `totalBufferedData`.
One of its callers leaves the connection running: when a peer sends more
data than the stream-level window allows, `onDataFrame` resets that
single stream with FLOW_CONTROL_ERROR and carries on. The bytes still
buffered for the stream, plus the payload of the frame that is dropped
without ever being buffered, keep counting as buffered for the lifetime
of the connection. The connection-level flow controller only emits a
WINDOW_UPDATE while `outstanding + buffered` stays below half of
`incoming-connection-level-buffer-size`, so the leaked accounting
permanently reduces - and once it reaches half the buffer size, stops -
the replenishment of the connection window, and every stream on that
connection stalls.
Modification:
Release the buffer in `IncomingStreamBuffer.shutdown()` via the existing
`discardBuffer()`, and subtract the payload of the frame that trips the
stream-level window check, since that frame is never buffered either.
Result:
Resetting a stream for a flow-control violation releases the
connection-level window its data reserved, so the connection keeps being
replenished and no longer stalls.
Tests:
- New test "release connection-level flow control accounting when a stream-level window is exceeded" makes a peer overrun the stream window with the connection-level buffer sized so the discarded frame is more than half of it, then asserts the connection window is handed back in full. Without the fix no WINDOW_UPDATE is emitted at all.
- sbt "http2-tests/testOnly org.apache.pekko.http.impl.engine.http2.Http2ServerSpec" - Not run - environment failure: `http2-tests / update` cannot resolve `io.github.summerwind:h2spec_darwin_amd64:2.6.0` ("h2spec_darwin_amd64.tar.gz not found under https://github.com/summerwind/h2spec/releases/download/v2.6.0/"), which blocks the whole module. Verified on CI instead.
- The first CI run failed on the new test only, with "requirement failed: incoming-connection-level-buffer-size must be > 0": the buffer size was a `val` in the anonymous setup class, which is initialised after the superclass constructor has already read `settings`. The constants now live outside that class.
References:
None - releases buffered-data accounting when the incoming side of a stream is shut down
pjfanning
force-pushed
the
http2-release-buffered-data-on-shutdown
branch
from
September 7, 2026 08:33
0fc233a to
62ddc4f
Compare
pjfanning
marked this pull request as ready for review
September 7, 2026 08:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
IncomingStreamBuffer.shutdown()fails the entity outlet and drops whatever is still buffered, but unlike the other discard paths (onRstStreamFrame,onDownstreamFinish, fixed in #1259) it never subtracted those bytes fromtotalBufferedData.One of its callers leaves the connection running: when a peer sends more data than the stream-level window allows,
onDataFrameresets that single stream withFLOW_CONTROL_ERRORand the connection carries on. The bytes still buffered for the stream, plus the payload of the frame that is dropped without ever being buffered, keep counting as buffered for the lifetime of the connection.The connection-level flow controller only emits a
WINDOW_UPDATEwhileoutstanding + bufferedstays below half ofincoming-connection-level-buffer-size, so the leaked accounting permanently reduces the replenishment of the connection window — and once it reaches half the buffer size, stops it entirely, at which point every stream on that connection stalls.Modification
Release the buffer in
IncomingStreamBuffer.shutdown()through the existingdiscardBuffer(), and subtract the payload of the frame that trips the stream-level window check, since that frame is never buffered either.The other two
shutdown()callers are GOAWAY paths where the connection is going away anyway, so the extra release is a no-op for them.Result
Resetting a stream for a flow-control violation releases the connection-level window its data reserved, so the connection keeps being replenished and no longer stalls.
Tests
sbt "http2-tests/testOnly org.apache.pekko.http.impl.engine.http2.Http2ServerSpec"— Not run - environment failure:http2-tests / updatecannot resolveio.github.summerwind:h2spec_darwin_amd64:2.6.0(h2spec_darwin_amd64.tar.gz not found under https://github.com/summerwind/h2spec/releases/download/v2.6.0/), which blocks the whole module locally. Needs the CI run — that is why this is a draft.release connection-level flow control accounting when a stream-level window is exceededinHttp2ServerSpec: a peer overruns the stream-level window withincoming-connection-level-buffer-sizesized so the discarded frame is more than half of it, then the test asserts the connection window is handed back in full. Without the fix noWINDOW_UPDATEis emitted at all, so the expectation times out.requirement failed: incoming-connection-level-buffer-size must be > 0: the buffer size was avalin the anonymous setup class, so the superclass constructor readsettingsbefore it was initialised and saw0. The constants now live outside that class, and the final check drains window updates withpollForWindowUpdatesinstead of expecting exactly one frame.scalafmt --mode diff-ref=upstream/main— pass.impl.engine.http2change with no public API or binary shape impact.References
Refs #1259 — same accounting leak on the remaining discard path.