Skip to content

fix: release buffered data accounting when an incoming HTTP/2 stream is shut down - #1281

Open
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:http2-release-buffered-data-on-shutdown
Open

fix: release buffered data accounting when an incoming HTTP/2 stream is shut down#1281
pjfanning wants to merge 1 commit into
apache:mainfrom
pjfanning:http2-release-buffered-data-on-shutdown

Conversation

@pjfanning

@pjfanning pjfanning commented Sep 7, 2026

Copy link
Copy Markdown
Member

Motivation

IncomingStreamBuffer.shutdown() fails the entity outlet and drops whatever is still buffered, but unlike the other discard paths (onRstStreamFrame, onDownstreamFinish, fixed in #1259) it never subtracted those bytes from totalBufferedData.

One of its callers leaves the connection running: when a peer sends more data than the stream-level window allows, onDataFrame resets that single stream with FLOW_CONTROL_ERROR and the connection carries on. The bytes still buffered for the stream, plus the payload of the frame that is dropped without ever being buffered, keep counting as buffered for the lifetime of the connection.

The connection-level flow controller only emits a WINDOW_UPDATE while outstanding + buffered stays below half of incoming-connection-level-buffer-size, so the leaked accounting permanently reduces the replenishment of the connection window — and once it reaches half the buffer size, stops it entirely, at which point every stream on that connection stalls.

Modification

Release the buffer in IncomingStreamBuffer.shutdown() through the existing discardBuffer(), and subtract the payload of the frame that trips the stream-level window check, since that frame is never buffered either.

The other two shutdown() callers are GOAWAY paths where the connection is going away anyway, so the extra release is a no-op for them.

Result

Resetting a stream for a flow-control violation releases the connection-level window its data reserved, so the connection keeps being replenished and no longer stalls.

Tests

  • sbt "http2-tests/testOnly org.apache.pekko.http.impl.engine.http2.Http2ServerSpec"Not run - environment failure: http2-tests / update cannot resolve io.github.summerwind:h2spec_darwin_amd64:2.6.0 (h2spec_darwin_amd64.tar.gz not found under https://github.com/summerwind/h2spec/releases/download/v2.6.0/), which blocks the whole module locally. Needs the CI run — that is why this is a draft.
  • New test release connection-level flow control accounting when a stream-level window is exceeded in Http2ServerSpec: a peer overruns the stream-level window with incoming-connection-level-buffer-size sized so the discarded frame is more than half of it, then the test asserts the connection window is handed back in full. Without the fix no WINDOW_UPDATE is emitted at all, so the expectation times out.
  • The first CI run failed on the new test only, with requirement failed: incoming-connection-level-buffer-size must be > 0: the buffer size was a val in the anonymous setup class, so the superclass constructor read settings before it was initialised and saw 0. The constants now live outside that class, and the final check drains window updates with pollForWindowUpdates instead of expecting exactly one frame.
  • scalafmt --mode diff-ref=upstream/main — pass.
  • MiMa not run: internal impl.engine.http2 change with no public API or binary shape impact.

References

Refs #1259 — same accounting leak on the remaining discard path.

…is shut down

Motivation:
`IncomingStreamBuffer.shutdown()` fails the entity outlet and drops
whatever is still buffered, but unlike the other discard paths
(`onRstStreamFrame`, `onDownstreamFinish`) it never subtracted those
bytes from `totalBufferedData`.

One of its callers leaves the connection running: when a peer sends more
data than the stream-level window allows, `onDataFrame` resets that
single stream with FLOW_CONTROL_ERROR and carries on. The bytes still
buffered for the stream, plus the payload of the frame that is dropped
without ever being buffered, keep counting as buffered for the lifetime
of the connection. The connection-level flow controller only emits a
WINDOW_UPDATE while `outstanding + buffered` stays below half of
`incoming-connection-level-buffer-size`, so the leaked accounting
permanently reduces - and once it reaches half the buffer size, stops -
the replenishment of the connection window, and every stream on that
connection stalls.

Modification:
Release the buffer in `IncomingStreamBuffer.shutdown()` via the existing
`discardBuffer()`, and subtract the payload of the frame that trips the
stream-level window check, since that frame is never buffered either.

Result:
Resetting a stream for a flow-control violation releases the
connection-level window its data reserved, so the connection keeps being
replenished and no longer stalls.

Tests:
- New test "release connection-level flow control accounting when a stream-level window is exceeded" makes a peer overrun the stream window with the connection-level buffer sized so the discarded frame is more than half of it, then asserts the connection window is handed back in full. Without the fix no WINDOW_UPDATE is emitted at all.
- sbt "http2-tests/testOnly org.apache.pekko.http.impl.engine.http2.Http2ServerSpec" - Not run - environment failure: `http2-tests / update` cannot resolve `io.github.summerwind:h2spec_darwin_amd64:2.6.0` ("h2spec_darwin_amd64.tar.gz not found under https://github.com/summerwind/h2spec/releases/download/v2.6.0/"), which blocks the whole module. Verified on CI instead.
- The first CI run failed on the new test only, with "requirement failed: incoming-connection-level-buffer-size must be > 0": the buffer size was a `val` in the anonymous setup class, which is initialised after the superclass constructor has already read `settings`. The constants now live outside that class.

References:
None - releases buffered-data accounting when the incoming side of a stream is shut down
@pjfanning
pjfanning force-pushed the http2-release-buffered-data-on-shutdown branch from 0fc233a to 62ddc4f Compare September 7, 2026 08:33
@pjfanning
pjfanning marked this pull request as ready for review September 7, 2026 08:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant