Skip to content

http2: an answer waiting on the client's credit has its unread upload credited, so a client that uploads everything first gets it (#588) - #604

Closed
MDA2AV wants to merge 1 commit into
mainfrom
fix-588
Closed

MDA2AV wants to merge 1 commit into
mainfrom
fix-588

Conversation

@MDA2AV

@MDA2AV MDA2AV commented Nov 14, 2026 •

Copy link
Copy Markdown
Owner

Part of #588.

What was wrong

RFC 9113 8.1 lets a server send a complete response before the client has sent its whole request. A client that reads its response only once its upload has ended, as .NET's HttpClient does with a content that is not duplex, also credits the response only then. ioxide.http2 credits a request's stream only as its handler reads the body, so a handler that answered without reading left the upload holding the server's stream window (InitialWindowSize, 256 KiB by default). An answer past the client's stream window (65,535 for .NET) then waited on credit the client would give only after an upload the server would not take: neither side could move, and the request hung until the client gave up. Since the connection window was opened to 2^31-1, the threshold is the server's stream window: at origin/main an upload of 262,144 bytes is answered and one of 262,145 hangs, and it follows InitialWindowSize (65,535 and 1 MiB hang at exactly those sizes).

Three handler shapes hung:

  • a streamed answer whose handler returns writer.CompleteAsync() (the issue's shape): CompleteAsync waited for the client's credit, so the handler never returned, nothing was credited, and the RST_STREAM(NO_ERROR) that follows an early answer's END_STREAM never went out;
  • a streamed answer its handler left to the stack's completion, which waited the same way after the handler returned;
  • a buffered answer returned once the upload had filled the window: the rest of the answer parked on credit, and the body queued in the reader was dropped uncredited, leaving the client no window.

A related gap with a client that does credit while it uploads: a buffered answer parked on credit sent its END_STREAM with no RST_STREAM(NO_ERROR) after it, and the stream was credited nothing from then on, so the upload stalled and the request never completed.

What changes

All in src/protocols/ioxide.http2; no core change, no native change.

  • Http2ResponseWriter.CompleteAsync no longer waits for the peer's credit. What the windows cannot take yet goes out in the background as credit arrives, END_STREAM last, from the writer's own staging, and the connection retires the stream behind it. A buffered response's rest parked on credit takes the same path, so the two share one tail.
  • A stream whose handler is done while its answer waits on credit stays in the connection's stream table until the answer has ended. What the handler left unread is credited and dropped, as the nghttp2 twin's reader already does, and DATA still arriving is credited as it lands, still charged to the stream window and checked against the content-length. A request that has ended is credited nothing, as before.
  • Once that answer's END_STREAM is out, a request still arriving is reset with RST_STREAM(NO_ERROR) after it, as an answer that ends at once already is.
  • HandleData no longer credits DATA for a stream that has left the table because a buffered rest is parked: that stream now stays in the table, so the special case is gone.

Removed: the private CompleteParkedAsync, replaced by the shared tail.

Behaviour a caller would notice

  • CompleteAsync returns before the client's credit for the end has arrived. IsCompleted is true once it has returned, even while its END_STREAM still waits on credit. ResetStreamAsync after that sends nothing, as after any completion. FlushAsync after CompleteAsync now reports IsCompleted; it used to flush nothing and report the response live, and with the end parked it would have waited on the same credit, over the staging the parked end sends from.
  • A client that uploads its whole body before reading the answer now uploads all of it to a handler that answered past its window unread (credited and dropped), and gets the answer, where it used to hang. A client that credits while it uploads has its upload credited while the answer waits on its credit, so it stops later than before: up to 2.5 MiB measured, where a streamed answer's stopped at the 256 KiB stream window and a buffered answer's never stopped.
  • A body reader that has ended takes no more DATA: DATA after END_STREAM is dropped, uncredited, instead of being queued for the reader. http2: DATA after END_STREAM reaches a streamed reader instead of resetting the stream with STREAM_CLOSED #589 resets such a stream with STREAM_CLOSED; this does not.

Not fixed, and documented on Http2BodyReader: a handler that waits on FlushAsync for credit while not reading its body still waits until such a client gives up. Nothing can be credited while the handler may still read the body; the nghttp2 twin behaves the same. That case is why this PR is "Part of": merging it leaves #588 open for it.

Tests

Unit, Http2EarlyAnswerParkedTests, on the wire with the strict client (a 131,070-byte answer, twice the client's 65,535 window; the upload fills the server's 256 KiB stream window):

  • a buffered answer returned once the upload filled its window credits the 262,144 bytes it left unread, then the DATA still arriving, nothing once the request has ended, and the answer then ends whole with no reset;
  • CompleteAsync returns without the client's credit, and the unread upload is credited once the handler returns, the same way;
  • a streamed answer left to the stack's completion credits the unread upload, the same way;
  • a handler that reads its body after CompleteAsync parked its answer gets all of it, credited once (nothing is drained from under a handler still running);
  • a FlushAsync after CompleteAsync parked the answer reports it ended, and the answer goes out once;
  • a parked streamed answer gives its writer back when the client closes the connection;
  • a buffered and a streamed answer parked on credit, their END_STREAM out with the upload still arriving, are followed by RST_STREAM(NO_ERROR) as the stream's last frame.

E2E, EarlyAnswerTests, .NET HttpClient over h2c posting 64 MiB to a handler that answers a 256 KiB 413 unread:

  • non-duplex StreamContent: a streamed answer whose handler returns CompleteAsync, one left to the stack's completion, and a buffered one returned once the upload had stalled each arrive whole, and the upload ends at its last byte;
  • a duplex HttpContent: a buffered and a streamed answer arrive whole, and the upload stops under 32 MiB (measured up to 2.5 MiB);
  • the stale "64 KiB on h2, the connection window" comment now gives the stop measured today, up to 320 KiB: the stream window and what the client had buffered.

Changed: Http2FlowControlTests' "a writer parked on credit is released when the peer closes the connection cleanly" parks its handler on FlushAsync, since CompleteAsync no longer waits for credit and the test would otherwise pass without a parked handler.

Confirmed both ways. With src/protocols/ioxide.http2 at origin/main and the new tests in place:

  • Unit, 7 of the 8 new tests fail (the writer-release guard passes there):
    • "the handler returned with 262144 bytes unread and 0 came back on its stream: a client that credits the answer only once its upload ends has no window left to end it" (buffered, and the same for the stack's completion)
    • "CompleteAsync held the handler on credit that a client reading its answer only once its upload ends gives only then"
    • "the handler read 0 of the 262144 bytes sent with its request, its answer parked"
    • "a flush after the response had completed returned nothing, still waiting on credit"
    • "the answer ended with its request still arriving, and no RST_STREAM asked the client to stop uploading" (buffered), and "expected [17384], got [0]" for the streamed one's drain credit
  • E2E, 4 of the 5 new tests fail: the three non-duplex ones with "the answer never arrived (A task was canceled.); the upload is at 327680 of 67108864 bytes", and the duplex buffered one with "the upload neither finished nor stopped, at 524288 bytes". The duplex streamed one passes there; it guards the new path.

Each part reverted on its own, the rest of the fix in place:

  • CompleteAsync waiting for credit again: 5 unit tests fail ("CompleteAsync held the handler...", "the handler read 0 of the 262144 bytes...", "...returned nothing, still waiting on credit", "...262144 bytes unread and 0 came back" for the stack's completion, "expected [17384], got [0]"), and the two streamed non-duplex E2E tests ("the answer never arrived ... 327680 of 67108864 bytes").
  • The drop crediting nothing it held: 5 unit tests fail ("...262144 bytes unread and 0 came back on its stream" three times, "expected [17384], got [16384]" twice), and the stalled buffered E2E test (327680). The other two E2E shapes return at the HEADERS, before anything is queued, so the drop is not on their path.
  • An ended reader queuing DATA uncredited: 6 unit tests fail ("16384 more bytes of the unread upload arrived and 0 were credited" three times, "expected [17384], got [1000]" twice, and the existing "an upload still arriving while a buffered answer is parked on credit is credited on its stream" with "expected [16384], got [0]", since its old special case is gone), and the three non-duplex E2E tests (327680, 327680, 589824).
  • No reset after the parked END_STREAM (the stream retired while the answer's window still stands): both unit reset tests fail ("...no RST_STREAM asked the client to stop uploading"), and both duplex E2E tests ("the upload neither finished nor stopped, at 589824 bytes" and "at 786432 bytes").
  • A FlushAsync after completion taking up the staging again: "a flush after the response had completed returned nothing, still waiting on credit".

Suites

Pinned to the safe CPUs, on the final tree:

  • Unit: 271 passed, 0 failed (263 before, and the 8 new)
  • E2E: 735 passed, 0 failed (730 before, and the 5 new)
  • Chaos: 105 passed, 0 failed
  • Tls: 269 passed, 0 failed
  • Http: 104 passed, 0 failed

File, Pg and Redis were not run: nothing they cover changed.

For the reviewer

… credited, so a client that uploads everything first gets it (#588)

RFC 9113 8.1 lets a server answer before its request has arrived. A client that reads its response
only once its upload has ended, as .NET's HttpClient does with a content that is not duplex, credits
the response only then. An answer past that client's stream window waited on the credit for its
END_STREAM, while the server credited the request's stream only as its handler read: a handler that
answered unread left the upload holding the server's stream window (InitialWindowSize, 256 KiB by
default), and neither side could move. A streamed answer hung whether its handler awaited
CompleteAsync, which waited for that credit, or left the completion to the stack, which waited the
same way; a buffered one hung once the upload had filled its window before the handler returned,
since the queued body was dropped uncredited. And a buffered answer parked on credit to a client
that does credit while it uploads ended with no RST_STREAM(NO_ERROR) after its END_STREAM, the
upload then credited nothing, so that request never completed.

- Http2ResponseWriter.CompleteAsync no longer waits for the peer's credit: what the windows cannot
  take yet goes out in the background as credit arrives, END_STREAM last, and the stream is retired
  behind it. A buffered rest parked on credit takes the same path.
- A stream whose handler is done while its answer waits on credit stays open behind it: what the
  handler left unread is credited and dropped, as the nghttp2 twin's reader does, and DATA still
  arriving is credited as it lands, still held to its stream window and content-length.
- Once that answer's END_STREAM is out, a request still arriving is reset with NO_ERROR after it,
  as an answer that ends at once already is.
- HandleData's crediting of DATA for a stream that had left the table under a parked buffered rest
  is gone: such a stream now stays in the table.

IsCompleted is true once CompleteAsync has returned, its end possibly still waiting on credit, and
a FlushAsync after CompleteAsync reports IsCompleted. A handler that waits on FlushAsync for that
credit, its body unread, still waits until such a client gives up; Http2BodyReader's docs say so.

Tests: Unit Http2EarlyAnswerParkedTests on the wire, and E2E EarlyAnswerTests with .NET's
HttpClient over h2c, non-duplex and duplex, each failing at origin/main or with its part of the
fix reverted.
This was referenced Nov 14, 2026
@MDA2AV MDA2AV added the do not merge Kept open for reference; not to be merged label Nov 14, 2026
@MDA2AV

MDA2AV commented Nov 15, 2026

Copy link
Copy Markdown
Owner Author

Closed: the clean close it added (RST_STREAM(NO_ERROR) after an answer that waited on the client's credit) is now in #633. The rest, draining the upload of a client that reads only after it has sent everything, is not something the server will do: such a client waits until its own timeout, as it would with nginx or Go.

@MDA2AV MDA2AV closed this Nov 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

do not merge Kept open for reference; not to be merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant