Repository navigation
Conversation
…G frames are refused with the error RFC 9113 names, and a GOAWAY carries the last stream (#641) PRIORITY and PUSH_PROMISE were ignored, an RST_STREAM's payload was never looked at, a SETTINGS ACK returned before its length check, SETTINGS on a stream and SETTINGS_ENABLE_PUSH past 1 passed, and a PING was checked only for its length. Each is now refused as RFC 9113 says: - PRIORITY (6.3): on stream 0, GOAWAY(PROTOCOL_ERROR). Not five octets, a stream error of FRAME_SIZE_ERROR. A stream that depends on itself, a stream error of PROTOCOL_ERROR: RFC 9113 dropped that rule with the priority scheme, but RFC 7540 5.3.1 had it, and h2spec and nghttp2 still hold a server to it. A PRIORITY may name an idle stream, which no RST_STREAM may (6.4), so for one the connection answers with GOAWAY of the same code. A stream that is open is reset as RefuseBody resets one: its body reader ends and its response window goes, so nothing more goes out on it. RefuseBody now shares that helper, ResetOpenStream. - HEADERS: the priority fields are read, and a self-dependency is refused the same way. The block is still decoded aside, so HPACK stays in step; a new stream is reset once its block is in, with PROTOCOL_ERROR rather than the REFUSED_STREAM of a stream past the concurrency limit, which DiscardHeaderBlock now takes from a field. - PUSH_PROMISE from a client: GOAWAY(PROTOCOL_ERROR) (8.4). - RST_STREAM: on stream 0, GOAWAY(PROTOCOL_ERROR); not four octets, GOAWAY(FRAME_SIZE_ERROR) (6.4). - SETTINGS: on a stream, GOAWAY(PROTOCOL_ERROR); an ACK with a payload, GOAWAY(FRAME_SIZE_ERROR) (6.5); SETTINGS_ENABLE_PUSH other than 0 or 1, GOAWAY(PROTOCOL_ERROR) (6.5.2). - PING on a stream: GOAWAY(PROTOCOL_ERROR) (6.7). - GOAWAY's last-stream-id was always 0. It is now the highest stream the client opened, refused ones included (6.8): the client learns which requests it may retry, and REFUSED_STREAM already told it that a refused one may be. Unknown frame types are still ignored (4.1). h2spec 2.6.0 against a reactor serving h2c: the 12 cases of this issue fail on origin/main (5.3.1/1, 5.3.1/2, 5.4.1/1, 6.3/1, 6.3/2, 6.4/1, 6.4/3, 6.5/1, 6.5/2, 6.5.2/1, 6.7/3, 8.2/1) and pass with this; 35 failures go to 23. Tests: Http2ControlFrameTests, in Unit. Each malformed frame after a served stream 1 ends the connection with the expected GOAWAY naming stream 1, and stream 3 behind it is not served; PRIORITY of four octets or depending on itself resets a request still arriving and the next is served; HEADERS depending on their own stream are reset with no handler run and HPACK in step. A control sends a well-formed PRIORITY on a closed and on an idle stream, ENABLE_PUSH 0 and 1, an empty ACK and a PING, and both streams are served with the PING echoed. Run against origin/main's source every refusal test fails, "expected [GOAWAY 1 0x1], got []" and the like, and a DATA on an idle stream, refused before as well, "expected [GOAWAY 1 0x1], got [GOAWAY 0 0x1]"; the control passes either way. Suites: Unit 321, Chaos 105, Http 113, none failing.
…fused as RFC 9113 5.1 says, so DATA after END_STREAM no longer reaches a body reader (#640) The connection kept no state for a stream outside _streams, so a frame on a stream that was not open was dropped or served. Now: - Idle (5.1): RST_STREAM and WINDOW_UPDATE on a stream the client has not opened are GOAWAY (PROTOCOL_ERROR), as DATA already was. An even stream counts as idle everywhere, since only a server opens one and this one never does. - Stream ids (5.1.1): HEADERS on an even stream, or on one below the highest opened that was never opened itself, are GOAWAY(PROTOCOL_ERROR). Before, they were decoded and dropped as trailers. - Half-closed (remote) (5.1): DATA or HEADERS after the request's END_STREAM, with the response still going, reset the stream with STREAM_CLOSED, through ResetOpenStream, before anything reaches the body reader. A request with a body reader stayed in _streams after its END_STREAM and its DATA was pushed into a reader that had already ended; ReadAsync hands out queued chunks before it checks the end, so a handler still reading got bytes the client sent after it ended the body. The reset now aborts the reader as the peer's own reset does, so a handler still reading is told the stream is gone. - Closed (5.1): the connection keeps a ring of the 128 streams closed last and how each closed. - The peer closed it, by its END_STREAM (recorded when a buffered request is ready, or when a streamed one is retired with its request ended) or by its RST_STREAM: DATA or HEADERS on it are GOAWAY(STREAM_CLOSED). RFC 9113 makes that a MAY; RFC 7540 made it a MUST, and h2spec holds a server to it. - This side reset it (every ResetStream: REFUSED_STREAM, a malformed request, flow control, the NO_ERROR reset after an early answer) or stopped reading it (a streamed request retired with the upload behind a parked answer): its frames are still decoded where HPACK needs it and dropped, as RFC 9113 5.1 requires. - Not remembered, because it is older than the ring: DATA is dropped as before, and HEADERS are an unexpected stream id. RFC 9113 5.1 lets a receiver bound how long it remembers a stream it reset. WINDOW_UPDATE, RST_STREAM and PRIORITY on a closed stream are still ignored (5.1 allows them for a while). The ring is an int[128] per connection, written once per stream and searched only for a frame on a stream that is not open; an id this side closed is stored negated. h2spec 2.6.0 against a reactor serving h2c with a body-reader handler: the 12 cases of this issue fail on the fix-641 base (5.1/2, 5.1/3, 5.1/5, 5.1/6, 5.1/8, 5.1/9, 5.1/11, 5.1/12, 5.1.1/1, 5.1.1/2, 6.1/2, 6.4/2) and pass with this; 23 failures go to 11, those of #642 and #643. Tests: Http2StreamStateTests, in Unit. RST_STREAM and WINDOW_UPDATE on an idle stream, WINDOW_UPDATE and HEADERS on an even one, HEADERS below the highest opened, and DATA and HEADERS on a stream closed both ways or after the peer's RST_STREAM, each end the connection with the expected GOAWAY; DATA after END_STREAM with the handler still reading resets the stream and the reader hands out none of it; HEADERS after END_STREAM with the answer still to come reset the stream, no answer goes out and HPACK stays in step. Controls: WINDOW_UPDATE, RST_STREAM and PRIORITY on a stream closed both ways, and DATA and trailers on a stream this side reset after an early answer, take nothing down. Run against the fix-641 source every refusal test fails, "expected [GOAWAY 1 0x1], got []" and the like, the reader test with "expected [|reset], got [bodylate]"; the controls pass either way. Changed: Chaos's "h2c: DATA on an idle stream is a PROTOCOL_ERROR" had, as a control, DATA after a stream answered after the client's END_STREAM, served on. That is the closed-by-peer case above, now GOAWAY(STREAM_CLOSED), which the control asserts instead (H2cClient names 0x5). Its other control, DATA on a stream closed by opening a higher one, is unchanged. Suites: Unit 333, Chaos 105, Http 113, none failing.
…s, a te other than trailers or missing pseudo-headers are reset before dispatch, and so are malformed trailers of a streamed request (#642) RFC 9113 8.1.1 makes a malformed request a stream error of PROTOCOL_ERROR that the server must not process. PendingRequest.AddHeader checked the order of pseudo-headers and the content-length, and nothing else, so a transfer-encoding reached the handler like any other field. Now each of these sets Malformed, which DecodeHeaderBlock's existing reset turns into RST_STREAM(PROTOCOL_ERROR) before the request is dispatched, buffered or with a body reader: - a field name with an uppercase letter (8.2.1); - connection, keep-alive, proxy-connection, transfer-encoding or upgrade (8.2.2); - a te other than "trailers" (8.2.2); - once the block that opens the request is decoded (PendingRequest.EndHeaderBlock, in place of ClosePseudoHeaders): no :method, no :scheme, or no :path or an empty one (8.3.1). A CONNECT needs an :authority and neither :scheme nor :path (8.5). Trailers are not held to this. The field checks are a vectorized range search and a switch on the name's length per field. Trailers of a request with a body reader were decoded into a sink that ignored every field, so a pseudo-header in them went unseen, and a trailer block without END_STREAM was taken, leaving the handler to wait for an end that never came. They are now decoded aside through CheckDiscarded, which holds them to the same field checks and refuses any pseudo-header, and their END_STREAM is taken once the block is whole (EndTrailers) rather than at its first frame. Malformed trailers, or trailers without END_STREAM, abort the reader with the reason and reset the stream through ResetOpenStream, as RefuseBody does, so the handler hears of it at its next read and nothing more goes out on the stream. DiscardHeaderBlock zeroes its stream id before it decodes, so EndTrailers is handed the id, and a discarded block no longer reaches DecodeHeaderBlock's reset, which would have named stream 0. h2spec 2.6.0 against a reactor serving h2c with a body-reader handler: the 9 cases of this issue fail on the fix-640 base (8.1/1, 8.1.2/1, 8.1.2.1/3, 8.1.2.2/1, 8.1.2.2/2, 8.1.2.3/1-4) and pass with this. 144 of 146 pass; the two left, hpack/4.2/1 and hpack/5.2/2, are #643's. Tests, in Unit: - Http2MalformedRequestTests: each of the eleven shapes above, with a body reader, resets stream 1 with PROTOCOL_ERROR, no handler sees it, and stream 3 resolves the field its block inserted into the dynamic table; transfer-encoding does the same before a buffered handler. Control: te: trailers, lowercase fields and a CONNECT with only :authority are served. - Http2MalformedTrailerTests: trailers with a pseudo-header, with transfer-encoding, or without END_STREAM reset the stream, the handler's read raises with the reason after the three bytes it got, and no response goes out on it. Control: well-formed trailers end the body, and HPACK stays in step. Run against the fix-640 source every refusal test fails, "expected [RST_STREAM 1 0x1], got []" for the requests, "the handler read: abc" for the malformed trailers and "(still waiting for the end of the body)" for the ones without END_STREAM; both controls pass either way. Suites: Unit 350, Chaos 105, Http 113, none failing.
…nection instead of resetting a stream that may never have been opened (#641) Review of #647. RefusePriority reset any stream at or below the highest the client opened. That includes an even stream, which only a server opens, and an odd one the client skipped, and RFC 9113 6.4 forbids an RST_STREAM on a stream never opened. Nothing on this branch can tell a skipped stream from one closed long ago, so only a stream that is still live (still arriving, or still being answered) is reset now. A PRIORITY on any other stream is answered with GOAWAY of the same code. Tests, Http2PriorityResetTests in Unit: - A PRIORITY of four octets on an even stream, and on an odd one skipped below the highest, ends the connection. Against the previous commit's source they fail with "expected [GOAWAY 5 0x6], got [RST_STREAM 2 0x6]" (and 3). - Coverage the review asked for, of what the previous commit already did, so these fail against origin/main's source and pass against the previous commit's: - A PRIORITY of four octets, and trailers whose priority depends on their own stream, on a stream whose handler has a body read parked and a flush parked on credit: the read raises, the flush wakes and reports the peer gone, only the RST_STREAM goes out on the stream, and a self-dependent trailer block still keeps HPACK in step. With ResetOpenStream's ReleaseCreditWaiters removed, both fail with "the flush parked on credit never woke". - A self-dependent HEADERS whose block continues in CONTINUATION is reset once the block is in, and HPACK stays in step. - Self-dependent trailers reset a buffered request that is still arriving. Http2ControlFrameTests.Fields is internal now, so these tests can use it. Suites: Unit 327, Chaos 105, Http 113, none failing.
# Conflicts: # src/protocols/ioxide.http2/Http2Connection.Frames.cs # tests/Ioxide.Tests.Unit/Program.cs
…mes instead of ending the connection, and DATA after the peer's RST_STREAM resets the stream (#640) Review of #648. The ring of 128 closed streams decided too much. Suppose this side reset stream A (the NO_ERROR reset after an early answer, REFUSED_STREAM, a content-length mismatch, flow control) or stopped reading it behind a parked answer. With up to MaxConcurrentStreams (default 100, up to 1000) streams in flight, more than 128 later closes evict A. Its trailers, already in flight, then fell to the never-opened branch and got GOAWAY(PROTOCOL_ERROR), which killed a connection that had done nothing wrong. Before #640 they were decoded and dropped. A miss in the ring is no longer read as "never opened". The connection now records the ids a client skips: when a stream opens above the next id expected, the range in between goes into a second ring of 16 ranges (a long each, low id over high id). Only an id in one of those ranges is an unexpected stream id (RFC 9113 5.1.1). Any other stream at or below the highest that is neither open nor remembered is treated as one this side reset: its DATA and HEADERS are dropped and its blocks decoded, so a client is never refused for frames that were only late. A client that skips ids more than 16 times keeps only the newest ranges, and HEADERS on an older skipped id is dropped rather than refused. Neither ring needed sizing from MaxConcurrentStreams for that guarantee. DATA or HEADERS after the peer's own RST_STREAM now reset the stream with STREAM_CLOSED rather than end the connection, as RFC 9113 5.1 says for a stream closed by RST_STREAM (h2spec accepts either). After that reset the stream is this side's, so a second frame is dropped. The ring therefore records three kinds: an id with the kind above it in a long (ClosedByPeer, ClosedByPeerReset, ClosedByUs), and ClosedByUs still wins over the others. DATA and HEADERS on a stream the peer ended are refused through one helper, RefuseOnPeerClosed. Tests: - Changed: "a DATA after the peer's RST_STREAM" now expects RST_STREAM 1 STREAM_CLOSED, once for two DATA frames, with the connection going on. Against the previous source: "expected [RST_STREAM 1 0x5], got [GOAWAY 1 0x5]". - New, Http2ClosedStreamTests in Unit: - Trailers on a stream answered early, after 131 other closes, are dropped and the next stream is served. Against the previous source: "got [RST_STREAM 1 0x0, GOAWAY 263 0x1]". - Guards for behaviour the previous commit had, each confirmed red by a temporary mutation of this worktree's source and then restored: - DATA on a stream the peer ended and this side then reset is dropped (ClosedByUs wins). With the precedence reversed: "got [RST_STREAM 1 0x6, GOAWAY 1 0x5]". - DATA and trailers behind an answer parked on credit are dropped and the answer still ends whole. With RetireStream recording that stream as ClosedByPeer: a GOAWAY appears. - HEADERS continued in CONTINUATION on a stream this side reset, and on one the peer ended with its answer still to come, keep HPACK in step. Without the discard block's stream id: "GOAWAY 1 0x1" on both. - Not tested: DATA that is dropped is still charged to the connection window. That code is unchanged, it runs before the stream lookup, and the connection window opens to 2^31-1, so observing a top-up would take about 1.5 GiB of DATA. Suites: Unit 344, Chaos 105, Http 113, none failing.
# Conflicts: # tests/Ioxide.Tests.Unit/Program.cs
…edge whitespace, and empty :method, :scheme or CONNECT :authority make a request malformed; te is compared case-insensitively (#642) Review of #649. RFC 9113 8.2.1 says more about field bytes than the previous commit checked: - A name may hold only 0x21-0x7E and no uppercase letter. The name is now checked with a vectorized search for any byte outside 0x21-0x7E next to the existing uppercase search, and an empty name is refused too. - A value may hold no NUL, CR or LF, and may not start or end with a space or a tab. A vectorized IndexOfAny and two end checks now enforce that, for pseudo-header values too. Before this, "x-a: b\r\ntransfer-encoding: chunked" reached the handler as one field, which an HTTP/1.1 hop behind it would read as two lines. The same checks apply to the trailers of a streamed request, which go through CheckDiscarded. te is compared to "trailers" ASCII case-insensitively, as nghttp2 compares it. An empty :method or :scheme makes a request malformed, and so does an empty :authority on a CONNECT (8.3.1, 8.5). Tests, in Unit: - Http2FieldValidityTests, which reuses Http2MalformedRequestTests.Serve, now internal. These shapes reset stream 1 before any handler runs, with HPACK in step: a name with a space, a name with a byte past ASCII, a value with CR LF and a second field after it, a value with NUL, one with a leading space, one with a trailing tab, a :path with LF, an empty :method, an empty :scheme, a CONNECT with :scheme and :path, a CONNECT without :authority, a CONNECT with an empty :authority, and a :protocol. Control: te: TRAILERS, a space inside a value and visible punctuation in a name are served. - Http2MalformedTrailerTests gains: - an uppercase name and a CR LF value in a streamed request's trailers; - trailers whose CONTINUATION carries a pseudo-header after their HEADERS ended the stream (the ordering the previous commit fixed), with a control for well-formed trailers split the same way; - well-formed trailers on a body short of its content-length, whose read says how short; - uppercase trailers on a buffered request, which reset it unserved. - Red-checks: - Against the previous commit's source, the eleven new refusals and the CR LF trailer fail ("expected [RST_STREAM 1 0x1], got []", "the handler read: abc"), and so does the te control, "got [RST_STREAM 1 0x1]". - Against fix-640's source, before #642, the split-trailer, uppercase-trailer, buffered-trailer and CONNECT tests fail as well. - :protocol and the content-length trailer pass at both bases, since an earlier check already refused them. Removing EndTrailers' BodyFits check makes the content-length one fail, "the handler read: abc". h2spec 2.6.0 against the body-reader h2c server still passes 144 of 146; the two left, hpack/4.2/1 and hpack/5.2/2, are #643's. Suites: Unit 381, Chaos 105, Http 113, none failing.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #642.
Stacked on #647 and #648: it targets main so GitHub links and closes the issue, so until #647 and #648 merges its diff also shows those commits. Merge in order.
RFC 9113 8.1.1 makes a malformed request a stream error of PROTOCOL_ERROR that the server must not
process. PendingRequest.AddHeader checked the order of pseudo-headers and the content-length, and
nothing else, so a transfer-encoding reached the handler like any other field. Now each of these sets
Malformed, which DecodeHeaderBlock's existing reset turns into RST_STREAM(PROTOCOL_ERROR) before the
request is dispatched, buffered or with a body reader:
ClosePseudoHeaders): no :method, no :scheme, or no :path or an empty one (8.3.1). A CONNECT needs
an :authority and neither :scheme nor :path (8.5). Trailers are not held to this.
The field checks are a vectorized range search and a switch on the name's length per field.
Trailers of a request with a body reader were decoded into a sink that ignored every field, so a
pseudo-header in them went unseen, and a trailer block without END_STREAM was taken, leaving the
handler to wait for an end that never came. They are now decoded aside through CheckDiscarded, which
holds them to the same field checks and refuses any pseudo-header, and their END_STREAM is taken
once the block is whole (EndTrailers) rather than at its first frame. Malformed trailers, or trailers
without END_STREAM, abort the reader with the reason and reset the stream through ResetOpenStream, as
RefuseBody does, so the handler hears of it at its next read and nothing more goes out on the stream.
DiscardHeaderBlock zeroes its stream id before it decodes, so EndTrailers is handed the id, and a
discarded block no longer reaches DecodeHeaderBlock's reset, which would have named stream 0.
h2spec 2.6.0 against a reactor serving h2c with a body-reader handler: the 9 cases of this issue fail
on the fix-640 base (8.1/1, 8.1.2/1, 8.1.2.1/3, 8.1.2.2/1, 8.1.2.2/2, 8.1.2.3/1-4) and pass with
this. 144 of 146 pass; the two left, hpack/4.2/1 and hpack/5.2/2, are #643's.
Tests, in Unit:
with PROTOCOL_ERROR, no handler sees it, and stream 3 resolves the field its block inserted into the
dynamic table; transfer-encoding does the same before a buffered handler. Control: te: trailers,
lowercase fields and a CONNECT with only :authority are served.
END_STREAM reset the stream, the handler's read raises with the reason after the three bytes it
got, and no response goes out on it. Control: well-formed trailers end the body, and HPACK stays
in step.
Run against the fix-640 source every refusal test fails, "expected [RST_STREAM 1 0x1], got []" for
the requests, "the handler read: abc" for the malformed trailers and "(still waiting for the end of
the body)" for the ones without END_STREAM; both controls pass either way.
Suites: Unit 350, Chaos 105, Http 113, none failing.