Skip to content

tcp: SendTimeoutMs can't tell a slow peer from a stalled one, though TCP_INFO can #538

Closed as not planned

Description

@MDA2AV

SendTimeoutMs is a deadline for the whole flush, and its doc is upfront about that: "a large body over a slow link is the false positive to watch for" (TcpOptions.cs:93-99). I'm filing it because of the reason given: "there is no per-chunk signal to measure progress against". That's true of the completions, but the kernel has one.

Measured at 0.25.482 (918ccc8), on x64 Linux 6.14. 12 MiB in one flush, to a client with a 4 KiB receive buffer that reads up to 16 KiB, then sleeps 5 ms.

SendTimeoutMs SendWaitAll Client received Took Longest gap between reads
1000 true 2.28 MiB of 12 MiB, then a clean EOF 2,974 ms 8 ms
1000 false 2.28 MiB of 12 MiB, then a clean EOF 2,987 ms 14 ms
0 true 12 MiB 15,601 ms 8 ms

So a client reading steadily is cut off, and what it sees looks like a normal end of stream. At about 0.77 MiB/s and the default 60 s, that client would be cut off in any single flush much above 45 MiB (an estimate, not run).

tcpi_bytes_acked from TCP_INFO tells the two cases apart. Sampled every 250 ms, the sweep's own interval, on a plain socket writing 12 MiB to the same kind of reader:

Peer tcpi_bytes_acked per tick
Reading slowly 204800, 409600, 614400, 815104, 1019904, 1224704, 1429504, 1634304
Stopped reading 1640448, 1640448, 1640448, 1640448

Fix options.

  • Restart the clock on progress. In TcpSweep, read TCP_INFO for connections with a flush outstanding, and restart the send clock when tcpi_bytes_acked moves (Reactor.Tcp.Sweep.cs:52-62). That's one syscall per sending connection per tick, on the sweep rather than the send path, so CompleteFlush stays untouched. The option then means what a reader would first assume: how long a peer may stop draining.
  • Keep it as it is, if a whole-flush deadline is wanted. A handler sending large bodies then has to flush in chunks.

Filed on kron too as MDA2AV/kron#24, where the doc lost the warning that ioxide's has.

Activity

  1. MDA2AV commented on Nov 8, 2026

    @MDA2AV
    OwnerAuthor

    Closing as working as designed: the deadline for the whole flush is what we want.

    A clock that restarts whenever tcpi_bytes_acked moves, as this issue proposed, would let a client that reads a little now and then hold a flush open for as long as it likes, and with it the connection and the slab holding the body. That's a slow-read attack. A deadline per flush bounds it, and nothing the client sends restarts that deadline.

    The false positive measured here, a large body to a slow client, is handled the way the doc already says: set SendTimeoutMs against the slowest legitimate response, or flush such a body in pieces. Each flush gets its own deadline, so the client only has to keep up the piece size per timeout, about 1 KiB/s for 64 KiB pieces at the default 60 s.

    One thing the doc could say better: the reason it gives, that there's no per-chunk signal to measure progress against, isn't the real one, since TCP_INFO has that signal. The reason to keep the whole-flush deadline is the slow-read case.

    On kron, MDA2AV/kron#24 is now a doc fix that brings kron's doc in line with this one: MDA2AV/kron#50.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions