Repository navigation
Description
Package: DotNext.Net.Cluster 6.11.1 (.NET 10), TCP and custom transports (ProtocolStream)
Summary
When a log entry's payload ends exactly where the leader's transmission buffer ends, the follower reads the next entry's
metadata 4 bytes early: it applies that entry with a term of 2147483648 (0x80000000), or the stream falls apart and the
member is reported unavailable.
On the leader, ProtocolStream.WriteAsync sends a frame that fills the buffer with a non-final header, so a payload ending
at the buffer's end is followed by an empty final frame (header 0x80000000). On the follower, ReceivedLogEntries.WriteToAsync
(Server.LogEntryProducer.cs) first tries ProtocolStream.TryReadFrameData. That succeeds whenever the frame holds the whole
payload, final or not, and leaves frameSize == 0 with readState == FrameStarted; the entry is then marked consumed, so
MoveNextAsync doesn't call SkipAsync, and the empty final frame's header is read as the start of the next entry's metadata.
// ProtocolStream.ReadOperations.cs, TryReadFrameData
EndReadFrameHeader();
var writtenBuffer = WrittenBuffer;
if (frameSize >= length && writtenBuffer.Length >= length) // also taken for a full, non-final frame
{
AdvanceReadCursor(length);
frame = writtenBuffer.Slice(0, length);
frameSize -= length;
return true;
}Why it matters
The follower's log silently diverges from the leader's: the entry is applied with a wrong term, or, when the misread
metadata's length is wrong too, the exchange fails (MemberUnavailableException on the leader). It depends only on where
an entry's payload ends in the leader's buffer, so any workload can hit it.
Reproduction
Three nodes on the TCP transport, TransmissionBlockSize 4096. F is cut off (behind a TCP proxy) while A and B commit two
entries, then let back in, so it receives both in one AppendEntries. For first-entry sizes from 3976 to 4116 bytes:
DotNext.Net.Cluster 6.11.1.0, TCP transport, TransmissionBlockSize 4096
size 4002: MISREAD. Leader sent entry 57 (term 0, 4002 bytes) and entry 58 (term 0, 100 bytes of 0xa3).
F applied entry 57: term 0, 4002 bytes, first bytes A2A2A2A2A2A2A2A2
F applied entry 58: term 2147483648, 100 bytes, first bytes A3A3A3A3A3A3A3A3
The same in every run. The smallest form is the transport test in the PR below, which sends two entries over TcpClient
and TcpServer (block size 400) for every first-entry size from 1 to 1000 bytes and checks what the server received;
on 6.11.1 it fails with MemberUnavailableException at the size that ends at the buffer's end.
Suggested fix
Take the payload in one go only from a final frame, so a full, non-final frame goes through the stream, which reads the
empty final frame too:
- if (frameSize >= length && writtenBuffer.Length >= length)
+ if (readState is ReadState.EndOfStreamReached && frameSize >= length && writtenBuffer.Length >= length)With it, the reproduction above finds no misread over the whole range, and the cluster tests pass.
Activity
sakno commentedon Nov 6, 2026
Fixed in 6.12.0.