Skip to content

e2e: cover actor UDP and raw TCP egress - #2580

Open
Bo-Cheng Chu (bochengchu) wants to merge 1 commit into
agent-substrate:mainfrom
bochengchu:egress-e2e-combined
Open

Bo-Cheng Chu (bochengchu) wants to merge 1 commit into
agent-substrate:mainfrom
bochengchu:egress-e2e-combined

Conversation

@bochengchu

@bochengchu Bo-Cheng Chu (bochengchu) commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

e2e: cover actor UDP and raw TCP egress

Combines the stalled #1641 (UDP) and #1024 (raw TCP), updated to the current network model and the GA spec.

  • TestActorEgressRawTCPBlocked: TCP that is neither HTTP nor TLS never reaches the origin. If the server speaks first, the gateway closes with no bytes; if the client speaks first (text line or binary), the gateway answers 400 Bad Request. In-cluster origins cover both. The test asserts the CONNECT record and, on Envoy, down_bytes=0 on both tunnel records for the server-speaks-first case.
  • TestActorUDPEgress: a control pod outside any sandbox must get an echo on 53, 443 and 9999. The actor must then get a read timeout on all 5 attempts at each port, and DNS must still resolve through the sandbox relay.
  • Supporting fixtures with unit tests: testserver tcpecho / udpecho, probe --listen, demo /tcp with skipBanner and closed, batched DeployServerPods, and a TCP readiness probe.

@bochengchu
Bo-Cheng Chu (bochengchu) force-pushed the egress-e2e-combined branch 2 times, most recently from 0c3614f to a2800db Compare October 13, 2026 21:56
@bowei Bowei Du (bowei) added area/network area/security Security related issue/pr area/tests Enhancing / fixing test coverage. kind/feature An enhancement / feature request or implementation labels Oct 14, 2026
@bochengchu
Bo-Cheng Chu (bochengchu) marked this pull request as ready for review October 14, 2026 18:16
Combines two egress e2e changes, rebased onto main:

- UDP egress: a udpecho origin publishing 53, 443 and 9999 onto one
  listener, and TestActorUDPEgress.
- Raw TCP egress: a tcpecho origin that greets or stays silent, a /tcp
  endpoint in the egress demo, TCP readiness probes and batched ServerPod
  deploys, and TestActorEgressRawTCPBlocked.

Each sandbox's only route out is its gateway address, where a relay serves
DNS, so UDP to any other address, port 53 included, goes nowhere.

TestActorUDPEgress first runs the probe binary as a plain pod outside any
sandbox, which must get an echo on every port, so a broken origin or
Service cannot make the drops pass. It retries while kube-proxy has not yet
programmed the new Service's UDP ports. The probe actor must then send all
of its attempts on every port and see each end in a read timeout, the
signature of a silent drop rather than a failed dial or send or an ICMP
error, and must still resolve the origin's Service name. The probe grows
--listen so it can serve on an unprivileged port as that pod.

Actor TCP that is neither HTTP nor TLS is refused by the gateway, and what
the actor sees depends on who speaks first. If the actor waits for the
server, the gateway has nothing to classify, gives up on its listener
timeout and closes the connection with no bytes returned. If the actor
sends first, the gateway treats bytes that are not TLS as cleartext HTTP
and its HTTP handler answers with its own 400 Bad Request. The test covers
both: against an in-cluster origin that greets and one that echoes, the
actor must never get the greeting or its payload back, and the gateway must
have logged the CONNECT, so the refusal is the gateway's. When the actor
waits, the connection must also be closed with no bytes, and on Envoy both
the establish and close records of the tunnel must show down_bytes=0. When
it sends a text line or binary bytes, it must get HTTP/1.1 400 Bad Request.

The demo's /tcp endpoint grows a skipBanner option so the client-speaks-
first cases write immediately rather than waiting out the gateway's
listener timeout, and reports closed when the peer closes or resets the
connection, which an empty read alone cannot tell from a timeout.

Other adjustments made while combining:

- The tests create their actors with an allow-all policy and use the
  atespace-aware assertEgressGatewayConnect.
- renderServerPod returns manifest text, so renderServerPodDocs and
  CreateServerService use it directly.
- serveLocal in the tcpecho tests waits for its serve goroutine in cleanup,
  so it can no longer log after the test returns (a race under -race).
@bochengchu

Copy link
Copy Markdown
Contributor Author

Bowei Du (@bowei) this is ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/network area/security Security related issue/pr area/tests Enhancing / fixing test coverage. kind/feature An enhancement / feature request or implementation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants