Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
356 changes: 3 additions & 353 deletions Cargo.lock

Large diffs are not rendered by default.

65 changes: 65 additions & 0 deletions changelog.d/11205-http-client-turnloop-drop-reqwest.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
`node:http` / `node:https` client: every request now runs on turnloop, and
`reqwest` and `tokio-rustls` are no longer dependencies of `perry-ext-http`
(tokio lane C). With #11144 having taken the server off hyper, this also removes
`reqwest`, `hyper`, `hyper-util`, `hyper-rustls`, `h2`, `tower` and
`tower-http` from `Cargo.lock` entirely. The tokio inventory goes from 11 to 9
manifest edges and from 14 to 7 tokio-family lockfile packages.

`client_turnloop` (now `src/client_turnloop/`) was lane 1's
bodyless-cleartext-GET-only path (#11091). It now carries every shape reqwest
did, plus the three raw tokio `TcpStream` bypasses:

- **Request bodies.** They are buffered at `end()`, so the length is always
known: `Content-Length`, or chunked when the caller set
`Transfer-Encoding: chunked`.
- **`options.timeout` / `req.setTimeout`.** A `tl::timer_arm` deadline over the
whole exchange, as reqwest's `RequestBuilder::timeout` was. It fires
`'timeout'` and tears the exchange down. The creation-time `'timeout'` timer
(`arm_client_timeout`) is a turnloop deadline too; it was a tokio sleep.
- **`https:`.** `perry_tls_session::TlsSession` runs above the same socket
handle, with the verifier `tls_client` already built from Node's options (CA,
`servername`/SNI, `rejectUnauthorized`, `checkServerIdentity`, PKCS#12 client
identities). Configs are cached per option identity, so TLS session
resumption still works.
- **Keep-alive.** An Agent with `keepAlive` reuses physical connections, using
the knobs reqwest's per-agent pool used (`maxFreeSockets`, `keepAliveMsecs`).
A connection goes back to the pool only after the decoder reports `End` and
`reusable()`. A reused connection that dies before any response byte is
retried once on a fresh one. `agent.destroy()` closes its idle connections.
- **`NODE_USE_ENV_PROXY=1`.** An `http:` target is sent in absolute-form
through the proxy; an `https:` target goes through a `CONNECT` tunnel. The
proxy URL's credentials become `Proxy-Authorization`.
- **`TE: trailers`, `Expect: 100-continue` and `Connection: Upgrade`** now run
on the codec's `Event::Trailers` / `Informational` / `Upgrade`. A `101` hands
the live handle to `net` with `turnloop_net::transfer`, as the server's
upgrade does. The old modules keep only their predicates and parsers.
- **Off-loop threads.** A thread that does not own its agent's loop posts the
request to the thread that does. A host with no loop at all reports
`ENOTSUP`.

Changes you can observe, each toward Node:

- `res.statusMessage` is now the server's own reason phrase, not the canonical
one.
- Unknown methods go out as written; reqwest sent them as `GET`.
- A caller's header names keep their case.
- `timeout: 0` means no timeout.
- `https` offers no ALPN, so there is no accidental HTTP/2.
- `https` requests get `'continue'` too.
- `req.destroy()` / `abort()` close the socket.
- Connect failures read `connect ECONNREFUSED 127.0.0.1:1` (lane 1 had dropped
the address), and a close before the response head is `socket hang up` /
`ECONNRESET`.
- TLS verification failures carry Node's `.code`
(`UNABLE_TO_VERIFY_LEAF_SIGNATURE`, `ERR_TLS_CERT_ALTNAME_INVALID`, …).
Before, they were an uncoded string.
- There is no 30-second default timeout any more (reqwest applied one; Node
does not).

Also fixed from lane 1: bytes of a response head (or chunk-size line) split
across two reads are kept for the next read instead of dropped.

Still on tokio in this crate (the one remaining `perry-ext-http -> tokio`
edge): the `agent.createConnection` / `createSocket` exchange
(`client_connect_override.rs` polls the raw-net vtable), and the keep-alive
socket facade's 40 ms idle-expiry sleep in `agent.rs`.
15 changes: 7 additions & 8 deletions crates/perry-ext-http/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "perry-ext-http"
version.workspace = true
edition.workspace = true
license.workspace = true
description = "Native bindings for Node's `http` / `https` modules — callback-style ClientRequest / IncomingMessage. Uses only `perry-ffi`. Async via spawn_blocking + reqwest."
description = "Native bindings for Node's `http` / `https` modules — callback-style ClientRequest / IncomingMessage over turnloop, with TLS via perry-tls-session."

[lints]
workspace = true
Expand All @@ -21,17 +21,16 @@ perry-ext-net.workspace = true
# see that module's header for why `turnloop_http::asynchronous` is not used.
turnloop-http.workspace = true
http = "1"
tokio-rustls.workspace = true
rustls = { workspace = true, features = ["std", "ring", "tls12"] }
rustls_webpki = { package = "rustls-webpki", version = "0.103" }
rustls-pemfile.workspace = true
reqwest = { version = "0.12", features = ["json", "rustls-tls", "http2"], default-features = false }
tokio = { workspace = true }
# Zero-copy body chunks: reqwest::Response::chunk() yields a refcounted
# `Bytes` that slices the receive buffer. Carrying that `Bytes` through the
# streaming event enum (instead of `.to_vec()`-ing it) drops one heap alloc
# + memcpy per response chunk. Already in the lockfile via reqwest/hyper, so
# declaring it pulls nothing new.
# The client transport (`client_turnloop`): the rustls session that runs
# `https:` above a turnloop socket, and the URL type requests are parsed into.
perry-tls-session.workspace = true
url.workspace = true
# Response body chunks travel through the streaming event enum as refcounted
# `Bytes`.
bytes.workspace = true
serde_json.workspace = true
lazy_static.workspace = true
Expand Down
Loading
Loading