Skip to content

fix: bound WebSocket handshakes and request header reads - #1657

Merged
bkchr merged 5 commits into
masterfrom
bkchr-timeouts
Sep 30, 2026
Merged

bkchr merged 5 commits into
masterfrom
bkchr-timeouts

Conversation

@bkchr

@bkchr bkchr commented Sep 29, 2026

Copy link
Copy Markdown
Member

Client: WsTransportClientBuilder::connection_timeout only covered the TCP connect and the TLS handshake. The WebSocket upgrade then waited for the server's response without any timeout, so a peer that accepted the connection but never answered left build() hanging forever. A single such attempt was enough to wedge reconnecting clients, e.g. subxt's. The handshake is now bounded by connection_timeout as well, for both build and build_with_stream, and fails with
WsHandshakeError::Timeout.

Server: hyper's HTTP/1 header_read_timeout was never active because no timer was configured, and hyper-util's auto builder reads the protocol preface without any timeout. Connections that never sent a (complete) request header therefore stayed open forever. The server now configures a TokioTimer for HTTP/1 and bounds the first read, so such connections are closed after header_read_timeout (default 30s, configurable via ServerConfigBuilder::set_header_read_timeout; serve and serve_with_graceful_shutdown use the default). Idle HTTP/1 keep-alive connections are closed after the same timeout; upgraded WebSocket connections are not affected.

Client: `WsTransportClientBuilder::connection_timeout` only covered the
TCP connect and the TLS handshake. The WebSocket upgrade then waited for
the server's response without any timeout, so a peer that accepted the
connection but never answered left `build()` hanging forever. A single
such attempt was enough to wedge reconnecting clients, e.g. subxt's.
The handshake is now bounded by `connection_timeout` as well, for both
`build` and `build_with_stream`, and fails with
`WsHandshakeError::Timeout`.

Server: hyper's HTTP/1 `header_read_timeout` was never active because no
timer was configured, and hyper-util's auto builder reads the protocol
preface without any timeout. Connections that never sent a (complete)
request header therefore stayed open forever. The server now configures
a `TokioTimer` for HTTP/1 and bounds the first read, so such connections
are closed after `header_read_timeout` (default 30s, configurable via
`ServerConfigBuilder::set_header_read_timeout`; `serve` and
`serve_with_graceful_shutdown` use the default). Idle HTTP/1 keep-alive
connections are closed after the same timeout; upgraded WebSocket
connections are not affected.
@bkchr
bkchr requested a review from a team as a code owner September 29, 2026 10:38
Comment thread server/src/utils.rs Outdated
let read = Pin::new(&mut this.io).poll_read(cx, buf);

if read.is_ready() {
this.deadline = None;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It is possible to outplay this timeout if the remote sends a partial prefix for HTTP/2.
The loop here calls our function repeatedly as long as the received bytes match the prefix. So if remote sends P and then nothing for example, we wait here forever. So to fix this we would need to check here if the first byte uniquely identifies the protocol.

Comment thread server/src/utils.rs Outdated
type Future = S::Future;

fn poll_ready(&mut self, cx: &mut Context<'_>) -> Poll<Result<(), Self::Error>> {
self.service.poll_ready(cx)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it would be smarter to call notify_one() here.

Otherwise even if a client sends us something and the inner service does not signal readiness immediately, we might abort the connection before reaching call below, even though its not the clients fault.

bkchr and others added 2 commits September 29, 2026 22:35
Updated release date for version 0.26.1 and clarified changes.
@bkchr
bkchr merged commit a765cb7 into master Sep 30, 2026
12 checks passed
@bkchr
bkchr deleted the bkchr-timeouts branch September 30, 2026 06:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants