RPC Connection Reuse and HTTP/2 Keep-Alive: Why New Connections Cost Latency

The per-request cost model: DNS, TCP, TLS, then JSON-RPC

A JSON-RPC call over HTTPS is not a single network event. Before the HTTP request line is sent, the client must resolve the hostname (DNS), complete a TCP three-way handshake, and negotiate TLS. Each of these steps adds at least one round trip to the server, and TLS 1.3 typically adds one round trip after TCP, while TLS 1.2 adds two. Only after the secure channel is established does the client send the HTTP POST containing the JSON-RPC payload.

When a connection is reused, the DNS lookup, TCP handshake, and TLS negotiation are skipped entirely. The client sends the request immediately on the existing socket. This is why connection reuse is one of the highest-leverage latency optimisations for RPC clients: it removes fixed per-call overhead that is independent of the method being called.

The HTTP/1.1 specification (RFC 9112, which obsoletes RFC 7230) defines persistent connections as the default, and RFC 9110 describes connection management semantics. HTTP/2 (RFC 9113) goes further by multiplexing many request/response streams over a single TCP connection. Both mechanisms exist to amortise connection setup cost.

• DNS resolution: often cached, but a cold client pays it once per new hostname.
• TCP handshake: one round trip (SYN, SYN-ACK, ACK).
• TLS handshake: one round trip for TLS 1.3, two for TLS 1.2, plus certificate verification.
• HTTP request/response: the actual JSON-RPC call, which is what you want to measure.
• Reused connection: removes DNS, TCP, and TLS from the critical path for every subsequent call.

HTTP/1.1 keep-alive: one request per connection, so you need a pool

HTTP/1.1 keep-alive allows a single TCP connection to carry multiple sequential request/response pairs. However, HTTP/1.1 does not multiplex: only one request can be in flight on a connection at a time. If a second request is sent before the first response arrives, it must wait, causing head-of-line blocking. To achieve concurrency, clients maintain a pool of sockets, each handling one request at a time.

The pool size (often called maxSockets) determines how many concurrent RPC calls you can make without queuing. If your application sends 50 concurrent calls but the pool has 10 sockets, 40 calls wait for a free socket. This waiting time appears as latency and inflates p95 and p99, even though the server may be fast.

Keep-alive also depends on both ends agreeing to keep the connection open. The server or an intermediary may send a Connection: close header or simply close idle connections after a timeout. Clients must handle this gracefully by opening a new connection when needed, but frequent reconnects reintroduce handshake costs.

• One in-flight request per connection; concurrency requires multiple sockets.
• Pool size must match expected concurrent request volume to avoid queuing.
• Idle connections may be closed by the server or a load balancer; clients should detect and replace them.
• HTTP/1.1 keep-alive is widely supported but less efficient than HTTP/2 for high concurrency.

HTTP/2 multiplexing: many streams over one connection, with limits

HTTP/2 multiplexes multiple request/response streams over a single TCP connection. This eliminates the need for a socket pool for concurrency: a client can send many JSON-RPC calls concurrently on one connection, and responses arrive interleaved. This reduces connection management overhead and improves latency under load because there is no pool queuing.

However, HTTP/2 is not unlimited. The server advertises SETTINGS_MAX_CONCURRENT_STREAMS, which caps how many streams can be active at once. If you exceed that limit, additional streams are queued by the client. Also, a single TCP connection is a single failure domain: if it breaks, all in-flight streams fail. Some clients mitigate this by maintaining a small number of HTTP/2 connections, but that reintroduces some pooling complexity.

HTTP/2 also has flow control at both the connection and stream level. A slow consumer or a large response can block other streams if flow control windows are exhausted. For typical JSON-RPC calls, which are small, this is rarely an issue, but it is worth monitoring if you fetch large payloads.

• One connection carries many concurrent streams; no per-socket pool needed for concurrency.
• SETTINGS_MAX_CONCURRENT_STREAMS limits active streams; excess streams queue.
• Single connection is a single point of failure; consider a small number of connections for redundancy.
• Flow control can introduce head-of-line blocking if windows are exhausted.

Idle timeouts and spurious resets: why ECONNRESET happens mid-use

Load balancers, NAT gateways, and provider edge proxies often close idle TCP connections after a fixed timeout (commonly 30–120 seconds, but this varies by provider and is documented / varies by provider). If your client believes the connection is still open and sends a request just as the intermediary closes it, the request fails with ECONNRESET or socket hang up. This is not a provider outage; it is a race between your send and the intermediary's idle timeout.

TCP keep-alive can help by sending periodic probes to keep the connection alive, but TCP keep-alive intervals are often too long (defaults of 2 hours on many systems) and must be tuned. Application-level heartbeats (e.g., a lightweight JSON-RPC call like eth_blockNumber or getHealth) are more reliable because they generate actual traffic that resets idle timers on all intermediaries.

For HTTP/2, the server may send GOAWAY frames to gracefully close a connection. Clients should handle GOAWAY by finishing in-flight streams and opening a new connection for subsequent requests. Ignoring GOAWAY leads to failed requests.

• Idle timeouts are common on load balancers and NATs; they close connections without notifying the client.
• TCP keep-alive helps but default intervals are often too long; tune them or use application heartbeats.
• HTTP/2 GOAWAY indicates connection shutdown; clients must reconnect.
• ECONNRESET is often a symptom of idle timeout, not a provider failure.

Keep-alive and WebSocket subscriptions: idle is the enemy

WebSocket subscriptions (e.g., Solana accountSubscribe or Ethereum eth_subscribe) use a long-lived connection that is idle by design: the server pushes data only when events occur. This makes them vulnerable to idle timeouts. Unlike request/response HTTP, where you can simply retry, a dropped subscription means missed events until the client reconnects and resubscribes.

To keep subscriptions alive, clients should send periodic ping/pong frames (WebSocket protocol-level) or application-level heartbeats. Many RPC providers document a maximum idle time for WebSocket connections; exceeding it results in disconnection. Always implement reconnection logic with exponential backoff and resubscribe on reconnect.

HTTP/2 is not used for WebSocket subscriptions in most JSON-RPC setups; WebSocket runs over its own TCP connection after an HTTP upgrade. So the connection reuse strategies for HTTP/2 do not apply to subscriptions. Treat them as separate concerns.

The multiplexing and stream-concurrency behaviour described here is defined by the HTTP/2 specification, RFC 9113; consult it for the exact SETTINGS_MAX_CONCURRENT_STREAMS and flow-control semantics your client must honour.

• WebSocket subscriptions are long-lived and idle; idle timeouts cause disconnects.
• Use ping/pong or application heartbeats to keep the connection alive.
• Implement reconnection and resubscribe logic; missed events are not replayed.
• HTTP/2 multiplexing does not apply to WebSocket subscriptions.

Serverless and new-client-per-request anti-patterns

Serverless functions (AWS Lambda, Cloudflare Workers, etc.) often create a new HTTP client per invocation, which means a new TCP and TLS handshake for every RPC call. This wastes tens to hundreds of milliseconds per call and inflates p95 latency. Some runtimes allow reusing clients across invocations via global variables, but cold starts still pay the handshake cost.

Similarly, creating a new client instance per request in a long-running server is an anti-pattern. The client should be instantiated once and reused. If your framework creates a new client per request by default, override that behaviour. Connection reuse is a client-side responsibility; the server cannot help if you keep knocking on the door with a new handshake.

For serverless, consider using a provider that supports HTTP/2 and keep-alive, and configure your client to reuse connections within the execution environment. If cold starts are unavoidable, measure the handshake cost and factor it into your latency budget.

• New client per request = new handshake per request = wasted latency.
• Reuse client instances across requests in long-running servers.
• Serverless cold starts pay handshake cost; reuse clients across invocations where possible.
• Measure handshake cost to understand its impact on your p95.

Rate limiting: connection count is not the limit, method weight is

RPC providers typically rate limit based on request count or method weight, not on the number of TCP connections. Pooling connections reduces latency but does not increase your rate limit. If you exceed the limit, you will receive 429 responses regardless of how many connections you use. Conversely, using fewer connections does not reduce your rate limit consumption.

Some providers may have separate limits on concurrent connections or streams, but these are usually generous compared to request limits. The key takeaway: optimise connection reuse for latency, but do not expect it to raise your throughput ceiling. For throughput, consider batching (see JSON-RPC batch requests and best practices) or upgrading your plan (see RPC pricing).

If you are hitting rate limits, the solution is to reduce request count (batching, caching) or increase your plan, not to open more connections.

• Rate limits are based on requests or method weight, not connection count.
• Connection pooling improves latency, not rate limit headroom.
• Batching and caching reduce request count and help stay under limits.
• Check your provider's documentation for specific rate limit semantics.

Runnable Node.js example: undici Agent with keep-alive and measurement loop

The following Node.js script uses undici (the HTTP/1.1 client used by Node.js fetch) to create an Agent with keep-alive enabled, a maxSockets pool, and a keepAliveTimeout. It then measures the latency of the first request (which pays the handshake) versus subsequent requests (which reuse the connection). Replace the RPC_URL with your endpoint.

The script also demonstrates how to set a short keepAliveTimeout to avoid idle timeouts, and how to measure the difference. Run it with Node.js 18+ (which includes undici). The output will show the first request taking significantly longer than steady-state requests, illustrating the handshake cost.

const { Agent, request } = require('undici');

const RPC_URL = 'https://your-rpc-endpoint.example.com';
const agent = new Agent({
  keepAliveTimeout: 10_000, // 10 seconds; tune to be less than intermediary idle timeout
  keepAliveMaxTimeout: 60_000,
  maxSockets: 20, // pool size for HTTP/1.1 concurrency
  pipelining: 1, // HTTP/1.1 pipelining is generally not recommended
});

async function rpcCall(method, params = []) {
  const start = process.hrtime.bigint();
  const { statusCode, body } = await request(RPC_URL, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ jsonrpc: '2.0', id: 1, method, params }),
    dispatcher: agent,
  });
  const text = await body.text();
  const end = process.hrtime.bigint();
  const ms = Number(end - start) / 1e6;
  return { statusCode, ms, text };
}

(async () => {
  console.log('First request (pays handshake):');
  const first = await rpcCall('eth_blockNumber');
  console.log(`  ${first.ms.toFixed(2)} ms, status ${first.statusCode}`);

  console.log('Steady-state requests (reuse connection):');
  for (let i = 0; i < 5; i++) {
    const res = await rpcCall('eth_blockNumber');
    console.log(`  ${res.ms.toFixed(2)} ms, status ${res.statusCode}`);
  }

  await agent.close();
})();

Measurement method: first-request vs steady-state latency table

To quantify the benefit of connection reuse on your endpoint, run a measurement loop that records the latency of the first request (cold connection) and the next N requests (warm connection). Use the same method and parameters for all calls. Run the test multiple times to account for network variability. The table below is a template; fill it with your own measurements.

Record the median and p95 of warm requests, and compare to the cold request. The difference is the handshake cost. If the difference is large relative to your latency budget, connection reuse is a high-priority optimisation. Also measure the effect of different maxSockets values under concurrency to find the point where queuing disappears.

• Run the script above against your endpoint.
• Record the first request latency (cold) and the next 20 requests (warm).
• Compute median and p95 of warm requests.
• Repeat with different maxSockets values (e.g., 5, 10, 20, 50) under a concurrent load of 50 requests.
• Document the results in a table like the one below.

| Run | Cold request (ms) | Warm median (ms) | Warm p95 (ms) | maxSockets | Concurrent requests |
|-----|-------------------|------------------|---------------|------------|---------------------|
| 1   |                   |                  |               | 10         | 50                  |
| 2   |                   |                  |               | 20         | 50                  |
| 3   |                   |                  |               | 50         | 50                  |

Tuning guide: HTTP/1.1 pooling vs HTTP/2 multiplexing

The tuning parameters differ between HTTP/1.1 and HTTP/2. For HTTP/1.1, the key is pool size (maxSockets) and keepAliveTimeout. For HTTP/2, the key is the number of connections (usually 1–2) and handling of SETTINGS_MAX_CONCURRENT_STREAMS. The table below summarises what to tune.

For HTTP/1.1, set maxSockets to at least your expected concurrent request count. Set keepAliveTimeout to a value shorter than the intermediary idle timeout (e.g., 10–30 seconds). For HTTP/2, use a single connection if the server supports enough concurrent streams; otherwise, use a small number of connections. Monitor for GOAWAY frames and reconnect as needed.

• HTTP/1.1: tune maxSockets (pool size) and keepAliveTimeout.
• HTTP/2: tune number of connections and respect SETTINGS_MAX_CONCURRENT_STREAMS.
• Both: implement retry with backoff for ECONNRESET and GOAWAY.
• Both: use application-level heartbeats if idle timeouts are aggressive.

| Protocol | Key parameter          | Typical starting value | Notes                                      |
|----------|------------------------|------------------------|--------------------------------------------|
| HTTP/1.1 | maxSockets             | 20–50                  | Match expected concurrency                 |
| HTTP/1.1 | keepAliveTimeout       | 10–30 s                | Less than intermediary idle timeout        |
| HTTP/2   | connections            | 1–2                    | One connection multiplexes many streams    |
| HTTP/2   | maxConcurrentStreams   | Server-advertised      | Client should not exceed                   |

Troubleshooting: common failures and how to diagnose them

If you see ECONNRESET or socket hang up, first check whether it correlates with idle periods. If it happens after a period of inactivity, it is likely an idle timeout. Increase heartbeat frequency or reduce keepAliveTimeout. If it happens under load, check if you are exceeding maxSockets and queuing, or if the server is closing connections due to rate limits.

If you accidentally disable keep-alive (e.g., by setting Connection: close in a framework default), you will pay handshake costs on every request. Check your client configuration. If you are using a load balancer that resets idle connections, ensure your keepAliveTimeout is shorter than the balancer's idle timeout. If you assume one connection gives more throughput than a bounded pool, measure: under HTTP/1.1, one connection serialises requests, so a pool is necessary for concurrency.

Misreading ECONNRESET as a provider outage is common. Before escalating, verify with a simple curl or a script that reuses connections. If the error disappears with connection reuse, it was an idle timeout, not an outage. For more on timeouts, see RPC timeout errors: causes and fixes.

• ECONNRESET after idle period → idle timeout; add heartbeats or reduce keepAliveTimeout.
• ECONNRESET under load → check maxSockets and rate limits.
• Keep-alive disabled by framework default → check client config.
• Load balancer resets idle connections → set keepAliveTimeout shorter than balancer idle timeout.
• Assuming one connection is enough for concurrency → measure and use a pool for HTTP/1.1.

Limitations, tradeoffs, and when to revisit

Connection reuse is not a silver bullet. It reduces handshake latency but does not reduce server processing time or network latency between you and the provider. If your RPC calls are slow due to server-side load, connection reuse will not fix that. Also, a single HTTP/2 connection is a single point of failure; if it breaks, all in-flight requests fail. Some clients mitigate this with multiple connections, but that adds complexity.

Idle timeouts are provider-specific and can change. What works today may not work tomorrow if the provider adjusts its infrastructure. Monitor your error rates and latency, and be prepared to adjust keepAliveTimeout and heartbeat intervals. For WebSocket subscriptions, reconnection logic is mandatory regardless of keep-alive settings.

Revisit your configuration when you change providers, when you see new error patterns, or when your concurrency profile changes. Use the measurement method above to validate that your settings still deliver the expected benefit. For a broader overview of latency causes, see RPC latency: causes, measurement and fixes.

• Connection reuse does not reduce server processing or network latency.
• Single HTTP/2 connection is a single point of failure.
• Idle timeouts are provider-specific and may change.
• Revisit configuration when changing providers or concurrency profiles.
• Always implement reconnection for WebSocket subscriptions.

Next steps: applying connection reuse to your stack

Start by measuring the handshake cost on your endpoint using the script above. If the cold request is significantly slower than warm requests, enable keep-alive and tune your pool size. For HTTP/1.1, set maxSockets to match your concurrency. For HTTP/2, use a single connection and respect stream limits. Implement heartbeats for WebSocket subscriptions.

If you are using OnFinality, review the RPC endpoints guide for endpoint-specific details. For Solana-specific latency considerations, see Solana RPC latency: measuring and optimizing. For a general overview of OnFinality's RPC offerings, visit OnFinality Learn hub and API service.

Finally, remember that connection reuse is one part of a broader performance strategy. Combine it with batching, caching, and appropriate rate limit management. Measure, tune, and monitor continuously.

• Measure handshake cost on your endpoint.
• Enable keep-alive and tune pool size or HTTP/2 connections.
• Implement heartbeats for WebSocket subscriptions.
• Combine with batching and caching for best results.
• Monitor and revisit as your usage changes.