Polkadot RPC Timeouts: WsProvider Reconnects, Block RPC Limits, and Reliable Queries

Direct Answer: What Causes a Polkadot RPC Timeout?

A Polkadot RPC timeout happens when a request to a node over WebSockets (wss://) or HTTP (https://) does not get a response within the client's time limit, typically 60 seconds. On Polkadot, this is often not a network issue but a consequence of the node being busy, the request being too heavy, or the client not handling reconnection properly. Unlike generic HTTP timeouts, Substrate's RPC layer has specific behaviors: WebSocket subscriptions can go stale without error, and heavy state queries can exceed time limits on public endpoints.

The most common scenario is a polkadot.js ApiPromise using WsProvider that silently disconnects and never reconnects, leaving your application hanging. This happens because the default WsProvider does not throw on timeout; it just emits a 'disconnected' event that many developers miss. Understanding these mechanics is the first step to building reliable queries.

• HTTP RPC timeouts are straightforward: the request fails if no response within the timeout.
• WebSocket subscriptions can become stale without any error, especially if the node restarts or the connection drops.
• Heavy calls like state_getPairs or state_getKeysPaged can time out on public endpoints due to rate limits or resource constraints.
• Block-completeness timeouts occur when the client waits for a finalized block that lags behind the best block.

How Substrate and polkadot.js Handle Timeouts Differently

Substrate's RPC layer is built on JSON-RPC over HTTP or WebSocket. For HTTP, the server has a default timeout (often 60 seconds) after which it returns an error. For WebSocket, the connection is persistent, and requests are matched by ID. If a request takes too long, the server may not respond, and the client's timeout mechanism kicks in.

polkadot.js's WsProvider has a built-in timeout of 60 seconds for individual RPC calls. When a timeout occurs, it does not throw an error by default; instead, it emits a 'disconnected' event and attempts to reconnect. However, if the connection is lost, the provider may not automatically resubscribe to existing subscriptions, leading to silent staleness. This is a documented behavior in the polkadot.js API documentation.

The difference between HTTP and WebSocket timeouts is crucial: HTTP requests are one-shot, so a timeout is a clear failure. WebSocket subscriptions are long-lived, so a timeout on a subscription means the subscription is dead, but the client may not know until it tries to use it.

• HTTP: request fails with an error if no response within timeout.
• WebSocket: request may hang, and the provider may disconnect without throwing.
• Subscriptions: chain_newHead and chain_subscribeFinalizedHeads can go stale after a reconnect.
• polkadot.js default timeout is 60 seconds, but it can be configured.

Why WsProvider Disconnects Silently (and How to Catch It)

The Substrate Stack Exchange question 'Why is WsProvider timeout error not caught in try-catch' highlights a common pitfall: wrapping an RPC call in try-catch does not catch timeouts because the error is emitted as an event, not thrown. The WsProvider emits a 'disconnected' event when the connection is lost, and a 'error' event for protocol errors. If you don't listen to these events, your application will hang.

To handle this, you must attach event listeners to the provider and implement a reconnection strategy. The provider has a built-in auto-reconnect, but it does not resubscribe to previous subscriptions. You need to manually resubscribe after a reconnect.

Here is a minimal example of how to listen for disconnects and errors:

• Listen to 'disconnected' and 'error' events on the provider.
• Use a flag to track connection state.
• After reconnect, resubscribe to all active subscriptions.
• Consider using a library like @polkadot/api-contract that handles reconnection internally.

const { ApiPromise, WsProvider } = require('@polkadot/api');

const provider = new WsProvider('wss://rpc.polkadot.io');
provider.on('disconnected', () => {
  console.log('Provider disconnected');
  // Set a flag to trigger resubscription
});
provider.on('error', (err) => {
  console.error('Provider error:', err);
});

const api = await ApiPromise.create({ provider });
// ... your code

Block RPC Limits: Finalized vs Best Head Lag

Polkadot nodes expose two block-related RPCs: chain_getHeader (best head) and chain_getFinalizedHead (finalized head). The finalized head lags behind the best head by a few blocks due to finality. If your application waits for a finalized block that is far behind, you may hit a timeout if the node is syncing or if the network is congested.

Public endpoints often have rate limits on block RPCs to prevent abuse. For example, a provider may limit the number of requests per second. If you exceed this, you may get a timeout or an error. This is documented behavior for many providers, but the exact limits vary. For instance, OnFinality's API service provides dedicated endpoints with higher limits.

To avoid timeouts, use the appropriate RPC for your use case: if you need the latest state, use best head; if you need finality, use finalized head. Also, consider using subscriptions instead of polling to reduce request load.

• chain_getHeader returns the best block header.
• chain_getFinalizedHead returns the last finalized block header.
• Finalized head lags behind best head by a few blocks.
• Public endpoints may rate-limit block RPCs; check provider docs.

Heavy RPC Calls: state_getPairs and Large Archive Scans

Calls like state_getPairs or state_getKeysPaged can be extremely heavy because they iterate over the entire storage. On a large chain like Polkadot, this can take minutes and will almost certainly time out on public endpoints. Even on a dedicated node, such calls can block the RPC thread and cause other requests to time out.

The recommended approach is to use the storage RPCs (state_getStorage) for light reads of specific keys, and to use batching for multiple reads. For iterating over storage, use state_getKeysPaged with a small page size and process pages incrementally, but be aware of rate limits.

Another technique is to use state_queryStorage to query historical storage at a specific block, which can be more efficient than scanning the entire state.

• state_getPairs and state_getKeysPaged are heavy and can time out.
• Use state_getStorage for single key reads.
• Use state_getKeysPaged with pagination for large scans.
• Consider state_queryStorage for historical queries.
• Batch multiple reads using JSON-RPC batch or api.rpc.batch.

Runnable Example: Timeout, Reconnect, and Resubscription Loop

The following Node.js script demonstrates a robust pattern: it creates an ApiPromise with a custom timeout, listens for disconnects, and resubscribes to chain_newHead after a reconnect. It also shows how to handle timeouts on individual calls using Promise.race.

To run it, install @polkadot/api and @polkadot/util-crypto, then execute with Node.js. The script will log new block headers and handle reconnections gracefully.

• Install dependencies: npm install @polkadot/api @polkadot/util-crypto
• Run with: node script.js
• Expected output: logs new block numbers and reconnection messages.

const { ApiPromise, WsProvider } = require('@polkadot/api');

const WS_URL = 'wss://rpc.polkadot.io';
const TIMEOUT = 60000; // 60 seconds

async function main() {
  const provider = new WsProvider(WS_URL, false); // autoConnect false to control manually
  provider.on('disconnected', () => {
    console.log('Disconnected, attempting reconnect...');
    // The provider auto-reconnects, but we need to resubscribe
  });
  provider.on('error', (err) => {
    console.error('Provider error:', err);
  });

  const api = await ApiPromise.create({ provider, timeout: TIMEOUT });
  await provider.connect();

  let unsubscribe;
  const subscribe = async () => {
    if (unsubscribe) await unsubscribe();
    unsubscribe = await api.rpc.chain.subscribeNewHeads((header) => {
      console.log(`New block #${header.number}`);
    });
  };

  await subscribe();

  // Example of a timeout on a heavy call
  const heavyCall = api.rpc.state.getPairs('0x');
  const timeoutPromise = new Promise((_, reject) => setTimeout(() => reject(new Error('Timeout')), TIMEOUT));
  try {
    const result = await Promise.race([heavyCall, timeoutPromise]);
    console.log('Heavy call result length:', result.length);
  } catch (err) {
    console.error('Heavy call failed:', err.message);
  }

  // Keep process alive
  process.on('SIGINT', async () => {
    if (unsubscribe) await unsubscribe();
    await api.disconnect();
    process.exit(0);
  });
}

main().catch(console.error);

Common Failures and Fixes

The paritytech/polkadot-sdk issue '[rpc] Polkadot assethub rpc's randomly stop providing...' describes a scenario where RPC endpoints on Asset Hub stop responding intermittently. This is often due to node resource exhaustion or network issues. The fix involves monitoring node health, implementing retries, and using multiple endpoints.

Another common failure is the 'Error while running the RPC call' from Substrate Stack Exchange, which often occurs when a call exceeds the server's timeout. The fix is to break the call into smaller chunks or use a different RPC method.

For WebSocket subscriptions, a common failure is the subscription not receiving updates after a reconnect. The fix is to resubscribe on the 'connected' event, as shown in the example.

• Intermittent RPC stops: use multiple endpoints and failover.
• Heavy call timeouts: use pagination or lighter RPCs.
• Subscription staleness: resubscribe on reconnect.
• Rate limiting: respect provider limits and use batching.
• Node sync issues: check node health and use a reliable provider.

Tradeoffs and Limitations

While setting a custom timeout and implementing reconnection logic improves reliability, it adds complexity. A shorter timeout may cause false positives on slow networks, while a longer timeout may delay error detection. The optimal timeout depends on your use case and network conditions.

Batching requests reduces the number of round trips but can increase the load on the node. Public endpoints may have limits on batch size. Always check the provider's documentation for specific limits.

Using a dedicated node or a premium RPC service like OnFinality's API service can provide more consistent performance and higher rate limits, but it comes at a cost. The tradeoff is between reliability and expense.

• Custom timeouts: balance between false positives and delayed errors.
• Batching: reduces round trips but may hit batch limits.
• Public vs premium endpoints: reliability vs cost.
• WebSocket vs HTTP: WebSocket is better for subscriptions, HTTP for one-off queries.

Next Steps: Build Reliable Polkadot Queries

To go deeper, explore the Polkadot WebSocket RPC tutorial for subscription patterns, and check the Polkadot RPC endpoints for a list of public and premium endpoints. If you're building production applications, consider using OnFinality's API service for dedicated endpoints and RPC pricing for cost-effective plans.

Remember to always test your timeout and reconnection logic under real network conditions. Use the OnFinality Learn hub for more tutorials on Polkadot and other chains. For a broader understanding of Polkadot, see the Polkadot network overview.

• Review the Polkadot WebSocket RPC tutorial for subscription best practices.
• Compare endpoints on the Polkadot RPC endpoints page.
• Consider a dedicated endpoint via the API service.
• Check RPC pricing for cost-effective options.
• Explore more on the OnFinality Learn hub.