eth_getLogs Block Range Limits and Safe Chunking

What eth_getLogs Arguments Mean for Range Queries

The Ethereum JSON-RPC specification defines eth_getLogs as a method that returns an array of log objects matching a filter. The filter object accepts fromBlock, toBlock, address, topics, and blockHash. The range is inclusive: fromBlock and toBlock are both included in the scan. The specification states that the filter is evaluated against a block range, and that a blockHash filter is mutually exclusive with fromBlock and toBlock. If you supply blockHash, the node ignores any range parameters and returns logs only for that single block. The Ethereum JSON-RPC specification documents these filter and block range semantics, and the blockHash option itself was introduced by EIP-234.

The special string latest is a moving target. It resolves to the current head at the moment the node processes the request. For reproducible historical scans, never use latest as a boundary. Instead, resolve the current head once, store that block number, and use it as the fixed toBlock for the entire scan. This makes the scan deterministic and allows you to reconcile against a known block hash later.

An over-large range is not a specification error. The JSON-RPC 2.0 specification defines error codes in the -32700 to -32600 range for parse and invalid request errors, but leaves the -32000 to -32099 range for implementation-defined server errors. Providers use this space to signal that a range exceeds their policy. Some return a numeric code such as -32005 or -32000 with a message like 'query returned more than 10000 results' or 'block range too large'. Others reject at the HTTP layer with a 400 or 413 status before the JSON-RPC layer is reached. This is why the same scan that works against one endpoint fails against another with a different message and frequently the same code.

• fromBlock and toBlock are inclusive; both endpoints are scanned.
• blockHash is mutually exclusive with fromBlock and toBlock.
• latest is a moving target and unsuitable for reproducible historical scans.
• Range caps are provider policy, not protocol errors, and surface in the implementation-defined -32000 range or as HTTP rejections.

Why the Cap Is a Cost Limit, Not a Block Count

The per-request cap is not a fixed block count. It is a cost limit. The node must execute the filter over every block in the range and return every matching log. The cost is proportional to the number of blocks scanned plus the number of logs returned. A scan with a narrow address and topic filter produces fewer matching logs and therefore lower cost, even over a wide range. A scan with a broad filter produces more logs and hits the cap sooner.

This has a practical consequence: a narrow filter can cover a far wider range than a wide filter. If you are indexing a single contract with a specific event signature, you may be able to scan tens of thousands of blocks in one request. If you are scanning all logs for all addresses, you may be limited to a few hundred blocks. The cap is documented or varies by provider, and it can change without notice. Never hard-code a block count as a universal constant.

The cost model also explains why timeouts occur. A node that is already under load may take longer to execute a wide-range query and return a timeout or 5xx error even if the range is within the provider's documented cap. This is a transient failure, not a capable failure. Distinguishing the two is essential for a correct backoff strategy.

• The cap is a cost limit based on blocks scanned and logs returned.
• Narrow filters survive wider ranges than broad filters.
• Timeouts and 5xx errors are transient; range errors are deterministic.

Adaptive Chunking Algorithm with Explicit Stop Conditions

The goal is a portable algorithm that survives any provider's undocumented cap. Start from a configured maximum span, execute the query, and classify the outcome. The outcome categories are: success, range-too-large, timeout or 5xx, and rate limited. On a range failure, halve the span and retry the same cursor position. Continue until the span reaches one block. If a single-block query still fails with a range error, record a failure and advance the cursor by one block to avoid an infinite loop.

On a timeout or 5xx, do not halve the span. The request is capable but the node was slow. Retry the same span after a backoff interval. On a rate limit (HTTP 429), respect the Retry-After header if present, or use exponential backoff. Do not reduce the span for transient errors, because that would unnecessarily increase the number of requests and worsen the rate limit.

The algorithm must have an explicit stop condition and an explicit failure record. A common mistake is to retry indefinitely on any error. That turns a deterministic range error into an infinite loop. Instead, after a configurable number of retries for transient errors, record the chunk as failed and move on. The failure record should include the fromBlock, toBlock, span, error code, and error message so you can diagnose the endpoint's behavior later.

async function adaptiveScan({ endpoint, address, topics, fromBlock, toBlock, maxSpan, maxRetries }) {
  const results = [];
  const failures = [];
  let cursor = fromBlock;
  let span = maxSpan;

  while (cursor <= toBlock) {
    const chunkTo = Math.min(cursor + span - 1, toBlock);
    let attempt = 0;
    let success = false;

    while (attempt <= maxRetries) {
      try {
        const logs = await rpcCall(endpoint, 'eth_getLogs', [{
          fromBlock: '0x' + cursor.toString(16),
          toBlock: '0x' + chunkTo.toString(16),
          address,
          topics
        }]);
        results.push(...logs);
        success = true;
        break;
      } catch (err) {
        const code = err.code;
        const message = (err.message || '').toLowerCase();
        const isRangeError = code === -32005 || code === -32000 || message.includes('range') || message.includes('too large') || message.includes('more than');
        const isRateLimit = err.status === 429;
        const isTransient = err.status >= 500 || message.includes('timeout') || isRateLimit;

        if (isRangeError && span > 1) {
          span = Math.max(1, Math.floor(span / 2));
          attempt = 0;
          break;
        }
        if (isTransient && attempt < maxRetries) {
          const delay = isRateLimit && err.retryAfter ? err.retryAfter * 1000 : Math.pow(2, attempt) * 1000;
          await new Promise(r => setTimeout(r, delay));
          attempt++;
          continue;
        }
        failures.push({ fromBlock: cursor, toBlock: chunkTo, span, code, message: err.message });
        success = true;
        break;
      }
    }

    if (!success) {
      failures.push({ fromBlock: cursor, toBlock: chunkTo, span, code: 'unknown', message: 'retries exhausted' });
    }
    cursor = chunkTo + 1;
  }

  return { results, failures };
}

Distinguishing Capable Failures from Transient Failures

A capable failure is deterministic. If you send the same fromBlock, toBlock, address, and topics to the same endpoint, you will get the same range error every time. A transient failure is non-deterministic. A timeout or 429 may succeed on the next attempt with the same parameters. The backoff strategy must not be applied to a capable failure, because retrying an impossible request wastes time and may trigger rate limits.

To classify an error, inspect both the JSON-RPC error code and the HTTP status. Range errors typically arrive as JSON-RPC errors with codes in the -32000 range. Timeouts and 5xx errors arrive as HTTP errors or as JSON-RPC errors with messages containing 'timeout' or 'gateway'. Rate limits arrive as HTTP 429 with a Retry-After header. If the error is ambiguous, log the full response and test the same parameters again. If the error reproduces, treat it as capable.

The practical rule: on a range error, halve the span and retry the same cursor. On a timeout or 5xx, retry the same span after backoff. On a 429, respect Retry-After and retry the same span. Never halve the span for a transient error, because that increases request count and can worsen the rate limit.

• Capable failure: reproduces deterministically with the same parameters.
• Transient failure: timeout, 5xx, or 429; may succeed on retry.
• Halve span only for range errors; backoff for transient errors.

Completeness Guarantees and Why They Are Not Automatic

A chunked scan does not automatically produce a complete, ordered, deduplicated log set. Two adjacent chunks can legitimately return the same log when the cursor overlaps. This happens if you advance the cursor by the chunk size instead of chunkTo + 1, or if the provider returns logs from a block that was re-scanned. You must deduplicate by the tuple (blockNumber, logIndex, transactionIndex) and sort by the same tuple to produce a deterministic order.

A chain reorganisation can invalidate a chunk that was already committed to storage. If a reorg removes a block that contained a log, your stored log is now orphaned. The reconciliation method must detect this. One approach is to store the highest processed block per address/topic pair and re-scan a configurable confirmation depth behind the tip on every pass. Another is to verify that total log counts are monotonically increasing and that no log references a block that is no longer canonical.

Completeness also requires that you never skip a block. If a chunk fails and you advance the cursor without recording the failure, you have a gap. The failure record is not optional. It is the evidence that a gap exists and the starting point for a targeted re-scan. For a deeper treatment of reconciliation, see Block-by-block EVM indexer reconciliation.

• Deduplicate by (blockNumber, logIndex, transactionIndex).
• Sort by the same tuple for deterministic ordering.
• Reorgs can invalidate committed chunks; re-scan a confirmation depth behind the tip.
• Never advance the cursor past a failed chunk without recording the failure.

Reconciliation Method to Prove the Scan Is Complete

Reconciliation is the process of proving that your stored logs match what the chain actually contains. It is not a one-time check. It must run on every pass. The method has four parts: re-run a small sample of chunks and compare, store the highest processed block per address/topic pair, verify total log counts monotonically, and re-scan a configurable confirmation depth behind the tip.

Re-running a sample of chunks is the strongest check. Pick a random set of previously processed chunks, re-execute the same eth_getLogs query, and compare the returned logs to what you stored. If the sets differ, you have a gap or an orphaned log. The sample size can be small, but it should be non-zero on every pass. For a broader discussion of reconciliation patterns, see Block-by-block EVM indexer reconciliation.

Storing the highest processed block per address/topic pair lets you resume without re-scanning everything. It also lets you detect if a reorg has moved the tip backward. If the current head is lower than your stored highest processed block, a reorg has occurred and you must re-scan from the new head minus the confirmation depth. Verifying that total log counts are monotonically increasing catches accidental deletions. Re-scanning a confirmation depth behind the tip catches reorgs that are deeper than one block.

• Re-run a random sample of chunks and compare results.
• Store the highest processed block per address/topic pair.
• Verify total log counts are monotonically increasing.
• Re-scan a configurable confirmation depth behind the tip on every pass.

Runnable Node.js Example for Adaptive Chunked Scans

The following Node.js script performs an adaptive chunked scan against an endpoint URL passed as an argument. It prints per-chunk output including fromBlock, toBlock, span, status, logCount, and elapsedMs. It also asserts that the merged log array is strictly ordered and free of duplicates. Run it with node scan.js <endpoint> <fromBlock> <toBlock> <address> <topic0>.

The script uses the adaptiveScan function from the previous section. It adds a merge step that deduplicates by (blockNumber, logIndex, transactionIndex) and sorts by the same tuple. The assertion at the end throws if the merged array is not strictly ordered or contains duplicates. This gives you a reproducible measurement of your endpoint's behavior.

Because the cap varies by provider and by chain, you should run this script against each endpoint you use. The per-chunk output is your evidence. Record the span at which the first range error occurs, the error code, and the error message. That is your endpoint's effective cap for that filter selectivity.

const https = require('https');

function rpcCall(endpoint, method, params) {
  return new Promise((resolve, reject) => {
    const url = new URL(endpoint);
    const body = JSON.stringify({ jsonrpc: '2.0', id: 1, method, params });
    const req = https.request({
      hostname: url.hostname,
      port: url.port || 443,
      path: url.pathname,
      method: 'POST',
      headers: { 'Content-Type': 'application/json', 'Content-Length': Buffer.byteLength(body) }
    }, res => {
      let data = '';
      res.on('data', chunk => data += chunk);
      res.on('end', () => {
        try {
          const json = JSON.parse(data);
          if (json.error) {
            const err = new Error(json.error.message);
            err.code = json.error.code;
            err.status = res.statusCode;
            reject(err);
          } else {
            resolve(json.result);
          }
        } catch (e) {
          const err = new Error('parse error: ' + data.slice(0, 200));
          err.status = res.statusCode;
          reject(err);
        }
      });
    });
    req.on('error', reject);
    req.write(body);
    req.end();
  });
}

async function adaptiveScan({ endpoint, address, topics, fromBlock, toBlock, maxSpan, maxRetries }) {
  const results = [];
  const failures = [];
  let cursor = fromBlock;
  let span = maxSpan;

  while (cursor <= toBlock) {
    const chunkTo = Math.min(cursor + span - 1, toBlock);
    let attempt = 0;
    let success = false;
    const start = Date.now();

    while (attempt <= maxRetries) {
      try {
        const logs = await rpcCall(endpoint, 'eth_getLogs', [{
          fromBlock: '0x' + cursor.toString(16),
          toBlock: '0x' + chunkTo.toString(16),
          address,
          topics
        }]);
        results.push(...logs);
        console.log(`from=${cursor} to=${chunkTo} span=${span} status=ok logs=${logs.length} ms=${Date.now() - start}`);
        success = true;
        break;
      } catch (err) {
        const code = err.code;
        const message = (err.message || '').toLowerCase();
        const isRangeError = code === -32005 || code === -32000 || message.includes('range') || message.includes('too large') || message.includes('more than');
        const isRateLimit = err.status === 429;
        const isTransient = err.status >= 500 || message.includes('timeout') || isRateLimit;

        if (isRangeError && span > 1) {
          console.log(`from=${cursor} to=${chunkTo} span=${span} status=range_error code=${code} msg=${err.message}`);
          span = Math.max(1, Math.floor(span / 2));
          attempt = 0;
          break;
        }
        if (isTransient && attempt < maxRetries) {
          const delay = isRateLimit && err.retryAfter ? err.retryAfter * 1000 : Math.pow(2, attempt) * 1000;
          console.log(`from=${cursor} to=${chunkTo} span=${span} status=transient code=${code} retry=${attempt + 1} delay=${delay}ms`);
          await new Promise(r => setTimeout(r, delay));
          attempt++;
          continue;
        }
        console.log(`from=${cursor} to=${chunkTo} span=${span} status=failed code=${code} msg=${err.message}`);
        failures.push({ fromBlock: cursor, toBlock: chunkTo, span, code, message: err.message });
        success = true;
        break;
      }
    }

    if (!success) {
      failures.push({ fromBlock: cursor, toBlock: chunkTo, span, code: 'unknown', message: 'retries exhausted' });
    }
    cursor = chunkTo + 1;
  }

  return { results, failures };
}

function mergeLogs(logs) {
  const seen = new Set();
  const merged = [];
  for (const log of logs) {
    const key = `${log.blockNumber}:${log.logIndex}:${log.transactionIndex}`;
    if (!seen.has(key)) {
      seen.add(key);
      merged.push(log);
    }
  }
  merged.sort((a, b) => {
    const bn = parseInt(a.blockNumber, 16) - parseInt(b.blockNumber, 16);
    if (bn !== 0) return bn;
    const li = parseInt(a.logIndex, 16) - parseInt(b.logIndex, 16);
    if (li !== 0) return li;
    return parseInt(a.transactionIndex, 16) - parseInt(b.transactionIndex, 16);
  });
  return merged;
}

(async () => {
  const [endpoint, fromBlock, toBlock, address, topic0] = process.argv.slice(2);
  const { results, failures } = await adaptiveScan({
    endpoint,
    address,
    topics: [topic0],
    fromBlock: parseInt(fromBlock),
    toBlock: parseInt(toBlock),
    maxSpan: 10000,
    maxRetries: 3
  });
  const merged = mergeLogs(results);
  console.log(`total logs=${results.length} merged=${merged.length} failures=${failures.length}`);
  for (let i = 1; i < merged.length; i++) {
    const prev = merged[i - 1];
    const curr = merged[i];
    const prevKey = `${prev.blockNumber}:${prev.logIndex}:${prev.transactionIndex}`;
    const currKey = `${curr.blockNumber}:${curr.logIndex}:${curr.transactionIndex}`;
    if (prevKey >= currKey) throw new Error('ordering violation at index ' + i);
  }
  console.log('ordering and deduplication assertions passed');
})();

Results Table for Measuring Your Own Endpoint

Because the cap varies by provider and by chain, you must measure it yourself. Use the script above to produce a results table. Run it against each endpoint you use, with the same address and topic filter, and record the span at which the first range error occurs. Also record the error code and message. This table becomes your reference for configuring maxSpan per endpoint.

The table should have columns for endpoint, chain, address, topic0, maxSpan attempted, first failing span, error code, error message, and notes. Fill it in with your own measurements. Do not rely on published numbers from other providers, because the cap can change without notice and differs by filter selectivity.

If you use multiple endpoints for redundancy, measure each one separately. A scan that works against one endpoint may fail against another with a different message and frequently the same code. Persist the measured maxSpan per endpoint so your scanner can adapt without manual intervention.

• Endpoint: the RPC URL you are testing.
• Chain: the network name or chain ID.
• Address and topic0: the filter selectivity used for the test.
• MaxSpan attempted: the starting span in your configuration.
• First failing span: the span at which the first range error occurred.
• Error code and message: the exact values returned by the endpoint.
• Notes: any observed rate limits, timeouts, or provider-specific behavior.

| Endpoint | Chain | Address | topic0 | maxSpan attempted | First failing span | Error code | Error message | HTTP status | Elapsed ms |
|---|---|---|---|---|---|---|---|---|---|
| ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ |
| ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ |
| ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ |
| ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ | ____ |

Integration with Production Pipelines

Chunking interacts with the rest of a production pipeline in three ways. First, chunk results should be committed per chunk so a crash does not lose the whole scan. If you buffer all logs in memory and commit at the end, a crash at 90% loses everything. Commit each chunk's logs and its cursor position in the same transaction. Second, parallelism must be bounded to avoid triggering rate limits. Do not exceed the endpoint's documented concurrency. If the endpoint does not document a concurrency limit, start with one or two parallel requests and increase only after measuring.

Third, chunk size should be persisted per endpoint because the cap differs between providers. Store the measured maxSpan alongside the endpoint URL. When the scanner starts, read the persisted value and use it as the starting span. If a range error occurs, halve the span and update the persisted value. This makes the scanner self-tuning over time.

For a broader discussion of endpoint selection and concurrency, see the RPC endpoints guide (RPC Assistant). For timeout-specific troubleshooting, see How to fix RPC timeout errors.

• Commit each chunk's logs and cursor position in the same transaction.
• Bound parallelism to the endpoint's documented concurrency.
• Persist the measured maxSpan per endpoint and update it on range errors.

Troubleshooting Common Chunking Failures

The most common failure is an infinite retry loop on a deterministic range error. If your scanner keeps halving the span but never reaches one block, check that your stop condition is span > 1 and that you advance the cursor after a failure. Another common failure is a gap caused by advancing the cursor past a failed chunk. Always record the failure and re-scan it later.

A second failure mode is ordering violations after merging. This happens when you sort by blockNumber only and ignore logIndex and transactionIndex. Two logs in the same block can have the same blockNumber but different logIndex values. Sort by the full tuple. A third failure mode is duplicate logs from overlapping chunks. If you advance the cursor by chunkTo + 1, you should not see duplicates, but providers can return logs from a block that was re-scanned. Deduplicate by the full tuple.

A fourth failure mode is rate limiting caused by unbounded parallelism. If you see HTTP 429 responses, reduce parallelism and respect Retry-After. Do not halve the span for a 429, because that increases request count and worsens the rate limit. For more on timeout and rate limit handling, see How to fix RPC timeout errors.

• Infinite retry loop: check stop condition and cursor advancement.
• Gaps: record failures and re-scan them later.
• Ordering violations: sort by (blockNumber, logIndex, transactionIndex).
• Duplicates: deduplicate by the full tuple.
• Rate limits: reduce parallelism and respect Retry-After.

Limitations and Tradeoffs

The cap varies by provider and by chain, is often undocumented, and can change without notice. A chunking algorithm that works today may need adjustment tomorrow. Persisting the measured maxSpan per endpoint helps, but it is not a guarantee. You must monitor for new error codes and messages.

Block-bounded scanning cannot see logs from blocks that were reorged out. If a log was emitted in a block that is no longer canonical, your scan will not return it. This is correct behavior for a canonical scan, but it means your stored logs can become stale after a reorg. The reconciliation method addresses this by re-scanning a confirmation depth behind the tip.

A chunked scan is not atomic. It reads different blocks at different times. If you need a point-in-time snapshot, you must reconcile against a fixed block hash. Use eth_getBlockByNumber to resolve the head block hash at the start of the scan, and use that hash for any blockHash-based queries. For range queries, store the head block number and use it as the fixed toBlock. This gives you a consistent view of the chain at that block.

For BSC-specific range limits and large scans, see BSC eth_getLogs range limits and large scans. For event and topic filtering, see eth_getLogs event and topic filtering.

• The cap varies by provider and chain, is often undocumented, and can change.
• Block-bounded scanning cannot see reorged-out logs.
• A chunked scan is not atomic; reconcile against a fixed block hash for snapshots.

Next Steps for Production Log Scans

Start by measuring your endpoint's effective cap with the script above. Fill in the results table for each endpoint you use. Then configure your scanner with the measured maxSpan and implement the adaptive chunking algorithm with explicit stop conditions and failure records. Add the reconciliation method to every pass.

If you are evaluating providers, compare their documented concurrency limits and error behavior. The RPC pricing page and the API service page describe OnFinality's offerings. For a general overview of EVM networks, see Ethereum network. For more integration and development guides, see the OnFinality Learn hub.

Finally, treat the cap as a moving target. Re-measure periodically and after any provider announcement. Persist the measured values and update them automatically when a range error occurs. This keeps your scanner resilient without manual intervention.

• Measure your endpoint's cap and fill in the results table.
• Implement adaptive chunking with explicit stop conditions and failure records.
• Add reconciliation to every pass.
• Re-measure periodically and persist measured values per endpoint.