Sui Checkpoint Streaming: gRPC Ledger Service and Continuity

What a Sui checkpoint actually commits to

A Sui checkpoint is the unit of data continuity: it commits to a set of transaction effects and is identified by a monotonically increasing sequence number and a digest. Each checkpoint also carries its epoch, and every epoch finalises with a checkpoint that is flagged as the end of that epoch. This makes the checkpoint chain the spine that epochs, objects (id + version), transaction effects and events hang from. The Sui developer documentation describes checkpoints, epochs and the API reference in more detail.

The summary fields and the heavier contents fields are separated in the response shape. The summary includes the sequence number, digest, epoch, end-of-epoch flag, network total transactions, and the list of transaction digests. The contents include the execution and effects data for those transactions, which is why per-checkpoint contents are retrieved separately from the summary.

Because the sequence number is monotonic and the digest is a commitment, you can prove continuity by checking that each checkpoint's sequence number is exactly one greater than the previous one, and that the digest chain is consistent. This is the basis for gap-free indexing. For a broader view of how Sui data is served, see the Sui network page and the OnFinality Learn hub.

• Sequence number: monotonically increasing identifier of the checkpoint.
• Digest: cryptographic commitment to the checkpoint contents.
• Epoch: the epoch this checkpoint belongs to.
• End-of-epoch flag: marks the final checkpoint of an epoch.
• Network total transactions: cumulative count at this checkpoint.
• Transaction digest list: the transactions included in this checkpoint.

The JSON-RPC surface: exact reads vs paged reads

The JSON-RPC surface has a per-checkpoint summary call, sui_getCheckpoint, which takes an exact sequence number or a digest and returns the summary fields plus a separate contents section. This is the right call when you know exactly which checkpoint you need, for example when reconciling a specific sequence number or verifying a digest.

It also has a range/paging call, sui_getCheckpoints (or suix_getCheckpoints depending on the client generation), which takes a cursor and a descending flag and returns a nextCursor. Paginating by cursor in ascending order is safe because page boundaries are sequence boundaries. Walking backwards with descending=true is the way to backfill from the tip.

The distinction matters: sui_getCheckpoint needs an exact sequence number, while sui_getCheckpoints pages over sequence numbers with a cursor. Mixing them up leads to unpaginated 'give me everything since genesis' requests, which are a common failure mode. For related pagination patterns, see Reading Sui objects, dynamic fields and pagination and Querying Sui events with suix_queryEvents.

• sui_getCheckpoint: exact sequence number or digest, returns summary + contents.
• sui_getCheckpoints / suix_getCheckpoints: cursor + descending flag, returns nextCursor.
• Ascending cursor pagination is safe; page boundaries are sequence boundaries.
• descending=true is the backfill path from the tip.

The gRPC ledger service and checkpoint subscription

The gRPC surface exposes the same log as a ledger service with a server-streaming checkpoint subscription that pushes every new checkpoint (with contents) as it is certified. This removes polling latency because you do not have to ask repeatedly; the server pushes each checkpoint as it becomes available.

The trade-off is that the stream couples the consumer to a single connection. If the connection drops, you must reconnect and reconcile. A production indexer typically streams for liveness and reconciles with a cursor read for correctness, using the digest as an idempotency key so a reconnect does not replay duplicates.

The authoritative primary sources for this are the Sui developer documentation, the Sui API Reference, and the sui-apis ledger_service.proto on GitHub, which defines the checkpoint subscription. Anything client- or provider-specific, such as connection limits or retention, is documented / varies by provider. For endpoint selection, see the Sui RPC providers and endpoints (RPC Assistant).

• Server-streaming subscription pushes each certified checkpoint with contents.
• Removes polling latency but couples the consumer to one connection.
• Stream for liveness, reconcile with a cursor read for correctness.
• Use the digest as an idempotency key on reconnect.

Runnable paginating reader with a persisted cursor

The following Node.js example pages forward over checkpoints using a cursor, persists the last processed sequence number to disk, and asserts that the first sequence number of the next page equals the last sequence number of the previous page plus one. It also checks that the union of all seen sequence numbers contains no hole below the tip.

Run it against your own endpoint. The code uses a generic JSON-RPC call; replace the method name with sui_getCheckpoints or suix_getCheckpoints depending on your client generation. The persistence file is a simple JSON file so you can interrupt the process mid-page and restart from the last persisted sequence number without skipping or duplicating.

const fs = require('fs');
const fetch = require('node-fetch');

const RPC_URL = process.env.SUI_RPC_URL || 'https://your-endpoint.example';
const STATE_FILE = './checkpoint-cursor.json';

function loadState() {
  if (fs.existsSync(STATE_FILE)) {
    return JSON.parse(fs.readFileSync(STATE_FILE, 'utf8'));
  }
  return { lastSeq: null, seen: [] };
}

function saveState(state) {
  fs.writeFileSync(STATE_FILE, JSON.stringify(state));
}

async function rpc(method, params) {
  const res = await fetch(RPC_URL, {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ jsonrpc: '2.0', id: 1, method, params })
  });
  const json = await res.json();
  if (json.error) throw new Error(JSON.stringify(json.error));
  return json.result;
}

async function main() {
  const state = loadState();
  let cursor = state.lastSeq ? String(state.lastSeq) : null;
  let descending = false;

  while (true) {
    const page = await rpc('sui_getCheckpoints', [cursor, descending, 50]);
    const data = page.data || [];
    if (data.length === 0) break;

    for (const cp of data) {
      const seq = Number(cp.sequenceNumber);
      if (state.lastSeq !== null && seq !== state.lastSeq + 1) {
        throw new Error(`Gap detected: expected ${state.lastSeq + 1}, got ${seq}`);
      }
      state.lastSeq = seq;
      state.seen.push(seq);
    }

    saveState(state);
    cursor = page.nextCursor;
    if (!cursor) break;
  }

  const sorted = [...state.seen].sort((a, b) => a - b);
  for (let i = 1; i < sorted.length; i++) {
    if (sorted[i] !== sorted[i - 1] + 1) {
      throw new Error(`Hole in seen sequence numbers: ${sorted[i - 1]} -> ${sorted[i]}`);
    }
  }
  console.log('No holes below tip. Last sequence:', state.lastSeq);
}

main().catch((err) => { console.error(err); process.exit(1); });

Resume test: interrupt mid-page and restart

To verify continuity, interrupt the process mid-page and restart. The reader must restart from the last persisted sequence number without skipping or duplicating. Assert that the first sequence number of the next page equals the last sequence number of the previous page plus one, and that the union of all seen sequence numbers contains no hole below the tip.

This test catches the common failure of cursor reuse across an epoch boundary, where a cursor from one epoch may not be valid in the next. It also catches confusing a checkpoint's epoch with its sequence number, and treating the end-of-epoch checkpoint as an ordinary one. The end-of-epoch checkpoint is still a checkpoint with a sequence number, but it carries the flag that closes the epoch.

If you are backfilling from the tip, set descending=true and walk backwards, then reverse the order before persisting. The same continuity assertion applies in reverse: each previous sequence number must be exactly one less than the current one. For retry patterns around timeouts, see Sui RPC timeouts and reliable retry patterns.

• Persist lastSeq after each page, not after each checkpoint, to bound replay.
• On restart, resume from lastSeq + 1 and assert the first seq matches.
• Check the union of seen sequence numbers for holes below the tip.
• Do not reuse a cursor across an epoch boundary.

Results table: measure your own endpoint

Use the table below to record your own endpoint's observed behaviour. Do not rely on vendor-published numbers; measure against your endpoint and fill in the results. This keeps the comparison honest and reproducible.

Run the paginating reader above, then record the page size, the observed nextCursor behaviour, and whether the endpoint returns contents in the paging call or requires a separate sui_getCheckpoint call. Note any rate limiting or retention differences, which are documented / varies by provider.

• Page size used: ____
• First sequence number of page: ____
• Last sequence number of page: ____
• nextCursor returned: ____
• Contents included in paging call: yes / no
• Gap detected: yes / no
• Reconnect replay duplicates: yes / no

Common failures and how to avoid them

Unpaginated 'give me everything since genesis' requests are the most common failure. They overload the endpoint and often time out. Always page with a cursor and a bounded page size.

Cursor reuse across an epoch boundary is another failure. A cursor is tied to the sequence space; when the epoch changes, re-anchor from a known sequence number. Confusing a checkpoint's epoch with its sequence number leads to off-by-epoch errors in analytics.

Treating the end-of-epoch checkpoint as an ordinary one can break epoch-boundary logic. Consuming the checkpoint stream without an idempotency key (digest) means a reconnect replays checkpoints, causing duplicates. Querying an exact old sequence number against a pruning endpoint fails because the data is no longer retained; use an archive endpoint for historical reads, as described in Sui archive nodes and historical RPC.

• Avoid unpaginated genesis-to-tip requests.
• Re-anchor cursors at epoch boundaries.
• Do not confuse epoch with sequence number.
• Handle end-of-epoch checkpoints explicitly.
• Use the digest as an idempotency key on stream reconnect.
• Use archive endpoints for old exact sequence numbers.

Troubleshooting checklist

Work through this checklist when your indexer reports a gap or a duplicate. Start with the persisted state file and confirm the last sequence number. Then confirm the next page's first sequence number equals lastSeq + 1.

If you see duplicates, check whether the stream reconnected without an idempotency key. If you see a gap, check whether a page was skipped due to a timeout or a cursor error. If exact old sequence numbers fail, check whether the endpoint prunes and switch to an archive endpoint.

• Confirm persisted lastSeq is present and readable.
• Assert next page first seq == lastSeq + 1.
• Check for duplicate digests after reconnect.
• Check for skipped pages after timeouts.
• Verify endpoint retention for old sequence numbers.
• Confirm epoch boundary handling in your logic.

Limitations, assumptions and trade-offs

This guide assumes you have a Sui JSON-RPC or gRPC endpoint that supports checkpoint reads and, optionally, the ledger service subscription. Provider-specific limits, retention windows, and subscription availability are documented / varies by provider. OnFinality does not assert specific latency, throughput, or rate numbers here.

Streaming gives lower latency but couples you to one connection; polling is simpler but adds latency and load. The recommended pattern is to stream for liveness and reconcile with a cursor read for correctness. This assumes your consumer can persist state durably and can tolerate at-least-once delivery with digest-based deduplication.

The continuity proof relies on the monotonic sequence number and the digest commitment. If your endpoint returns a cursor that skips sequence numbers, treat that as a provider-specific behaviour and reconcile with an exact sui_getCheckpoint read for the missing range.

• Provider limits and retention: documented / varies by provider.
• Streaming: low latency, single-connection coupling.
• Polling: simple, higher latency and load.
• Recommended: stream for liveness, reconcile with cursor read.
• Continuity proof depends on monotonic sequence and digest.

Next steps: build a gap-free Sui indexer

Start by selecting an endpoint that supports the checkpoint methods you need. The Sui RPC providers and endpoints (RPC Assistant) page helps you compare options, and the Sui network page lists network details.

Then implement the paginating reader with a persisted cursor, add the resume test, and wire in the gRPC ledger subscription for liveness. Use the digest as an idempotency key and reconcile with a cursor read on reconnect. For historical backfill, use an archive endpoint as described in Sui archive nodes and historical RPC.

For related reading, see Querying Sui events with suix_queryEvents, Reading Sui objects, dynamic fields and pagination, and Sui RPC timeouts and reliable retry patterns. For service options, see RPC pricing and the API service.

• Choose an endpoint with checkpoint and ledger service support.
• Implement persisted cursor pagination and the resume test.
• Add gRPC streaming for liveness with digest deduplication.
• Reconcile with cursor reads on reconnect.
• Use archive endpoints for historical backfill.