Solana getProgramAccounts: dataSlice, Filters, and Safe Pagination

Why getProgramAccounts Is a Full Account Scan

The official Solana getProgramAccounts method reference describes getProgramAccounts as returning all accounts owned by a program, optionally filtered by dataSize or memcmp criteria. Unlike an index lookup, the node walks its account index for the program and evaluates each candidate against the supplied filters. That design is what makes the method powerful for discovery and expensive for production: the work scales with the number of accounts owned by the program, not with the size of the result you asked for.

Because the scan happens on the RPC node, the practical failure mode is rarely a protocol-level JSON-RPC error. It is usually a provider-side response cap, a request timeout, or a request killed for cost. These behaviors are documented / varies by provider, so the same call can succeed on one endpoint and be truncated or rejected on another. Treat the method as a discovery primitive, not a high-frequency query, and design your scanner to be resumable from the first request.

The account model itself is described in the Solana accounts documentation: every account has an owner, lamports, executable flag, rent epoch, and a data byte array. getProgramAccounts returns that data array, which is why filters and dataSlice operate on bytes rather than on named fields. For a broader orientation to Solana RPC surfaces, see the Solana networks overview and the OnFinality Learn hub.

• The node scans accounts owned by the program; cost scales with program account count.
• Provider caps and cost-based termination are documented / varies by provider, not JSON-RPC errors.
• Filters narrow the set; they do not paginate it.
• dataSlice trims returned bytes; it does not reduce the number of accounts returned.

The RpcResponse Context Slot and Why Every Page Must Record It

A getProgramAccounts response is wrapped in the standard RpcResponse envelope, which includes a context object with a slot. That slot identifies the bank the node used to answer the request. Because accounts change between requests, two pages of the same logical scan can be answered at different slots, and an account created or closed in between may appear on one page and not the other.

Recording the context slot on every page is therefore not bookkeeping; it is the only way to reason about consistency after the fact. If your scan spans many requests, store the slot alongside the last key you processed. When you later reconcile the scan against a second source, you can compare counts at approximately the same slot rather than across an unbounded window.

The commitment option controls which bank the node answers from. A weaker commitment returns faster but can be rolled back; a stronger commitment is more stable but may lag. The method reference documents commitment as a request parameter, and the exact default and supported levels are documented / varies by provider. For a related discussion of slot-level consistency, see Solana getBlocks and skipped slots.

• Every page carries a context slot; store it with the last key.
• Accounts can change between pages, so a scan is a sequence of snapshots, not one snapshot.
• Commitment affects which bank answers; defaults and supported levels vary by provider.

Filter Semantics: dataSize, memcmp, and AND Composition

The Solana account model documentation defines how account data is laid out and owned, and the method reference defines two filter kinds. A dataSize filter matches the length of the account data array in bytes. A memcmp filter compares a byte string at a given offset within the account data. Both are evaluated against the full account data on the node, before any dataSlice trimming is applied.

Filters are ANDed. Adding a filter narrows the candidate set; it never pages it. This is the single most common misreading of getProgramAccounts: developers add a memcmp filter expecting the next page and instead receive a smaller first page. If your filter set is too broad, you get a large response; if it is too narrow, you get a small one. Neither outcome is pagination.

A practical consequence is that filter design is a selectivity exercise. A dataSize filter is cheap and often highly selective for fixed-layout accounts. A memcmp filter on a discriminator or a known field is more precise but requires you to know the byte offset. Combining both is common: dataSize to select the account type, memcmp to select a subset within that type.

• dataSize matches the account data length in bytes.
• memcmp compares a byte string at a given offset in the account data.
• Filters are ANDed; adding one narrows the set rather than advancing a cursor.
• Filters are evaluated against full account data, before dataSlice.

memcmp Offsets and the Account Struct Layout Contract

A memcmp offset is a byte offset into the bincode-serialized account data. It is not a field name and not a logical index. The offset therefore depends on the exact account struct layout produced by the program. For Anchor programs, the first eight bytes are the account discriminator, so a field that appears first in the Rust struct begins at offset 8, not 0.

This makes any offset computed from a struct definition a versioned contract. If the program later adds a field before the one you filter on, or changes a field type, your offset silently points at the wrong bytes. The filter still executes; it just matches the wrong data. Treat offsets as part of your integration surface and pin them to a program version.

The Solana accounts documentation describes account data as an opaque byte array owned by the program, which is why the RPC layer can only offer byte-level filters. For a single-account read that avoids this problem entirely, see Reading Solana account info and rent.

• memcmp offset is a byte offset into bincode-serialized account data.
• Anchor accounts begin with an 8-byte discriminator, so the first struct field starts at offset 8.
• An offset derived from a struct definition is a versioned contract that breaks when the layout changes.
• Pin offsets to a program version and re-verify after program upgrades.

dataSlice as a Bandwidth Control, Not a Pagination Mechanism

The method reference documents dataSlice as an optional offset and length that returns only that slice of each account's data. It reduces bytes on the wire. It does not reduce the number of accounts returned. A scan that returns ten thousand accounts still returns ten thousand accounts with dataSlice; each one is simply smaller.

This distinction matters because dataSlice alone never makes a large scan cheap. The node still scans and still serializes a response containing every matching account. What dataSlice changes is the payload size per account, which can move a response from over a provider cap to under it, but it cannot move a response from ten thousand accounts to a hundred.

There is also a usability trap: if you slice away the field you need to sort or resume by, the result set becomes unusable for pagination. Choose the slice to cover exactly the key field you intend to page on, and keep that field in every response.

• dataSlice returns offset..offset+length of each account's data.
• It reduces bytes on the wire but not the number of accounts returned.
• Slicing away your sort or resume key makes the result set unusable.
• Choose the slice to cover the key field you page on.

How dataSlice Interacts with Filters

Filters are evaluated against the full account data on the node. dataSlice is applied only to the returned payload. This ordering means a filter on a field you also sliced away is legal: the node can match on bytes it will not return. You can filter on a discriminator at offset 0 and return only bytes 8 through 40, for example.

The practical implication is that you can keep responses small while still using precise filters. The risk is that you lose the ability to verify the match locally, because the matched bytes are not in the response. If you need to audit filter correctness, temporarily widen the slice or run a separate verification query.

This separation is also why dataSlice cannot be used to implement pagination. Pagination requires a stable ordering key in the response; dataSlice only controls which bytes of each account you see. For a related pattern on a different method, see Solana getSignaturesForAddress pagination.

• Filters run against full account data; dataSlice trims only the returned payload.
• Filtering on a field you sliced away is legal.
• Widen the slice temporarily if you need to audit filter correctness.
• dataSlice cannot implement pagination because it does not order results.

Designing a Resumable Scan on a Stable Key

Solana does not expose a numeric offset or cursor for getProgramAccounts. Any pagination scheme must therefore be built on a field inside the account data. The workable pattern is to page on a monotonically increasing on-chain field, record the last key seen, and resume from it. Each request filters for accounts whose key is greater than the last key, sorts the results locally, and advances the cursor to the maximum key in the batch.

This is a keyset pagination pattern adapted to a method that has no native cursor. It is deterministic as long as the key field is unique and monotonic. If the key is not unique, you need a tiebreaker, and if it is not monotonic, the scan can miss or duplicate accounts. The context slot from each response tells you which snapshot the batch came from.

For a broader treatment of querying historical state over RPC, see Querying Solana historical data over RPC. The same discipline of recording slot and cursor applies there.

• Solana exposes no numeric offset or cursor for getProgramAccounts.
• Page on a monotonically increasing on-chain field and resume from the last key.
• Sort each batch locally and advance the cursor to the maximum key.
• Record the context slot with the last key for later reconciliation.

Runnable Node.js Scanner with dataSize, memcmp, and dataSlice

The scanner below issues getProgramAccounts with a dataSize filter, a memcmp filter on a discriminator, and a dataSlice covering only the key field. It emits a per-batch summary and can be re-run from a recorded last key. Replace the program ID, discriminator, and offsets with values for your program.

The script uses the standard JSON-RPC 2.0 request shape and reads the context slot from each response. It does not assume any provider-specific extension. If your provider offers a getProgramAccountsV2-style endpoint, the same cursor logic applies, but the request shape is documented / varies by provider.

// scan.js — resumable getProgramAccounts scanner
// Usage: node scan.js [lastKeyBase58]
const RPC_URL = process.env.RPC_URL || 'https://api.mainnet-beta.solana.com';
const PROGRAM_ID = process.env.PROGRAM_ID; // your program id
const DATA_SIZE = Number(process.env.DATA_SIZE || 165);
const KEY_OFFSET = Number(process.env.KEY_OFFSET || 8);
const KEY_LENGTH = Number(process.env.KEY_LENGTH || 32);
const DISCRIMINATOR_B58 = process.env.DISCRIMINATOR_B58; // optional

async function rpc(method, params) {
  const res = await fetch(RPC_URL, {
    method: 'POST',
    headers: { 'content-type': 'application/json' },
    body: JSON.stringify({ jsonrpc: '2.0', id: 1, method, params }),
  });
  const json = await res.json();
  if (json.error) throw new Error(JSON.stringify(json.error));
  return json.result;
}

async function scan(lastKey) {
  const filters = [{ dataSize: DATA_SIZE }];
  if (DISCRIMINATOR_B58) {
    filters.push({ memcmp: { offset: 0, bytes: DISCRIMINATOR_B58 } });
  }
  const result = await rpc('getProgramAccounts', [
    PROGRAM_ID,
    {
      encoding: 'base64',
      commitment: 'confirmed',
      withContext: true,
      filters,
      dataSlice: { offset: KEY_OFFSET, length: KEY_LENGTH },
    },
  ]);
  const slot = result.context.slot;
  const accounts = result.value;
  const keys = accounts.map((a) => Buffer.from(a.account.data[0], 'base64'));
  keys.sort(Buffer.compare);
  const last = keys.length ? keys[keys.length - 1].toString('hex') : lastKey;
  const bytes = accounts.reduce((n, a) => n + Buffer.from(a.account.data[0], 'base64').length, 0);
  console.log(JSON.stringify({
    filter: filters,
    accountsReturned: accounts.length,
    bytesReturned: bytes,
    contextSlot: slot,
    lastKey: last,
  }));
  return last;
}

scan(process.argv[2]).catch((e) => { console.error(e); process.exit(1); });

Verifying Scan Completeness Against a Second Source

A scan is only useful if you can argue it is complete. The cheapest cross-check is a second source that reports the same account count: a provider indexing product, a getProgramAccountsV2-style endpoint where offered, or a separate RPC endpoint queried at a nearby slot. Compare counts and investigate any gap before trusting the scan.

Because accounts change between requests, exact equality is not always achievable. Record the context slot of each batch and compare counts at approximately the same slot. If the second source reports a materially different count, the likely causes are a filter that is too narrow, a cursor that skipped a range, or a provider cap that truncated a response.

For endpoint selection and failover behavior, see the RPC endpoints guide (RPC Assistant). Running the same scan against two endpoints is a practical way to detect provider-specific truncation.

• Cross-check the account count against a second source.
• Compare counts at approximately the same context slot.
• Investigate gaps before trusting the scan.
• Run the same scan against two endpoints to detect provider-specific truncation.

Results Table: Measuring Your Own Endpoint

Provider behavior is documented / varies by provider, so the only reliable numbers are the ones you measure against your own endpoint. Run the scanner with a fixed filter set and record the following columns for each batch. Do not compare your numbers to published figures; compare them to your own baseline over time.

The table below is a template. Fill it with values from your endpoint and keep it with the scan output. If a batch returns zero accounts or a truncated payload, note the context slot and the filter used so you can reproduce the condition.

• Batch index
• Filter used (dataSize, memcmp offset/bytes)
• Accounts returned
• Bytes returned
• Context slot
• Last key
• Wall-clock duration
• Provider response status or error

Limitations and Tradeoffs of getProgramAccounts Pagination

The honest limitation is that a program whose accounts have no monotonic field cannot be fully paginated over getProgramAccounts at all. Without a stable ordering key, there is no cursor to resume from, and any offset-like scheme will miss or duplicate accounts as the set changes. In that case the reader must use a provider indexing product or a getProgramAccountsV2-style endpoint where the provider offers one, which is documented / varies by provider.

Even with a monotonic key, the scan is a sequence of snapshots rather than a single consistent view. Accounts created or closed between batches can be missed or double-counted. The context slot lets you reason about this after the fact, but it does not eliminate it. If you need a consistent snapshot, you need an indexing product that maintains one.

Finally, cost and rate limits are real constraints. A full scan is expensive on the node, and providers may cap response size or terminate requests for cost. Design your scanner to be resumable, to record its cursor, and to tolerate partial batches. For pricing and service context, see RPC pricing and the API service.

• No monotonic field means no full pagination over getProgramAccounts.
• A scan is a sequence of snapshots, not one consistent view.
• Provider caps and cost-based termination are documented / varies by provider.
• Design for resumability and partial batches.

Troubleshooting Common getProgramAccounts Failures

The most common failure is a response that is smaller than expected. Check whether a filter is too narrow, whether the memcmp offset points at the wrong bytes after a program upgrade, or whether the provider truncated the response. The context slot and the filter set in your batch summary are the first things to inspect.

The second common failure is a request that is rejected or killed. This is usually a provider-side cap or cost control, not a JSON-RPC protocol error. Reduce the result set with a more selective filter, shrink the dataSlice, or switch to an endpoint with different limits. The behavior is documented / varies by provider.

The third common failure is a scan that appears to loop or skip. This usually means the cursor key is not unique or not monotonic. Add a tiebreaker, verify the key field is actually increasing, and confirm that the dataSlice still includes the key field. For a related pagination pattern, see Solana getSignaturesForAddress pagination.

• Smaller-than-expected response: check filter selectivity, memcmp offset, and provider truncation.
• Rejected or killed request: provider cap or cost control, documented / varies by provider.
• Looping or skipping scan: cursor key is not unique or not monotonic.
• Confirm the dataSlice still includes the key field.

Next Steps for Production Account Enumeration

If your program has a monotonic key, the scanner in this article is a workable starting point. Add persistence for the last key and context slot, schedule the scan, and cross-check counts against a second source. If your program lacks a monotonic key, evaluate a provider indexing product or a getProgramAccountsV2-style endpoint before building a custom workaround.

For endpoint selection and failover, review the RPC endpoints guide (RPC Assistant). For broader Solana RPC coverage, see the Solana networks overview and the OnFinality Learn hub. For pricing and service details, see RPC pricing and the API service.

Authoritative primary sources for the semantics described here are the Solana getProgramAccounts method reference at https://solana.com/docs/rpc/http/getprogramaccounts and the Solana accounts documentation at https://solana.com/docs/core/accounts. Provider-specific limits and extensions should always be confirmed against your provider's current documentation.

• Persist last key and context slot; schedule and cross-check the scan.
• Evaluate indexing products if no monotonic key exists.
• Confirm provider limits and extensions against current documentation.