RPC API Key Rotation: Zero-Downtime Credential Lifecycle

What an RPC credential actually is and the shapes it takes

An RPC credential is the secret that authorizes your application to a provider's endpoint. It is not the endpoint hostname; it is the token, key, or project identifier that the provider maps to your account, quota, and billing. The shape varies by provider and is documented / varies by provider, but four forms dominate: a key in the URL path or query string, an Authorization: Bearer header, a JWT with an expiry and scopes, and an IP-allowlisted key.

The URL-embedded form is the most exposure-prone because the credential becomes part of every request line. It lands in reverse-proxy access logs, browser history, HTTP Referer headers, error trackers, and support screenshots. Header-based auth keeps the secret out of the URL, which is why the OAuth 2.0 Bearer Token Usage specification (RFC 6750) defines the Bearer scheme for exactly this purpose: the token travels in the Authorization header, not the request target.

A JWT adds structure. RFC 7519 defines the JSON Web Token format, and providers that issue JWTs typically attach an exp claim and a scope or project claim. That gives you two extra lifecycle levers: the token expires on its own, and the scope limits what a leaked token can do. Not every provider offers scopes or expiry, so treat those as capabilities to verify, not assumptions.

• URL path or query key: simplest to use, hardest to keep secret.
• Authorization: Bearer header: keeps the secret out of logs and referrers.
• JWT: adds exp and scope claims, enabling short-lived credentials.
• IP-allowlisted key: binds the credential to a network origin, which breaks serverless egress without a fixed IP.

The threat model: why an unrotated key is the most common RPC incident

A key in a client bundle or a public repository is effectively public. The moment it is committed, pushed, or shipped to a browser, you must assume an adversary has it. The OWASP API Security Top 10 lists API2: Broken Authentication as a top risk, and credential leakage is a direct path into that category because the attacker does not need to break authentication; they simply reuse a valid credential.

Leakage is rarely a single dramatic event. Keys leak through CI logs that print environment variables, screen shares during incident calls, error trackers that capture full request URLs, and third-party proxies that log upstream requests. Each of these is a copy of your credential in a system you do not control.

The abuse is usually compute, not data theft. A leaked key is used to run queries against your quota, which inflates your bill and can exhaust the capacity your production traffic depends on. In worse cases it becomes a foothold: the attacker can read your project's metrics, observe your traffic patterns, and time further abuse to avoid alerting thresholds.

• Assume any key in a bundle, repo, or log is compromised.
• CI logs, screen shares, error trackers, and proxies are common leak channels.
• Primary abuse is quota consumption and billing inflation, not necessarily data exfiltration.
• A leaked key can expose project metrics and traffic patterns.

The two-key overlap: the only rotation pattern that avoids an outage

Zero-downtime rotation is a sequencing problem, not a tooling problem. The safe order is: provision key B while key A is still serving, deploy the code or config that reads B from a secret store, verify B is receiving traffic, then revoke A only after a drain window. The unsafe order is rotate A first, then deploy B, because between those two steps every request is authenticated with a credential the provider has already invalidated.

The drain window is the interval during which both keys are valid and you are watching traffic migrate from A to B. Its length depends on your deploy topology: a single service may drain in minutes, while a fleet with long-lived connections, caches, or edge nodes may need longer. You cannot pick the window from a blog post; you measure it against your own endpoint.

This pattern composes with the failover design described in RPC node monitoring and failover. Rotation is a credential change, but it rides on the same health-check and traffic-shifting machinery you already use to move between endpoints.

• Provision B, deploy B, verify B, then revoke A.
• Never revoke A before B is confirmed serving traffic.
• The drain window is measured, not assumed.
• Rotation should reuse your existing failover and health-check tooling.

Making the swap atomic with a single indirection point

A rotation is only a config change if the credential is read from one place. If the key is a string literal scattered across services, rotation becomes a code change with a deploy per service, and the probability of missing one rises with every file. The fix is a single indirection point: an environment variable loaded from a secret manager at process start, or a runtime lookup from a secret store.

The indirection point should be the only code that knows the credential's name. Everything else asks for 'the RPC credential' and receives whatever the store currently holds. That way, rotating the key is a write to the store plus a restart or reload, not a search-and-replace across the repository.

Pair the indirection with a secret scanner in CI so a literal never reaches the repository in the first place. The scanner is a backstop, not the primary control; the primary control is that developers never have a reason to paste the key into source.

• One indirection point: env var from a secret manager, or runtime secret-store lookup.
• No credential literals in source, config files, or container images.
• Run a secret scanner in CI as a backstop.
• Per-environment keys so a staging leak cannot touch production.

Coordinating rotation with your endpoint pool and failover

A key belongs to an endpoint. If you run a pool of endpoints for redundancy, as described in Multi-region RPC failover routing, rotation must be coordinated per endpoint: rotate one endpoint's key, verify it, move to the next. A naive global swap of every endpoint at once can trip rate limits or a provider's per-key concurrency, because all your traffic briefly concentrates on the new credential.

The safe cadence is sequential with verification between steps. Rotate endpoint one, confirm its health check passes and its traffic is served by the new key, then rotate endpoint two. If a step fails, you still have the remaining endpoints on the old key, which keeps the service up while you investigate.

This is also where the distinction between Public RPC endpoints vs dedicated matters. Public endpoints may not issue per-project credentials at all, so the rotation problem is specific to dedicated or authenticated endpoints. Confirm which endpoints in your pool actually carry a credential before you plan a rotation.

• Rotate one endpoint at a time, verifying between steps.
• A global simultaneous swap can concentrate traffic and trip limits.
• Keep at least one endpoint on the old key until the new key is proven.
• Confirm which endpoints in your pool are authenticated before planning.

A runnable Node.js example with observable key selection

The example below loads two keys, prefers the new one, falls back to the old one during the overlap window, and logs which key served each request. The log line is the point: it makes the drain observable, so you can watch traffic migrate from A to B before you revoke A.

The code reads both keys from environment variables, which is the single indirection point. In production you would load those variables from a secret manager at process start. The fallback is deliberately simple: try the new key, and if the provider rejects it with an authentication error, retry once with the old key. That retry is what keeps the service up if the new key is misconfigured.

Do not copy the retry logic blindly. Some providers count failed auth attempts against your quota or trigger lockouts, so verify the behavior against your provider's documentation before enabling the fallback in production.

const NEW_KEY = process.env.RPC_KEY_NEW;
const OLD_KEY = process.env.RPC_KEY_OLD;
const ENDPOINT = process.env.RPC_ENDPOINT; // e.g. https://rpc.example.com/<key>

function urlFor(key) {
  return ENDPOINT.replace('{key}', key);
}

async function callRpc(method, params) {
  const body = JSON.stringify({ jsonrpc: '2.0', id: 1, method, params });

  // Prefer the new key; fall back to the old key during the overlap window.
  for (const [label, key] of [['new', NEW_KEY], ['old', OLD_KEY]]) {
    if (!key) continue;
    const res = await fetch(urlFor(key), {
      method: 'POST',
      headers: { 'content-type': 'application/json' },
      body,
    });

    // 401/403 means this key is not accepted; try the next one.
    if (res.status === 401 || res.status === 403) {
      console.warn(`[rpc] key=${label} rejected status=${res.status}`);
      continue;
    }

    // Log which key served the request so the drain is observable.
    console.log(`[rpc] key=${label} status=${res.status} method=${method}`);
    return res.json();
  }

  throw new Error('all RPC keys rejected');
}

callRpc('eth_blockNumber', []).catch((e) => {
  console.error('[rpc] fatal', e.message);
  process.exit(1);
});

Measuring the drain against your own endpoint

You cannot know your drain window from documentation alone. Measure it. The method below is reproducible against any endpoint and produces a table you fill with your own numbers. Run it once per rotation and keep the results; over time they become your rotation budget.

The measurement is simple: after deploying the new key, sample the key-selection log at a fixed interval and record the fraction of requests served by the new key. When the fraction reaches 100 percent and stays there for the length of your longest-lived connection, the drain is complete and A can be revoked.

Record the same table for a rollback drill. If you never practice revoking B and restoring A, you do not know whether your rollback works, and the first time you need it will be during an incident.

• Results Table columns: timestamp, requests sampled, percent served by new key, percent served by old key, auth failures, notes.
• Sample at a fixed interval (for example every 30 seconds) until the new-key fraction is 100 percent.
• Hold at 100 percent for the duration of your longest-lived connection before revoking the old key.
• Repeat the table during a rollback drill to prove the reverse path works.

Detecting a leaked or misused key

Detection is about anomalies, not signatures. Watch for usage or spend that rises without a matching rise in your own traffic, requests originating from IP ranges you do not operate, and a sudden 429 on a key you are not actively using. That last signal is particularly telling: if a key you believe is idle is rate-limited, something else is using it.

The How to fix RPC 429 errors guide covers the rate-limit mechanics, but the security reading is different. A 429 on an idle key is evidence of unauthorized use, not a capacity problem. Treat it as a potential incident until you can attribute the traffic.

Alert on authentication failures as well. A spike in 401 or 403 responses usually means a deploy shipped the wrong key or a rotation was incomplete. Either way, it is a signal that your credential state and your deployed state have diverged.

• Usage or spend anomalies without matching traffic.
• Requests from unexpected IP ranges.
• A 429 on a key you are not actively using.
• A spike in 401/403 responses after a deploy or rotation.

Incident response: revoke first, investigate second

When you suspect a key is leaked, revoke it before you investigate. A revoked key is cheaper than a live one, and the cost of a brief outage during revocation is almost always lower than the cost of continued abuse. Investigation can proceed against the revoked credential's logs without the attacker continuing to consume your quota.

The order is: revoke the suspect key, confirm the revocation took effect, then pull logs and metrics to determine scope. If the key was in a client bundle, assume every copy is public and rotate the entire chain, including any derived credentials.

After the incident, close the leak channel, not just the credential. If the key leaked through CI logs, fix the logging. If it leaked through an error tracker, redact URLs before they are captured. Rotating without fixing the channel guarantees a repeat.

• Revoke the suspect key first; investigate against its logs afterward.
• Confirm revocation took effect before declaring the incident contained.
• Assume every copy of a bundled key is public and rotate the chain.
• Fix the leak channel, not just the credential.

Operational checklist for the credential lifecycle

The checklist below is the minimum viable lifecycle. It assumes a secret store, a CI scanner, and per-environment keys. If any of those are missing, add them before the next rotation, because each one removes a class of leak.

The API service and RPC pricing pages describe how credentials map to plans and quotas, which is useful when you are deciding how many keys to issue and how to scope them. The RPC endpoints guide (RPC Assistant) covers endpoint selection, which is the other half of the credential decision.

• Store credentials in a secret manager, never in source or images.
• Run a secret scanner in CI and block merges on findings.
• Issue per-environment keys so staging cannot touch production.
• Apply least-privilege scopes where the provider supports them.
• Set an expiry on JWTs and rotate before it lapses.
• Alert on authentication failures and on 429s for idle keys.
• Keep a written rotation runbook with the two-key overlap sequence.

Limitations and tradeoffs you cannot engineer away

URL-embedded keys are inherently leakier than header auth. No amount of process discipline changes the fact that a credential in the request target is captured by more systems than one in a header. If your provider supports header auth, prefer it; if it does not, treat the URL key as a higher-risk credential and rotate it more often.

Not every provider offers scopes or expiry. Where they are absent, you cannot limit the blast radius of a leaked key through the credential itself, so you compensate with network controls, alerting, and shorter rotation intervals. This is a real constraint, not a configuration gap you can close.

IP allowlisting breaks serverless egress without a fixed IP. If your workloads run on ephemeral compute, an allowlisted key will fail as soon as the egress address changes. You either pin the egress through a NAT or proxy, or you forgo allowlisting and accept the higher exposure. Choose deliberately, and document the choice.

• URL keys leak through more channels than header keys; rotate them more often.
• Missing scopes or expiry means compensating controls, not a fix.
• IP allowlisting requires a fixed egress address; serverless without one will break.
• Every tradeoff here should be documented, not discovered during an incident.

Troubleshooting rotation failures

Most rotation failures fall into four buckets: the new key is not actually deployed, the old key was revoked too early, the endpoint pool was rotated all at once, or the credential is cached somewhere you forgot. The first three are sequencing errors; the fourth is an indirection failure.

If you see 401 or 403 responses immediately after deploying the new key, check that the secret store write propagated to every instance. A rolling deploy can leave old instances running with the old key, which is fine during the overlap but becomes an outage if you revoke A before the rollout completes.

If you see 429 responses during rotation, you likely concentrated traffic on one key. Slow the rotation down and rotate endpoints sequentially. The RPC endpoints guide (RPC Assistant) and the OnFinality Learn hub both cover endpoint-level behavior that helps you diagnose this.

• 401/403 after deploy: verify the secret propagated to every instance.
• Outage after revoke: you revoked A before the rollout completed.
• 429 during rotation: you concentrated traffic on one key; rotate sequentially.
• Persistent auth failures: a cached credential in a proxy, CDN, or sidecar.

Next steps: build the rotation into your normal operations

The goal is not a heroic rotation during an incident; it is a boring rotation on a schedule. Pick an interval, put it in the runbook, and practice the two-key overlap until it is routine. A rotation you have done ten times is a config change; a rotation you have never done is an outage waiting for a trigger.

Start with one endpoint. Provision a second key, deploy the example code, watch the drain table fill, and revoke the old key. Then extend the pattern to the rest of your pool and to your Base or other network endpoints as needed.

If you are choosing where to run this lifecycle, the API service and RPC pricing pages describe the credential and quota model, and the OnFinality Learn hub collects the related operational guides. The rotation pattern itself is provider-agnostic; only the credential shape changes.

• Schedule rotations; do not wait for an incident.
• Practice the two-key overlap until it is routine.
• Start with one endpoint, then extend to the pool.
• Keep the drain table from every rotation as your rotation budget.