LLM API rate limits: how to handle 429 errors from OpenAI, Claude and Gemini in your backend

LLM API rate limits are the first production problem most AI features hit. The demo works, the launch goes well, and then a burst of users or one enthusiastic power user pushes your backend over a limit you never saw in testing. The provider answers with HTTP 429, your extension shows a spinner that never ends, and the first one-star review mentions "doesn't work anymore."

This guide explains how LLM API rate limits work at OpenAI, Anthropic (Claude) and Google (Gemini), how to tell a real rate limit from a billing problem that happens to share the same status code, and how to build a small Node.js layer that retries correctly, slows down before it is blocked, and keeps one user from draining the quota for everyone else.

Key takeaways

• A 429 is not always a rate limit. All three providers also use it for spend caps or exhausted credit, and retrying those never helps. Read the error body before you retry.
• Honor the retry-after header when it is present, add jitter, and cap both the number of attempts and the total wait. Turn off SDK retries if you add your own, or the two loops multiply.
• Limits are counted in requests and tokens per minute, per organization or project, and usually per model. Gemini limits are per project, "not per API key", so a second key does not add capacity.
• Prevent 429s instead of only reacting: limit concurrency, keep max_tokens realistic on OpenAI, cache repeated prompt prefixes on Claude, and move background work to the batch APIs.
• In a browser extension, add your own per-user quota in front of the provider and return a clear message to the UI, so a single heavy user cannot take the feature down for everyone.

How LLM API rate limits work#

Every provider measures traffic on several axes at once and blocks you on whichever runs out first. OpenAI's rate limits guide lists requests per minute (RPM), requests per day (RPD), tokens per minute (TPM), tokens per day (TPD) and images per minute, and notes that "rate limits can be hit across any of the options depending on what occurs first." Anthropic's Claude rate limits page splits tokens into input tokens per minute (ITPM) and output tokens per minute (OTPM) on top of RPM. Google's Gemini rate limits page uses RPM, input TPM and RPD.

Two details matter more than the tier tables. First, limits are enforced over short windows. Anthropic warns that "a rate of 60 requests per minute (RPM) might be enforced as 1 request per second", so a burst of 20 parallel calls can fail even when your per-minute average is low. Second, failed calls are not free. OpenAI states that "unsuccessful requests contribute to your per-minute limit," which is why a tight retry loop makes a rate limit worse.

Not every 429 is a rate limit#

Search any developer forum for "429" and the most common thread is not about traffic at all. The OpenAI Developer Community has years of posts titled some variation of 429 - You exceeded your current quota, usually from someone who made a handful of requests on a new account with no credit. The status code is the same as a rate limit; the fix is completely different.

Each provider gives you a way to tell them apart:

• OpenAI. The error codes page lists several 429 variants. "Rate limit reached for requests" and the slow_down code mean you are going too fast. Codes such as credit_balance_exhausted, organization_spend_limit_exceeded, project_spend_limit_exceeded and organization_usage_limit_exceeded mean money or an assigned limit, and OpenAI says to identify billing errors by error.code, since error.type can still read insufficient_quota.
• Anthropic. When an organization reaches its tier's monthly spend cap, the API returns a 429 with type rate_limit_error, but "the response has no retry-after header" and error.details.error_code is enforced_spend_limit_reached. A spend limit you set yourself returns a 400 instead.
• Gemini. Spend-based limits on paid tiers return 429 RESOURCE_EXHAUSTED, the same status Google's troubleshooting guide tells you to retry with exponential backoff. Check the message and your billing page before assuming it is traffic.

A simple rule covers most cases: a 429 with a retry-after header is a rate limit; a 429 without one is suspect. Log the full error body, classify it, and only retry what can succeed later.

Read the rate limit headers#

Successful responses tell you how close you are to the limit, which is far more useful than finding out from a 429. OpenAI returns x-ratelimit-limit-requests, x-ratelimit-remaining-requests, x-ratelimit-reset-requests and the same three for tokens, with resets written as durations such as 1s or 6m0s. Anthropic returns anthropic-ratelimit-requests-remaining, anthropic-ratelimit-input-tokens-remaining, anthropic-ratelimit-output-tokens-remaining and matching -reset headers as RFC 3339 timestamps. Both send retry-after in seconds on a rate-limited response.

This helper normalizes the parts you need for logging and pacing:

// rate-limit-headers.mjs
// Turns OpenAI-style durations ("6m0s", "1.5s", "250ms") into milliseconds.
function parseDuration(value) {
  if (!value) return null;
  let ms = 0;
  for (const [, num, unit] of value.matchAll(/([\d.]+)(ms|h|m|s)/g)) {
    ms += Number(num) * { ms: 1, s: 1000, m: 60_000, h: 3_600_000 }[unit];
  }
  return ms;
}

export function readRateLimit(headers) {
  const retryAfter = headers.get('retry-after');
  return {
    retryAfterMs: retryAfter ? Number(retryAfter) * 1000 : null,
    // OpenAI header first, then the Anthropic equivalent.
    remainingRequests: Number(headers.get('x-ratelimit-remaining-requests') ??
      headers.get('anthropic-ratelimit-requests-remaining') ?? NaN),
    remainingTokens: Number(headers.get('x-ratelimit-remaining-tokens') ??
      headers.get('anthropic-ratelimit-input-tokens-remaining') ?? NaN),
    resetTokensMs: parseDuration(headers.get('x-ratelimit-reset-tokens')) ??
      (headers.get('anthropic-ratelimit-input-tokens-reset')
        ? Date.parse(headers.get('anthropic-ratelimit-input-tokens-reset')) - Date.now()
        : null),
  };
}

If you use the official SDKs, the raw headers are still reachable. The OpenAI Node SDK exposes them through .withResponse(), and the Anthropic SDKs have an equivalent raw-response accessor.

Retry with retry-after, backoff and jitter#

Both OpenAI and Anthropic now give the same advice: treat retry-after as a minimum, add a small random delay, fall back to exponential backoff with jitter when the header is missing, and cap the attempts and total time. Google's guidance for Gemini is to retry only transient errors (429, 408 and 5xx) with backoff and jitter.

Before writing your own loop, know what the SDK already does. The OpenAI Node SDK retries connection errors, 408, 409, 429 and 5xx responses "2 times by default, with a short exponential backoff." Anthropic's SDKs also retry twice by default and honor retry-after. If you wrap an SDK call in your own retry loop, set maxRetries: 0 on the client, or three of your retries become nine requests.

Here is a provider-neutral wrapper around fetch that retries only what can succeed:

// with-retry.mjs
import { readRateLimit } from './rate-limit-headers.mjs';

const RETRYABLE = new Set([408, 409, 429, 500, 502, 503, 504, 529]);
const BILLING_CODES = new Set([
  'insufficient_quota', 'credit_balance_exhausted',
  'organization_spend_limit_exceeded', 'project_spend_limit_exceeded',
  'organization_usage_limit_exceeded', 'enforced_spend_limit_reached',
]);

const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));

export class BillingError extends Error {}

export async function withRetry(doFetch, { maxAttempts = 4, maxTotalMs = 30_000 } = {}) {
  const started = Date.now();
  for (let attempt = 1; ; attempt++) {
    const res = await doFetch();
    if (res.ok) return res;

    const body = await res.clone().json().catch(() => ({}));
    const code = body?.error?.code ?? body?.error?.details?.error_code ?? body?.error?.type;
    if (BILLING_CODES.has(code)) throw new BillingError(`Provider billing limit: ${code}`);
    if (!RETRYABLE.has(res.status) || attempt >= maxAttempts) return res;

    const { retryAfterMs } = readRateLimit(res.headers);
    // Anthropic bodies start with type "error". Their 429 without retry-after
    // is the tier spend cap, not traffic, so retrying cannot succeed.
    if (res.status === 429 && retryAfterMs === null && body?.type === 'error') return res;

    const backoff = Math.min(500 * 2 ** (attempt - 1), 8_000);
    const wait = Math.max(retryAfterMs ?? 0, backoff) + Math.random() * 250;
    if (Date.now() - started + wait > maxTotalMs) return res;
    await sleep(wait);
  }
}

Two choices in this code are deliberate. The total time cap matters more than the attempt count for an interactive feature: a user staring at a popup will not wait two minutes for an answer, so it is better to fail fast with a helpful message. And billing errors throw a distinct error type so your monitoring can page someone, instead of being retried silently until the month rolls over.

Stop 429s before they happen#

Retries handle the occasional spike. If you see 429s every day, your backend is sending more than your tier allows and needs to pace itself.

Limit concurrency#

Most 429s in small backends come from bursts: a page with 30 items that each trigger a summary, or a nightly job that fires every request at once. A concurrency limiter is a few lines and removes most of them:

// limiter.mjs
export function createLimiter(maxConcurrent) {
  let active = 0;
  const queue = [];
  const next = () => {
    if (active >= maxConcurrent || queue.length === 0) return;
    active++;
    const { task, resolve, reject } = queue.shift();
    task().then(resolve, reject).finally(() => { active--; next(); });
  };
  return (task) => new Promise((resolve, reject) => {
    queue.push({ task, resolve, reject });
    next();
  });
}

// Usage: at most 4 provider calls in flight from this process.
// const limit = createLimiter(4);
// const res = await limit(() => withRetry(() => fetch(url, options)));

This only covers one process. If you run several instances, put the counter in Redis or your database, or route all model calls through one worker queue.

Spend tokens deliberately#

Token limits are usually the binding constraint, and each provider counts them differently.

• OpenAI counts a request as "the maximum of max_tokens and the estimated number of tokens" based on its characters. A generous max_tokens on every call reserves capacity you never use, so set it close to the response size you expect.
• Anthropic evaluates output tokens "in real time as output tokens are produced," and max_tokens "does not factor into OTPM rate limit calculations." On input, "for most Claude models, only uncached input tokens count toward your ITPM rate limits." If every request repeats a long system prompt or the same reference document, prompt caching both lowers the bill and raises your effective throughput.
• Gemini limits are per project. Spreading traffic over several API keys in one project does not add capacity.

Also watch for sudden growth. OpenAI's slow_down code and Anthropic's acceleration limits both trigger when traffic jumps sharply, even below your nominal limit. If a launch or a Chrome Web Store feature is about to multiply your traffic, ramp it gradually instead of switching everything on at once.

Move background work to the batch APIs#

Anything that does not need an answer within seconds, such as nightly review summaries, tagging a backlog or re-scoring old data, should not compete with interactive traffic. OpenAI's Batch API has separate queue limits "calculated based on the total number of input tokens queued for a given model," and Anthropic's Message Batches API "has its own set of rate limits which are shared across all models." Moving bulk jobs there frees the per-minute budget for users who are waiting.

Add per-user quotas in an extension backend#

Provider limits apply to your whole organization or project. In a browser extension with thousands of installs, that pool is shared by every user, so one person running a script against your feature can trigger 429s for everyone else. The fix is your own quota in front of the provider, keyed by a user identity your server trusts.

That identity has to come from a real sign-in, not a value the extension makes up, because anyone can read and modify extension code. Our guide to Google sign-in in a Chrome extension shows how to verify the user on your server and issue a session you can key quotas on.

// quota.mjs: a fixed daily quota per user, stored in any key-value store.
const DAILY_LIMIT = 200;

export async function checkQuota(store, userId) {
  const day = new Date().toISOString().slice(0, 10);
  const key = `quota:${userId}:${day}`;
  const used = await store.incr(key); // e.g. Redis INCR
  if (used === 1) await store.expire(key, 60 * 60 * 26);
  if (used > DAILY_LIMIT) {
    const midnight = Date.parse(`${day}T00:00:00Z`) + 86_400_000;
    return { ok: false, retryAfterSec: Math.ceil((midnight - Date.now()) / 1000) };
  }
  return { ok: true };
}

When a user is over their quota, return your own 429 with a retry-after header and a short, honest message the extension can show ("Daily limit reached, resets at midnight UTC"). When the provider limits you, return a 503 with a different message ("Busy right now, try again in a minute"). Users forgive a clear limit far more readily than a spinner, and the difference shows up in your reviews.

On the extension side, keep the UI responsive: show partial results if you stream (see our guide on streaming AI responses in a Chrome extension), offer a retry button, and never retry automatically in a loop from thousands of clients at once.

Monitor rate limits before users notice#

Rate limit problems are easy to see in logs and easy to miss in practice. A few signals catch them early:

1. Remaining capacity. Log remaining tokens and requests from the headers on every call and alert when they stay under about 10 percent for several minutes. That is your cue to request a higher tier or pace traffic.
2. 429 rate by type. Count rate limits, billing errors and overloads separately. A single billing error deserves an immediate alert; a few traffic 429s an hour do not.
3. Fallback usage. If you fall back to a smaller model when the main one is overloaded, track how often. A rising fallback rate is an early warning.
4. Model changes. Limits are per model, and a model retirement can move you to a model with different limits. Our post on LLM model retirements in late 2026 lists the dates to plan around.

Keep alerts readable: one channel for "act now" (billing errors, sustained 429s) and a daily digest for the rest. Our guide to store change alerts your team will actually read applies the same thinking to store data.

Your users will also tell you when the AI feature breaks, just not in your logs. Slow answers and failures show up as new reviews and rating drops within days. Extenify's extension review monitoring records new reviews, rating changes and weekly users for your extension and your competitors on the Chrome Web Store, Edge Add-ons and Firefox Add-ons, and sends a daily alert when something moves. Tracking one extension is free, and the pricing page covers the paid plans. Pair it with our guide to reading review and rating trends to tell a real regression from noise.

Frequently asked questions#

Why do I get a 429 error when I have barely used the API?#

Usually because of billing, not traffic. New accounts with no prepaid credit, an exhausted credit balance, or a monthly spend limit all return 429 at OpenAI, and a tier spend cap returns 429 at Anthropic. Check the error code in the response body and your billing page. Retrying will not fix it.

How long should I wait after a 429?#

At least the number of seconds in the retry-after header, plus a little random jitter. If there is no header, use exponential backoff starting around half a second and cap the total wait. For a user-facing request, give up after a few seconds and show a message instead of waiting indefinitely.

Are LLM API rate limits per API key?#

Generally no. OpenAI limits apply at the organization and project level, Anthropic at the organization level with optional workspace limits, and Gemini says limits are "applied per project, not per API key." Creating more keys in the same project or organization does not add capacity.

Do failed requests count against my rate limit?#

At OpenAI, yes: "unsuccessful requests contribute to your per-minute limit." That is why hammering the API with immediate retries keeps you rate limited. Always back off between attempts.

Does streaming help with rate limits?#

Streaming improves perceived speed, not capacity. The request still counts toward requests per minute and its tokens toward token limits. At Anthropic, output tokens are counted as they are generated, so a stream that is cancelled early only uses the tokens produced up to that point.

Should I retry 529 or 503 overloaded errors?#

Yes, with backoff. Anthropic's 529 overloaded_error and OpenAI's 503 server_is_overloaded are temporary capacity problems on the provider side. If they persist, check the provider's status page and consider falling back to another model or provider for non-critical features.

Summary#

LLM API rate limits are predictable once you read what each provider sends back. Classify every 429 before retrying, honor retry-after, add jitter and a hard time cap, and make sure only one retry loop is active. Then prevent most limits from triggering at all with a concurrency limiter, realistic max_tokens, prompt caching and the batch APIs for background work. In an extension, add a per-user quota tied to a verified sign-in, return clear messages to the UI, and watch both your logs and your reviews for the first signs of trouble.

Image credits#

• Cover photo: Night traffic light trails by oonjirachantanupa on the WordPress Photo Directory, CC0.