API Rate Limiting Explained

What Is API Rate Limiting?

API rate limiting is a technique that controls how many requests a client can make to an API within a defined time window. At its core, it is a counter: every time a client makes a request, a counter increments. When the counter reaches the limit, the API rejects further requests until the window resets. This simple mechanism protects your infrastructure from being overwhelmed, prevents any single client from monopolizing shared resources, and provides the foundation for usage-based billing.

Rate limiting serves three distinct purposes simultaneously. From a security perspective, it limits the blast radius of DDoS attacks, brute-force attempts on authentication endpoints, and scraping attacks — even if an attacker controls thousands of IPs, per-IP limits contain the damage per source. From a fairness perspective, it ensures that one customer with a runaway script cannot degrade the API experience for thousands of other customers on the same infrastructure. From a business perspective, it provides the enforcement mechanism for usage tiers — free users get 1,000 requests/day, paid users get 100,000, enterprise gets custom limits — creating a natural upgrade path.

Without rate limiting, a single misconfigured client — not even a malicious one — can bring down your entire API. A developer accidentally running a script in an infinite loop, a mobile app with a bug that retries every 100ms, or a batch job that parallelizes 500 simultaneous requests can exhaust your database connection pool, spike CPU, and create cascading failures affecting every other user on the platform. Rate limiting is not optional infrastructure — it is the circuit breaker that prevents these scenarios from becoming production incidents.

The challenge is implementing rate limiting in a way that is fair, transparent, and developer-friendly. Heavy-handed rate limiting that blocks legitimate traffic and provides no useful feedback creates a terrible developer experience. Well-designed rate limiting — with clear headers, informative error responses, and reasonable limits — is a feature rather than a constraint. Developers appreciate knowing their quota status and being able to design their applications to stay within limits proactively.

Rate Limiting Algorithms

Four main algorithms implement rate limiting, each with different trade-offs in memory usage, burst handling, and implementation complexity. Choosing the right one depends on your traffic patterns, infrastructure capabilities, and how strictly you need to enforce limits.

Token Bucket is the most widely used algorithm for REST APIs. Tokens are added to a bucket at a fixed rate — say, 1.67 tokens per second for a 100-requests-per-minute limit. Each request consumes one token. When the bucket is full, excess tokens are discarded (the bucket has a maximum capacity). When the bucket is empty, the next request is rejected. The key advantage is that a client who has been idle accumulates tokens and can make several requests quickly — a legitimate mobile app that wakes from the background and needs to sync data. Token bucket is the right default for most APIs.

Sliding Window counts requests in a moving time range. For a 60-requests-per-minute limit with sliding window, the server counts how many requests the client made in the last 60 seconds from the current moment, not from the last minute boundary. This eliminates the boundary exploit where a client sends requests at 11:59:59 and 12:00:00 to get double quota. The downside is memory: you must store a timestamp for each request in the window, which can become significant at high request rates or long window durations.

Leaky Bucket processes requests at a constant rate, queuing excess requests. Unlike token bucket, it does not allow bursts — all requests flow out at the same fixed rate. This makes it suitable for traffic shaping to a downstream service that cannot handle bursts: you want requests to arrive at exactly 50/minute, not 100 in the first second and 0 for the next 59 seconds. The risk is queue buildup: if clients send requests faster than the leak rate indefinitely, the queue grows and requests are eventually dropped.

Fixed Window is the simplest: count requests in the current hour, reset the counter at the top of each hour. It is O(1) in time and space, trivially implemented with a Redis INCR and EXPIRE. Its weakness is the boundary exploit — a determined client can double their effective quota by making maximum requests at 11:59:59 and again at 12:00:00. For most public APIs where the threat model is misconfigured clients rather than adversarial exploitation, fixed window is acceptable. For security-sensitive endpoints (authentication, financial operations), always use sliding window or token bucket.

Rate Limiting Strategies

Rate limits can be applied at different granularities. Most production APIs combine multiple strategies simultaneously — per-IP, per-user, per-API-key, and global — because each catches a different attack vector that the others miss.

The recommended layered approach is to enforce all three simultaneously: a global limit that prevents server overload regardless of source, a per-IP limit that contains anonymous traffic and catches distributed attacks from a single botnet, and a per-API-key limit that enforces individual user quotas and supports tiered pricing. A user with a valid Pro API key gets 10,000 requests/hour while an anonymous IP gets 100 requests/hour — they operate independently, and hitting one limit does not change the other.

Per-endpoint limits deserve special attention for sensitive operations. Your search endpoint might allow 100 requests/minute, but your login endpoint should allow only 5 attempts/minute per IP — regardless of whether the user has an authenticated API key. SMS OTP endpoints, password reset endpoints, and financial transaction endpoints should all have their own aggressively conservative limits that operate independently from the global API quota. This prevents credential-stuffing attacks that exploit normal-looking traffic volumes at auth endpoints.

Rate Limit HTTP Headers

Rate limit information must be communicated to clients via HTTP response headers on every request, not just on 429 responses. This allows well-behaved clients to monitor their remaining quota and slow down proactively before hitting the limit — resulting in a better experience for your users and less 429 traffic for your servers.

The IETF has published a draft standard (draft-ietf-httpapi-ratelimit-headers) that defines RateLimit-Limit, RateLimit-Remaining, and RateLimit-Reset as the canonical header names. The older X-RateLimit-* variants (with the X- prefix) are vendor-specific extensions — they were introduced before the standard existed and are used by GitHub, Stripe, Twilio, and most major APIs. New APIs should implement the IETF standard headers and provide the X- variants as aliases for backwards compatibility with existing client SDKs.

When a 429 response is returned, always include Retry-After in seconds, telling the client exactly how long to wait. A response of Retry-After: 30 means the client should wait 30 seconds before its next request. Some APIs set this to the time until the window resets; others use a fixed backoff suggestion. Either is acceptable, but be consistent. Clients that respect Retry-After create dramatically less unnecessary traffic than clients that poll on a fixed interval.

Rate Limits in Real APIs

Understanding how major APIs implement rate limiting gives you benchmarks for designing your own limits and helps you build integrations that respect quotas. The approaches vary significantly in granularity, header conventions, and enforcement strategies.

GitHub's rate limiting is one of the most well-designed in the industry: 5,000 requests/hour for authenticated requests, with the X-RateLimit-Remaining header on every response so clients always know their status. GitHub also uses a separate search API limit (30 requests/minute) and GraphQL point-based limits. Their secondary rate limits (triggered by high concurrency rather than request counts) use 429 with a Retry-After delay. GitHub's documentation is the gold standard for communicating rate limit policies to developers.

OpenAI's rate limiting is model-dependent and multi-dimensional: it limits both requests per minute (RPM) and tokens per minute (TPM). A request that processes a short prompt counts as 1 request but only a few hundred tokens; a request with a large context window counts as 1 request but tens of thousands of tokens. OpenAI returns both x-ratelimit-limit-requests and x-ratelimit-limit-tokens, allowing clients to track which dimension is the binding constraint. This multi-dimensional approach is appropriate whenever a single request has variable resource cost.

Stripe's 100 requests/second limit in live mode is generous for most applications, but Stripe is interesting because it returns error.type: "rate_limit_error" in the JSON body in addition to the 429 status code. Their official recommendation is exponential backoff with initial delay of 1 second. Stripe's test mode has lower limits, which is worth noting during development and load testing — your test environment behavior may not reflect what live mode can handle.

Implementation Patterns

Rate limiting can be implemented at different layers of your architecture, and the right choice depends on your infrastructure scale, latency requirements, and deployment model.

Redis-Based Rate Limiting

Redis is the de facto standard for distributed rate limiting because it provides atomic operations, sub-millisecond latency, and native TTL support. The core pattern uses two Redis commands: INCR (atomic increment) and EXPIRE (set TTL on a key). The implementation is elegant and fast.

For a fixed window implementation, the Redis key encodes the client identifier and the current time window: rate:user:123:minute:202605211430 — user 123, minute window starting at 14:30 UTC on May 21, 2026. On each request, you call INCR on this key. If the key didn't exist before, also call EXPIRE with 60 seconds to ensure it cleans up automatically. If the returned value is greater than the limit, return 429. This entire operation should be wrapped in a Lua script that executes atomically on Redis to avoid race conditions between INCR and EXPIRE.

For sliding window in Redis, you use a sorted set. Each request adds a member with the current Unix timestamp as both member and score: ZADD rate:user:123 1716278400.123 1716278400.123. Before counting, remove all members older than the window: ZREMRANGEBYSCORE rate:user:123 0 (now-window). Then count remaining members with ZCARD. This approach stores every request timestamp and gives exact sliding window semantics, but uses more memory for high-volume APIs.

For token bucket in Redis, you store two values per client: the current token count and the last refill timestamp. On each request, calculate how many tokens have been added since the last refill (elapsed_time × refill_rate), add them to the stored count up to the bucket maximum, subtract one for the current request, and store the updated values. Because this involves read-modify-write, it must be wrapped in a Redis transaction (MULTI/EXEC) or Lua script to prevent race conditions in concurrent environments.

The key operational concern with Redis-backed rate limiting is what happens when Redis is unavailable. You have two choices: fail open (allow all requests when Redis is down, accepting that limits aren't enforced temporarily) or fail closed (reject all requests when Redis is down, accepting availability impact). For most APIs, fail open is correct — a short Redis outage should not cause a user-facing outage. For security-critical endpoints like authentication, fail closed is safer. Whichever you choose, alert immediately on Redis connectivity failures so the issue is resolved before it becomes significant.

Client-Side Rate Limit Handling

Well-designed API clients are as important as well-designed rate limiting. A client that doesn't handle 429 responses gracefully can amplify the problem: if 1,000 clients all hit the limit simultaneously and all retry after exactly the same delay, they create a synchronized spike that immediately exhausts the quota again. This is the thundering herd problem.

Exponential backoff is the standard solution. On the first 429, wait 1 second before retrying. On the second consecutive 429, wait 2 seconds. On the third, 4 seconds. On the fourth, 8 seconds, up to a maximum cap (typically 64 seconds or the Retry-After value, whichever is larger). The doubling ensures that a cluster of clients naturally spreads out — some retry sooner and succeed, reducing load, allowing later retriers to succeed with remaining quota.

Jitter enhances exponential backoff by adding a random variation to the delay. Instead of exactly 4 seconds, you wait between 2 and 6 seconds (4 seconds ± 50%). This prevents clients that started simultaneously from becoming synchronized again after backoff. The two common jitter strategies are full jitter (delay = random(0, base_delay)) and equal jitter (delay = base_delay/2 + random(0, base_delay/2)). AWS published research showing equal jitter provides the best balance of reduced load and reasonable latency distribution.

Proactive rate limiting is even better than reacting to 429s. Read the RateLimit-Remaining header on every response and slow down requests as the remaining count decreases. If remaining drops below 10% of the quota, reduce your request rate to spread the remaining quota over the full reset window. This avoids hitting the limit entirely, meaning no requests are ever rejected — the best user experience possible.

For batch processing jobs that need to make many API calls, implement a request queue with a rate limiter on the client side. Instead of sending all requests in parallel and relying on the server to reject the overflow, pre-limit your own send rate to stay within the quota. A token bucket on the client side, seeded with the API's quota, naturally paces requests at or below the limit without any 429 responses. This approach is significantly more efficient for batch jobs than the retry-on-rejection pattern.

Common Scenarios and Solutions

Rate limiting problems in production follow predictable patterns. Recognizing the scenario quickly leads you to the right solution.

Debugging Rate Limit Issues

When rate limiting behaves unexpectedly, systematic diagnosis beats guessing. Most issues fall into one of these patterns.

Monitoring and Alerting

Proactive monitoring detects rate limiting issues before they impact users. These are the metrics that matter most in production.

Best Practices

Common Mistakes to Avoid

Worked Examples

Frequently Asked Questions

Related Concepts

Related Tools