Rate Limiting Algorithms Explained: Token Bucket, Sliding Window, and More

Rate limiting algorithms decide whether a request is allowed based on how many requests a client has made recently. The four you will see most are token bucket, leaky bucket, fixed window counter, and sliding window (log or counter). Token bucket is the most common default because it allows short bursts while enforcing an average rate, and sliding window counters are a popular choice when you want smooth, accurate limits with little memory.

This guide explains how each algorithm works, where it breaks, and how to run a rate limiter across many servers.

Why rate limiting matters

A rate limiter protects a system from being overwhelmed, whether the cause is a buggy client in a retry loop, a scraper, a credential-stuffing attack, or a sudden traffic spike. It also enforces fairness between tenants and lets you offer tiered API plans. The rate limiting guide covers the broader design space.

Every algorithm needs three decisions:

• Key: what you limit by, such as user ID, API key, IP address, or route.
• Limit: how many requests per unit of time.
• Response: what happens when the limit is hit, usually HTTP 429 Too Many Requests with a Retry-After header.

How does the token bucket algorithm work?

Picture a bucket that holds up to capacity tokens. Tokens are added at a steady refillRate. Each request takes one token. If the bucket is empty, the request is rejected.

This gives you two knobs: the average rate (refill speed) and the maximum burst (capacity). A client that has been idle can spend a full bucket at once, then settles to the refill rate.

class TokenBucket {
  private tokens: number;
  private lastRefill: number;

  constructor(
    private readonly capacity: number,
    private readonly refillPerSecond: number,
  ) {
    this.tokens = capacity;
    this.lastRefill = Date.now();
  }

  tryConsume(cost = 1): boolean {
    const now = Date.now();
    const elapsedSeconds = (now - this.lastRefill) / 1000;
    this.tokens = Math.min(
      this.capacity,
      this.tokens + elapsedSeconds * this.refillPerSecond,
    );
    this.lastRefill = now;

    if (this.tokens >= cost) {
      this.tokens -= cost;
      return true;
    }
    return false;
  }
}

// Example: average 5 requests per second, bursts up to 20
const limiter = new TokenBucket(20, 5);
console.log(limiter.tryConsume()); // true

Notice there is no background timer. Tokens are computed lazily from elapsed time whenever a request arrives, which keeps the state to two numbers per key.

How does the leaky bucket algorithm work?

The leaky bucket treats incoming requests as water poured into a bucket with a hole in the bottom. Requests queue in the bucket and leave at a constant rate. If the bucket overflows, new requests are dropped.

The key difference from token bucket is output shape. Token bucket allows bursts through; leaky bucket smooths them into a steady stream. That makes leaky bucket a good fit when a downstream system, such as a legacy service or a third-party API, needs a constant, predictable request rate. The downside is added latency, since requests may wait in the queue.

Fixed window counter

Divide time into fixed windows, for example each minute, and keep a counter per key per window. Increment on each request; reject once the counter exceeds the limit.

It is simple and cheap, and it maps nicely onto a key-value store with expiry. The weakness is the boundary problem. With a limit of 100 per minute, a client can send 100 requests at 12:00:59 and another 100 at 12:01:00, which is 200 requests in about one second while never breaking the rule.

Sliding window log

Store a timestamp for every accepted request. On each new request, drop timestamps older than the window, count what remains, and allow the request only if the count is below the limit.

This is perfectly accurate, with no boundary spikes. The cost is memory: you store one entry per request per key, which becomes expensive for high limits or many clients.

Sliding window counter

The sliding window counter blends the fixed window's efficiency with the sliding log's smoothness. Keep counters for the current and previous fixed windows, then estimate the count in the rolling window by weighting the previous window by how much of it still overlaps.

For example, with a one-minute window, if we are 25 percent into the current minute, the estimate is:

estimated = current_count + previous_count * 0.75

It assumes requests in the previous window were evenly spread, so it is an approximation, but in practice it removes most of the boundary spike while using only two counters per key.

Rate limiting algorithms compared

Distributed rate limiting across servers

A limiter in each server's memory only works if every client always hits the same server. Behind a load balancer, that is rarely true, so limits need shared state.

Centralized store

The common approach keeps counters or buckets in a fast shared store such as Redis. To avoid race conditions between reading and writing a counter, do the check-and-update atomically, for example with INCR plus EXPIRE for fixed windows, or a server-side script for token buckets.

Where to enforce limits

• At the edge or API gateway, to reject abusive traffic before it reaches your services.
• In the service, for business-specific limits such as "5 password resets per hour".
• At outbound calls, to respect limits imposed by third-party APIs.

Handling failures

Decide in advance what happens if the shared store is unavailable. Failing open keeps the product working but removes protection; failing closed protects backends but blocks users. Many teams fail open with a conservative local fallback limit. For the object-oriented design of a limiter class, see rate limiter design.

Key takeaways

• Token bucket is the usual default: it enforces an average rate and allows controlled bursts.
• Leaky bucket smooths traffic into a constant rate, at the cost of queueing delay.
• Fixed windows are cheap but allow double bursts at window boundaries.
• Sliding window counters approximate exact limits using only two counters per key.
• Distributed limiters need atomic updates in a shared store and a clear failure policy.
• Always return 429 with a Retry-After header so well-behaved clients can back off.

Frequently asked questions

What is the best rate limiting algorithm?

There is no single best choice, but token bucket is the most widely used because it is simple, memory-efficient, and burst-friendly. Use sliding window counters when boundary spikes matter, and leaky bucket when a downstream needs a steady rate.

What is the difference between token bucket and leaky bucket?

Token bucket limits the average rate while letting bursts through up to the bucket capacity. Leaky bucket queues requests and releases them at a fixed rate, so its output never bursts.

What HTTP status code should a rate limiter return?

Return 429 Too Many Requests. Include a Retry-After header, and optionally headers describing the limit and remaining quota, so clients know when to try again.

How do you rate limit across multiple servers?

Store rate limit state in a shared, low-latency store such as Redis and update it atomically. Enforcing limits at the API gateway also centralizes the logic before traffic fans out to services.