Load balancing algorithms decide which backend server handles each incoming request. The main families are static algorithms like round robin and weighted round robin, dynamic algorithms like least connections and least response time, and hash-based algorithms like IP hash and consistent hashing. The right choice depends on whether your servers are identical, whether requests vary in cost, and whether a request must land on the same server every time.
What does a load balancer do?
A load balancer sits between clients and a pool of servers. It spreads traffic so no single server is overwhelmed, removes unhealthy servers using health checks, and lets you add or remove capacity without clients noticing. It can operate at Layer 4 (TCP/UDP connections) or Layer 7 (HTTP requests, where it can route by path, header, or cookie).
The algorithm is only one part of the picture, but it has an outsized effect on tail latency and on how evenly servers are used. For the broader concepts, see the Load balancing guide.
Static load balancing algorithms
Static algorithms do not look at the current state of servers. They are simple, fast, and predictable.
Round robin
Requests go to servers in order: A, B, C, A, B, C. It works well when servers have similar capacity and requests have similar cost. It breaks down when one request takes a hundred times longer than another, because a server can pile up slow work while still receiving its "fair" share.
Weighted round robin
Each server gets a weight proportional to its capacity. A server with weight 3 receives three requests for every one sent to a server with weight 1. This is useful during migrations, canary releases, or when mixing instance sizes.
Random
Pick a server uniformly at random. Over many requests it evens out like round robin, and it needs no shared counter, which makes it easy to run across many load balancer instances.
Dynamic load balancing algorithms
Dynamic algorithms use live information from servers, such as open connections or response times.
Least connections
Send the request to the server with the fewest active connections. This adapts naturally to requests of different lengths, which makes it a good fit for long-lived connections like WebSockets, database proxies, or streaming. Weighted least connections divides the connection count by a server weight.
Least response time
Choose the server with the lowest combination of active connections and recent latency. It reacts to slow servers quickly, but it needs accurate, fresh measurements, and it can oscillate if every balancer piles onto the same "fastest" server at once.
Power of two choices
Pick two servers at random and send the request to the less loaded one. This sounds too simple to matter, but it avoids the herd behavior of always picking the global minimum and gets load distribution close to least connections with far less coordination. It is a strong default when you run many independent load balancer instances.
import random
class Server:
def __init__(self, name: str):
self.name = name
self.active = 0
def pick_p2c(servers: list[Server]) -> Server:
a, b = random.sample(servers, 2)
return a if a.active <= b.active else b
servers = [Server(n) for n in ("a", "b", "c", "d")]
chosen = pick_p2c(servers)
chosen.active += 1 # decrement when the request finishes
print(chosen.name)
Hash-based load balancing algorithms
Hash-based algorithms send the same key to the same server. That is essential when servers hold state, such as a local cache, a shard of data, or a user session.
IP hash and key hash
Compute hash(client_ip) % N or hash(user_id) % N to pick a server. It gives stickiness without cookies. The problem is the modulo: when N changes from 4 to 5, almost every key maps to a different server. For stateless services that is fine; for caches it means a sudden wave of misses.
Consistent hashing
Consistent hashing places both servers and keys on a ring of hash values. A key belongs to the first server clockwise from its position. When a server joins or leaves, only the keys between it and its neighbor move, roughly 1/N of the total instead of nearly all of them.
To avoid uneven arcs, each physical server is placed on the ring many times as virtual nodes. More virtual nodes give a smoother distribution at the cost of a larger lookup table. Variants such as rendezvous (highest random weight) hashing and bounded-load consistent hashing solve the same problem with different trade-offs. The consistent hashing guide walks through the ring in detail, and it is the same idea behind many database sharding schemes.
Load balancing algorithms compared
| Algorithm | Uses server state? | Sticky? | Handles uneven requests? | Typical use |
|---|---|---|---|---|
| Round robin | No | No | Poorly | Identical stateless servers |
| Weighted round robin | No | No | Poorly | Mixed instance sizes, canaries |
| Random | No | No | Poorly | Many balancers, no shared state |
| Least connections | Yes | No | Well | Long-lived connections |
| Least response time | Yes | No | Well | Latency-sensitive APIs |
| Power of two choices | Yes (sampled) | No | Well | Large, distributed balancer fleets |
| IP / key hash | No | Yes | Poorly | Simple session affinity |
| Consistent hashing | No | Yes | Poorly (unless bounded) | Caches, sharded state |
Health checks, draining, and other practical details
The algorithm only chooses among servers the balancer believes are healthy. A few operational details matter as much as the algorithm:
- Active health checks probe an endpoint on an interval; passive checks mark a server unhealthy after repeated errors.
- Connection draining lets in-flight requests finish before a server is removed during deploys.
- Slow start ramps traffic to a newly added server gradually, so a cold cache or JIT does not get flooded.
- Sticky sessions via cookies are convenient but make scaling and failover harder. Prefer moving session state to a shared store.
- Retries should go to a different server and respect a budget, or they amplify outages.
How to choose a load balancing algorithm
- Stateless servers, similar requests: round robin or random.
- Mixed server sizes: weighted round robin.
- Variable request duration or long-lived connections: least connections or power of two choices.
- Servers hold per-key state such as caches or shards: consistent hashing.
- Many independent balancer instances: prefer random or power of two choices over algorithms that need a global view.
Key takeaways
- Static algorithms are simple and predictable but ignore how busy each server actually is.
- Least connections and power of two choices handle uneven request costs well.
- Plain modulo hashing remaps almost every key when the pool size changes.
- Consistent hashing moves only a small fraction of keys when servers are added or removed.
- Health checks, draining, and slow start matter as much as the algorithm itself.
Frequently asked questions
What is the most common load balancing algorithm?
Round robin is the most common default because it is simple and works well for identical stateless servers. Many production setups switch to least connections or power of two choices once request costs start to vary.
What is the difference between round robin and least connections?
Round robin sends requests in a fixed rotation without checking server load. Least connections sends each request to the server with the fewest active connections, so it adapts when some requests take much longer than others.
When should I use consistent hashing for load balancing?
Use it when the same key should keep landing on the same server, such as with distributed caches, sharded data, or stateful connections. It keeps most keys in place when servers join or leave, which avoids cache-miss storms and large data movements.
Is Layer 4 or Layer 7 load balancing better?
Neither is universally better. Layer 4 is faster and protocol-agnostic because it only sees connections. Layer 7 understands HTTP, so it can route by path or header, terminate TLS, and apply per-request algorithms, at the cost of more processing per request.