Overview
A performance bottleneck is the single resource that limits the whole system's throughput or latency, like the narrowest part of a bottle. Improving anything else barely helps. Common bottlenecks are the database (slow queries, locks, connections), CPU-heavy code, memory and garbage collection, disk I/O, network bandwidth, and slow external dependencies.
Finding bottlenecks is a measurement exercise: the USE method checks utilization, saturation, and errors for each resource, while the RED method tracks rate, errors, and duration for each service. After removing one bottleneck, the next one appears, so optimization is iterative.
Widening the highway before and after the bridge does nothing; traffic still crawls across the bridge. Once you widen the bridge, the next narrowest point becomes the new limit.
When to use it
- The system cannot handle more load despite adding servers.
- Latency spikes during peaks.
- Planning capacity and cost.
- Interview questions about scaling limits.
Where it shows up in interviews
Recognize it when: what breaks first at 10x traffic?
- Scale a ticket booking system
- Prepare for a product launch
Recognize it when: one key, row, or partition gets most traffic.
- Design a like counter for viral posts
- Design celebrity timelines on Twitter
Where it is used in real software
Widely used in Netflix and elsewhere to systematically check every resource for utilization, saturation, and errors.
CPU profiles visualized as flame graphs show which functions consume the most time.
PostgreSQL pg_stat_statements and MySQL slow query logs reveal the queries worth optimizing.
Key terms
- Utilization
- How busy a resource is (percent of time working).
- Saturation
- Queued work the resource cannot yet handle.
- Hot spot
- A small part of the data or system receiving disproportionate load.
- Amdahl's Law
- Speedup is limited by the part that cannot be parallelized.
- Profiling
- Measuring where time or memory is spent in code.
How it works, step by step
- 1Reproduce with load
Load test or observe production peaks.
- 2Check each resource (USE)
CPU, memory, disk, network, connection pools, thread pools, database.
- 3Trace slow requests (RED)
Identify which service and span dominates.
- 4Fix the top bottleneck
Index, cache, batch, parallelize, or scale that specific resource.
- 5Repeat
Measure again; a new bottleneck will appear.
Symptoms and likely bottlenecks
Diagnosis cheat sheet
| Symptom | Likely bottleneck | Typical fix |
|---|---|---|
| High CPU on app servers | Serialization, hashing, inefficient loops | Profile, optimize, scale out |
| App idle, requests slow | Waiting on DB or downstream | Indexes, caching, parallel calls |
| DB CPU high, many sequential scans | Missing indexes | Add indexes, rewrite queries |
| Latency spikes every few seconds | GC pauses | Tune heap, reduce allocations |
| Errors under load, pool timeouts | Connection pool exhaustion | Resize pools, add a pooler |
| One shard or partition hot | Skewed keys | Better partition key, split hot keys |
NOWSymptom: High CPU on app servers | Likely bottleneck: Serialization, hashing, inefficient loops | Typical fix: Profile, optimize, scale out
Adding servers only helps when the bottleneck is CPU on a stateless tier. Most real bottlenecks are in data access.
Implementation
-- PostgreSQL: top queries by total time (requires pg_stat_statements)SELECT query, calls, round(total_exec_time) AS total_ms, round(mean_exec_time, 2) AS mean_msFROM pg_stat_statementsORDER BY total_exec_time DESCLIMIT 10; -- Why is this query slow? Look for Seq Scan on large tablesEXPLAIN (ANALYZE, BUFFERS)SELECT * FROM orders WHERE customer_id = 42 ORDER BY created_at DESC LIMIT 20; -- Fix: composite index matching the filter and sortCREATE INDEX CONCURRENTLY idx_orders_customer_created ON orders (customer_id, created_at DESC);Complexity and performance
p = parallel fraction, n = workers.
On large tables.
Trade-offs
Optimizing code or queries is cheaper long-term; scaling hardware is faster short-term but costs more every month.
Optimize measured bottlenecks, not guesses.
Variants and related techniques
When saturated, reject low-priority requests to protect critical ones.
Slow producers down when consumers cannot keep up.
Common mistakes
- Scaling app servers when the database is the bottleneck.
Fix: More app servers mean more DB load; fix data access first.
- Optimizing without profiling.
Fix: Measure CPU, queries, and spans before changing code.
Interview questions
What usually breaks first when traffic grows?
Usually the database: connection limits, slow queries without indexes, write contention, or storage I/O. Then hot keys in caches and synchronous calls to slow dependencies.
How do you handle a hot key, like a viral post's like counter?
Split the counter into shards and sum them on read, cache it with a short TTL, buffer increments in memory or a queue and flush in batches, and serve approximate counts.
Practice problems
| Problem | Difficulty | What it trains |
|---|---|---|
| Diagnose a slow query with EXPLAIN ANALYZE | Easy | Indexes. |
| Find the bottleneck in a system that plateaus at 3,000 RPS | Medium | USE method. |
| Design a like counter for a post with 10M likes/hour | Hard | Hot keys. |