Overview
A distributed counter tracks counts such as views, likes, or API calls across many servers and regions at very high write rates. A single database row with UPDATE count = count + 1 fails quickly, because every increment contends for the same lock and one node cannot absorb millions of writes per second.
The central decision is accuracy versus cost. Some counts must be exact, like inventory or billing. Others only need to be close and fast, like a view counter. Designs range from sharded counters in a database, to in-memory aggregation with periodic flushes, to event logs rolled up by stream processors, to probabilistic structures such as HyperLogLog for unique counts. Most large systems combine a fast approximate path for display with a slower exact path for reporting.
Every polling station counts its own ballots and reports totals periodically, and a central office adds up the reports. No voter waits for a single central clerk. The central total lags a little, but every ballot is counted.
Requirements
- Functional: increment a named counter; read its current value; optionally count unique users; support per-time-window counts.
- Non-functional: absorb 1M+ increments per second; reads under 10 ms; counts visible within a few seconds; no lost increments for exact counters.
- Scale assumption: 100M active counters with a heavy skew where a few viral items receive most of the traffic.
- Out of scope: complex analytics queries and dashboards.
Where it shows up in interviews
Recognize it when: many writers update the same value.
- Design a like counter for viral posts
- Design Rate Limiter
Recognize it when: counts shown to users vs counts used for billing.
- Design YouTube view counts
- Design an ad click aggregator
Where it is used in real software
Large streaming companies have described counter services that offer both a best-effort mode and an eventually consistent exact mode built on event logs and periodic rollups.
Displayed like counts are often approximate and cached, while an accurate count is reconciled in the background.
Counts unique visitors with about 12 KB per counter and under 1 percent error.
Key terms
- Sharded counter
- A counter split into N sub-counters; writes pick one at random and reads sum them all.
- Write-behind buffer
- Increments aggregate in memory and flush to storage periodically.
- Idempotency key
- A unique event ID so retried increments are counted once.
- Rollup
- Periodically aggregating raw events into totals per time window.
- HyperLogLog
- A probabilistic structure that estimates unique counts with small, fixed memory.
How it works, step by step
- 1Classify each counter
Decide whether it must be exact, can be approximate, or needs unique counting.
- 2Spread writes
Split hot counters into shards or buffer increments in memory on each app server.
- 3Record durably
For exact counts, write increment events to a log such as Kafka with event IDs for deduplication.
- 4Aggregate
Stream processors or periodic jobs roll events into totals per counter and time window.
- 5Serve reads from cache
Store current totals in a cache and accept slight staleness for display.
STEP 1Each server adds increments to an in-memory buffer instead of hitting the database.
Choosing a strategy
Four counters in one product.
| Counter | Accuracy need | Strategy |
|---|---|---|
| Video view count shown on page | Approximate | Buffered increments plus cache |
| Ad clicks for billing | Exact | Event log with dedupe and rollups |
| Unique daily visitors | Approximate unique | HyperLogLog per day |
| Remaining stock | Exact and immediate | Single-row transaction or reservation service |
NOWCounter: Video view count shown on page | Accuracy need: Approximate | Strategy: Buffered increments plus cache
One counter design does not fit all. Match the strategy to the business impact of an error.
Implementation
// Buffered sharded counter: batches increments in memory and flushes to Redis shards.const SHARDS = 16;const buffer = new Map<string, number>(); export function increment(counter: string, by = 1) { buffer.set(counter, (buffer.get(counter) ?? 0) + by);} setInterval(async () => { if (buffer.size === 0) return; const batch = new Map(buffer); buffer.clear(); const pipeline = redis.pipeline(); for (const [counter, delta] of batch) { const shard = Math.floor(Math.random() * SHARDS); pipeline.incrby(`count:${counter}:${shard}`, delta); } await pipeline.exec();}, 1000); export async function read(counter: string): Promise<number> { const keys = Array.from({ length: SHARDS }, (_, i) => `count:${counter}:${i}`); const values = await redis.mget(...keys); return values.reduce((sum, v) => sum + Number(v ?? 0), 0);}Complexity and performance
Contention divided by shard count.
Cache the sum to keep reads cheap.
Regardless of cardinality.
Trade-offs
Longer flush intervals mean fewer writes but staler counts and more loss if a server crashes before flushing.
Exact counts need durable logs, deduplication, and reconciliation. Approximate counts are far cheaper.
Variants and related techniques
Each region keeps its own counter and merges by summing, allowing multi-region writes without coordination.
Estimates frequencies of many items, useful for top-K and heavy-hitter detection.
Common mistakes
- Incrementing a single database row for a hot key.
Fix: Shard the counter or buffer increments.
- Counting retries twice.
Fix: Attach event IDs and deduplicate in the aggregator.
- Using approximate counts for billing.
Fix: Keep a separate exact pipeline for anything with money or compliance impact.
Interview questions
How do you count likes on a post receiving 100K likes per second?
Avoid a single hot row. Buffer increments in memory on app servers or use a sharded counter, flush batched deltas to a durable log, aggregate them asynchronously, and serve the displayed count from cache with a few seconds of staleness. Store who liked what separately for deduplication.
How do you count unique visitors per day across billions of events?
Use a HyperLogLog per day (and per page if needed). It estimates cardinality with about 1 percent error in fixed small memory, and daily sketches can be merged for weekly or monthly counts.
Practice problems
| Problem | Difficulty | What it trains |
|---|---|---|
| Implement a sharded counter and measure write throughput | Medium | Contention. |
| Design an exact ad-click counter with deduplication | Hard | Event logs and rollups. |