Caching

Overview

A cache stores copies of frequently used data in a faster location so future requests avoid slow work. Reading from memory takes about 100 nanoseconds; a database query across the network commonly takes 1 to 10 milliseconds or more.

Caching appears at every layer: the browser, CDN, reverse proxy, application memory, distributed caches like Redis, and the database's own buffer pool. Each layer trades freshness for speed, so the core design question is always: how stale is acceptable?

When caching helps

• Reads greatly outnumber writes (for example 100:1 on product pages).
• The same data is requested repeatedly (skewed access: a few items are very popular).
• The source is slow or expensive: complex queries, external APIs, rendering, ML inference.
• Slightly stale data is acceptable for a short time.

Where it shows up in interviews

Where it is used in real software

Key terms

Hit / miss

A hit finds data in the cache; a miss must fetch from the source.

Hit ratio

hits / (hits + misses). A higher ratio means fewer slow source calls.

TTL

Time-to-live: how long an entry stays valid before expiring.

Eviction

Removing entries when the cache is full, using a policy like LRU or LFU.

Invalidation

Removing or updating cached data when the source changes.

Cache-aside (the most common pattern)

1. 1
   Check the cache

   The application looks up the key, for example user:42.

2. 2
   Return on hit

   If present, return it immediately. No database call is made.

3. 3
   Load on miss

   On a miss, query the database for the data.

4. 4
   Populate the cache

   Store the result with a TTL so it expires even if invalidation fails.

5. 5
   Invalidate on write

   When the data changes, write to the database first, then delete the cache key so the next read reloads fresh data.

Why the hit ratio matters

Cache latency 1 ms, database latency 50 ms, 10,000 requests per second.

Average latency = hit% x cache latency + miss% x (cache + database latency). Going from 95% to 99% cuts database load by 5x, which is why tuning TTLs and key design matters.

Implementation

1async function getUser(id: string): Promise<User> {2  const key = `user:${id}`;3 4  const cached = await redis.get(key);5  if (cached) return JSON.parse(cached);          // hit6 7  const user = await db.users.findById(id);       // miss8  await redis.set(key, JSON.stringify(user), "EX", 300); // 5-minute TTL9  return user;10}11 12async function updateUser(id: string, changes: Partial<User>) {13  await db.users.update(id, changes);  // 1. source of truth first14  await redis.del(`user:${id}`);       // 2. invalidate, do not overwrite15}

Complexity and performance

Trade-offs

Variants and related techniques

Common mistakes

• Cache stampede: a hot key expires and thousands of requests hit the database at once.

  Fix: Use request coalescing or a lock per key, add TTL jitter, and refresh hot keys early.

• Cache penetration: repeated requests for keys that do not exist.

  Fix: Cache negative results briefly or use a Bloom filter.

• No TTL on entries.

  Fix: Always set a TTL as a safety net against missed invalidations.

• Caching personalized data under a shared key.

  Fix: Include the user or tenant in the key and never cache authenticated responses at a public CDN.

Interview questions

Practice problems