Retry mechanisms - System Design Guide

Overview

Retry mechanisms re-attempt failed operations to recover from transient errors such as timeouts, dropped connections, throttling, or brief outages. Done naively, retries make things worse: immediate retries from thousands of clients at once (a retry storm) can overwhelm a struggling service and prevent it from recovering.

Good retries use exponential backoff (increasing delays), jitter (randomization to spread clients out), a maximum number of attempts, retry budgets, and only retry idempotent operations or errors known to be transient. In messaging systems, retries are implemented with redelivery, delay queues, and retry topics, ending in a dead-letter queue.

Calling a busy phone line

If a line is busy, you do not redial every second; you wait a minute, then five, then fifteen. If everyone waited the exact same time, they would all call back at once, so each person adds a random delay.

When to use it

Network calls between services.
Message consumers handling transient failures.
Cloud APIs returning throttling errors (429, 503).
Database connection hiccups and failovers.

Where it shows up in interviews

Resilient service calls

Recognize it when: downstream sometimes times out.

Design a payment gateway integration
Design a microservice client library

Async retries

Recognize it when: failed messages should be retried later.

Design webhook delivery
Design a notification system

Where it is used in real software

AWS SDKs

Retry with exponential backoff and jitter by default; AWS published analysis showing 'full jitter' performs best.

Stripe webhooks

Retries failed webhook deliveries with exponential backoff for up to three days.

Envoy and Istio

Configure retries with per-try timeouts and retry budgets in the service mesh.

Key terms

Exponential backoff: Delay doubles each attempt: 100 ms, 200 ms, 400 ms...
Jitter: Random variation in delays to avoid synchronized retries.
Retry budget: Limit retries to a percentage of total requests.
Retry storm: Retries amplifying load on a failing service.
Retryable error: Transient failure likely to succeed later.

How it works, step by step

1
Classify the error
Retry timeouts, 429, 503; do not retry 400 or 401.
2
Check idempotency
Only retry operations that are safe to repeat, or use idempotency keys.
3
Wait with backoff and jitter
delay = random(0, min(cap, base x 2^attempt)).
4
Respect limits
Max attempts, overall deadline, retry budget, Retry-After headers.
5
Give up gracefully
Return an error, fall back, or send to a DLQ.

Backoff schedules

Base 100 ms, cap 5 s

Attempt	Exponential	Exponential + full jitter
1	100 ms	random 0-100 ms
2	200 ms	random 0-200 ms
3	400 ms	random 0-400 ms
4	800 ms	random 0-800 ms
5	1,600 ms	random 0-1,600 ms

NOWAttempt: 1 | No backoff: 0 ms | Exponential: 100 ms | Exponential + full jitter: random 0-100 ms

Jitter spreads thousands of clients over time, turning a synchronized spike into smooth load that the service can recover from.

Implementation

1const RETRYABLE = new Set([408, 429, 500, 502, 503, 504]);2 3export async function retry<T>(fn: () => Promise<T>, { attempts = 5, baseMs = 100, capMs = 5000, deadlineMs = 10_000 } = {}) {4  const start = Date.now();5  for (let attempt = 0; ; attempt++) {6    try {7      return await fn();8    } catch (err) {9      const status = (err as { status?: number }).status;10      const retryable = status === undefined || RETRYABLE.has(status); // network errors have no status11      if (!retryable || attempt + 1 >= attempts) throw err;12 13      const retryAfter = Number((err as { retryAfterMs?: number }).retryAfterMs ?? 0);14      const backoff = Math.random() * Math.min(capMs, baseMs * 2 ** attempt); // full jitter15      const delay = Math.max(retryAfter, backoff);16      if (Date.now() - start + delay > deadlineMs) throw err;17      await new Promise((r) => setTimeout(r, delay));18    }19  }20}21 22// Usage: retry(() => paymentsApi.charge(req, { idempotencyKey }))

Complexity and performance

Load amplificationUp to attempts^depth

Retries at every layer multiply.

Worst-case latencySum of delays + attempts x timeout

Bound with a deadline.

Trade-offs

Recovery vs amplification

Retries recover from blips but multiply load during outages; combine with circuit breakers and budgets.

Latency

Retries add latency; set per-try timeouts and an overall deadline.

Variants and related techniques

Hedged requests

Send a second request after a delay and use the first response.

Retry queues

Asynchronous retries with delay queues or scheduled topics.

Common mistakes

Retrying at every layer.
Fix: 3 layers x 3 retries = 27x load; retry at one layer (usually the edge or client).
Retrying non-idempotent operations.
Fix: Use idempotency keys first.
No jitter.
Fix: Synchronized retries hit the service in waves.

Interview questions

Why add jitter to exponential backoff?

Without jitter, clients that failed at the same moment retry at the same moments, creating repeated load spikes. Random delays spread retries out so the service can recover.

How do retries cause cascading failures?

When a service slows, callers retry, increasing its load further; if every layer retries, load multiplies. Use backoff, jitter, retry budgets, circuit breakers, and retry at only one layer.

Practice problems

Problem	Difficulty	What it trains
Implement retry with full jitter	Easy	Backoff math.
Design webhook delivery with retries for 3 days	Medium	Schedules and DLQs.