Timeout

Overview

A timeout limits how long a caller waits for an operation before giving up. Without timeouts, a slow or hung dependency makes callers wait indefinitely, holding threads, connections, and memory until the caller itself fails. Timeouts are the most basic and most important resilience mechanism.

There are several kinds: connection timeouts (establishing a connection), request or read timeouts (waiting for a response), and overall deadlines that cover the entire operation including retries. Good timeouts are based on measured latency (for example slightly above the dependency's P99) and deadlines should propagate downstream so services do not keep working on requests whose callers have already given up.

When to use it

• Every network call: HTTP, gRPC, database, cache, message broker.
• Long-running operations that could hang.
• User-facing requests with latency budgets.
• Locks and leases.

Where it shows up in interviews

Where it is used in real software

Key terms

Connect timeout

Max time to establish a connection.

Read / request timeout

Max time to receive a response.

Deadline

Absolute time by which the whole operation must finish.

Deadline propagation

Passing the remaining time to downstream calls.

Cancellation

Stopping work once the result is no longer needed.

How it works, step by step

1. 1
   Measure dependency latency

   P99 and P99.9.

2. 2
   Set timeouts slightly above P99

   Plus margin; shorter connect timeouts.

3. 3
   Set an overall deadline

   From the user-facing budget.

4. 4
   Propagate remaining time

   Headers or gRPC deadlines.

5. 5
   Cancel on timeout

   Abort in-flight work and free resources.

Timeout budget for a 1-second API

Checkout endpoint with 3 dependencies

Downstream timeouts must fit within the caller's budget; a 5-second payment timeout inside a 1-second API is meaningless.

Implementation

1// Deadline-aware fetch using AbortSignal2export async function callWithDeadline(url: string, deadline: number, init: RequestInit = {}) {3  const remaining = deadline - Date.now();4  if (remaining <= 0) throw new Error("Deadline exceeded before call");5 6  const res = await fetch(url, {7    ...init,8    signal: AbortSignal.timeout(Math.min(remaining, 2_000)), // never wait longer than the remaining budget9    headers: { ...init.headers, "x-request-deadline": String(deadline) }, // propagate downstream10  });11  if (!res.ok) throw Object.assign(new Error(`HTTP ${res.status}`), { status: res.status });12  return res.json();13}14 15// In a handler: derive the deadline from the incoming request or a default budget16app.get("/checkout", async (req, res) => {17  const deadline = Number(req.header("x-request-deadline")) || Date.now() + 1_000;18  const cart = await callWithDeadline(`${CART_URL}/carts/${req.query.id}`, deadline);19  res.json(cart);20});

Complexity and performance

Trade-offs

Variants and related techniques

Common mistakes

• Relying on library defaults.

  Fix: Many are infinite or very long; set them explicitly.

• Downstream timeouts longer than upstream.

  Fix: Callers give up while downstream keeps working; propagate deadlines.

• Timing out without cancelling work.

  Fix: Abort queries and requests to free resources.

Interview questions

Practice problems