Idempotency

Overview

An operation is idempotent if performing it multiple times has the same effect as performing it once. In distributed systems, retries are unavoidable: a client times out and does not know if the server processed the request, a message broker redelivers a message, or a user double-clicks. Idempotency makes these duplicates harmless.

Some operations are naturally idempotent (set status = shipped, PUT a resource, DELETE by ID). Others, like charging a card or incrementing a balance, need an idempotency key: a unique ID for the logical operation stored with its result, so a repeated request returns the stored result instead of acting again.

When to use it

• Payments and any money movement.
• APIs that clients retry after timeouts.
• Message consumers with at-least-once delivery.
• Webhooks, which are often delivered more than once.

Where it shows up in interviews

Where it is used in real software

Key terms

Idempotent

f(f(x)) = f(x): repeating has no additional effect.

Idempotency key

Client-generated unique ID for one logical operation.

Deduplication table

Stores processed keys and their results.

At-least-once

Delivery that may duplicate; requires idempotent processing.

How it works, step by step

1. 1
   Client generates a key

   A UUID per logical operation, reused on retries.

2. 2
   Server checks the key

   If seen and complete, return the stored response.

3. 3
   Reserve the key atomically

   Insert with a unique constraint; concurrent duplicates fail or wait.

4. 4
   Perform the operation and store the result

   In the same transaction as the business write when possible.

5. 5
   Expire old keys

   Keep them for longer than the maximum retry window.

A retried payment with and without idempotency

Client sends POST /charges, the response is lost, client retries

Retries are only safe when the server can recognize the repeated operation.

Implementation

1// Express-style handler with an idempotency table:2// CREATE TABLE idempotency_keys (key TEXT PRIMARY KEY, status TEXT, response JSONB, created_at TIMESTAMPTZ DEFAULT now());3export async function createCharge(req: Request, res: Response) {4  const key = req.header("Idempotency-Key");5  if (!key) return res.status(400).json({ error: "Idempotency-Key header required" });6 7  const inserted = await db.query(8    "INSERT INTO idempotency_keys (key, status) VALUES ($1, 'processing') ON CONFLICT (key) DO NOTHING RETURNING key",9    [key],10  );11  if (inserted.rowCount === 0) {12    const existing = await db.query("SELECT status, response FROM idempotency_keys WHERE key = $1", [key]);13    if (existing.rows[0].status === "processing") return res.status(409).json({ error: "Request in progress" });14    return res.status(200).json(existing.rows[0].response); // replay original result15  }16 17  const charge = await payments.charge(req.body.customerId, req.body.amountCents, { idempotencyKey: key });18  await db.query("UPDATE idempotency_keys SET status = 'done', response = $2 WHERE key = $1", [key, charge]);19  res.status(201).json(charge);20}

Complexity and performance

Trade-offs

Variants and related techniques

Common mistakes

• Generating a new key on each retry.

  Fix: The key must be created once per logical operation and reused.

• Checking and writing non-atomically.

  Fix: Use a unique constraint or atomic insert to reserve the key.

• Same key with a different payload.

  Fix: Store a request hash and reject mismatches.

Interview questions

Practice problems