MESSAGING & EVENT-DRIVEN ARCHITECTURE / SYSTEM CONCEPT BRIEF

Dead-letter queues

A dead-letter queue (DLQ) is where messages go when they cannot be processed after a set number of attempts, for example because of malformed data, a missing record, or a persistent bug.

BeginnerPhase 06 / Topic 14 of 18RequirementsTrade-offsFailure modes
01

Overview

A dead-letter queue (DLQ) is where messages go when they cannot be processed after a set number of attempts, for example because of malformed data, a missing record, or a persistent bug. Without a DLQ, such 'poison messages' are retried forever, wasting resources and blocking other messages.

A DLQ isolates failures so the main queue keeps flowing, while preserving the failed messages for investigation and redrive (reprocessing after a fix). DLQs need monitoring and alerting; a silently growing DLQ is lost work.

The post office's undeliverable mail desk

Letters that cannot be delivered after several attempts go to a special desk instead of circulating forever. Someone investigates, fixes the address, and resends them.

02

When to use it

  • Every production queue or event consumer.
  • Handling poison messages that always fail.
  • Preserving failed work for later reprocessing.
  • Separating transient failures from permanent ones.
03

Where it shows up in interviews

Reliable async processing

Recognize it when: some messages will fail permanently.

  • Design a notification system
  • Design a payment webhook processor
04

Where it is used in real software

SQS redrive policy

After maxReceiveCount receives, SQS moves messages to the DLQ; redrive sends them back after a fix.

RabbitMQ dead-letter exchanges

Rejected, expired, or overflow messages are routed to a DLX.

Kafka DLQ topics

Kafka Connect and Spring Kafka publish failed records to a separate topic with error headers.

05

Key terms

Poison message
A message that fails every time.
maxReceiveCount
Attempts before moving to the DLQ.
Redrive
Moving messages from the DLQ back to the source queue.
Error metadata
Headers with the failure reason, stack trace, attempt count.
06

How it works, step by step

  1. 1
    Consumer fails a message

    Throws or nacks.

  2. 2
    Retry with backoff

    Transient errors often succeed later.

  3. 3
    Exceed the retry limit

    Move the message to the DLQ with error details.

  4. 4
    Alert and investigate

    Alarm on DLQ depth greater than zero.

  5. 5
    Fix and redrive

    Deploy the fix, then reprocess DLQ messages idempotently.

Poison message isolated in a DLQ
Step 1 / 4
Main queue
Consumer
Retry 1
Retry 2
Retry 3
DLQ

STEP 1Message 42 has a malformed field; processing throws.

07

Transient vs permanent failures

Classify errors before retrying

Step 1 / 5
ErrorTypeAction
Timeout calling a serviceTransientRetry with backoff
HTTP 429 or 503TransientRetry with backoff
JSON parse errorPermanentSend to DLQ immediately
Referenced record not foundMaybe transient (lag)A few retries, then DLQ
Validation failure (400)PermanentDLQ, alert the producer team

NOWError: Timeout calling a service | Type: Transient | Action: Retry with backoff

Retrying permanent errors wastes capacity; sending them straight to the DLQ keeps the pipeline healthy.

08

Implementation

const MAX_ATTEMPTS = 5; await consumer.run({  eachMessage: async ({ message }) => {    const attempts = Number(message.headers?.attempts?.toString() ?? "0");    try {      await handle(JSON.parse(message.value!.toString()));    } catch (err) {      const permanent = err instanceof SyntaxError || (err as { status?: number }).status === 400;      const target = permanent || attempts + 1 >= MAX_ATTEMPTS ? "orders.dlq" : "orders.retry";      await producer.send({        topic: target,        messages: [{          key: message.key,          value: message.value,          headers: { attempts: String(attempts + 1), error: String((err as Error).message).slice(0, 500) },        }],      });    }  },});
09

Complexity and performance

Retry attemptsTypically 3-10

With exponential backoff.

DLQ retentionLonger than main queue

Time to investigate.

10

Trade-offs

Retry count

More retries recover more transient failures but delay detection of permanent ones.

Ordering

Moving a message to a DLQ lets later messages for the same key proceed, which can break per-key order.

11

Variants and related techniques

Retry topics with delays

orders.retry.1m, orders.retry.10m before the DLQ.

Parking lot queue

Manual-review queue for messages needing human action.

12

Common mistakes

  • No alert on DLQ depth.

    Fix: Alert when the DLQ has messages; they represent failed business work.

  • Redriving before fixing the bug.

    Fix: Messages will fail again; fix first.

  • Losing error context.

    Fix: Store error reason, attempts, and original topic in headers.

13

Interview questions

What is a dead-letter queue and why use one?

A separate queue for messages that fail after several attempts. It prevents poison messages from blocking or wasting resources, keeps the main flow healthy, and preserves failed messages for debugging and reprocessing.

How do you handle ordering with DLQs?

If per-key order matters, pause processing of that key (or partition) when a message fails permanently, or record the failure and make later events tolerate a missing predecessor.

14

Practice problems

ProblemDifficultyWhat it trains
Configure an SQS DLQ with alarmsEasyRedrive policy.
Design retry topics and a DLQ for KafkaMediumError classification.