Idempotency in Distributed Systems

An operation is idempotent if performing it multiple times produces the same result as performing it once.

Why Idempotency Matters

In distributed systems, network failures cause ambiguity:

Client → Request → Server (processes)
       ← [network drops]
Client doesn't know if request succeeded!
→ Retries → Potential duplicate processing

Idempotency Key Pattern

Client generates unique key per logical operation.

POST /payments
Idempotency-Key: a7f3c2e1-4b8d-9f0a-1234-56789abcdef0

{ "amount": 100, "to": "merchant_123" }

Server:

1. Check if idempotency_key seen before
2. If YES → return stored result (don't reprocess)
3. If NO  → process + store result by key

Storage Schema

CREATE TABLE idempotency_records (
  key         UUID PRIMARY KEY,
  response    JSONB,
  created_at  TIMESTAMP,
  expires_at  TIMESTAMP  -- TTL: 24 hours
);

HTTP Method Idempotency

GET    → Idempotent (read-only)
PUT    → Idempotent (replace resource)
DELETE → Idempotent (delete again = same result)
PATCH  → NOT idempotent by default
POST   → NOT idempotent by default

Delivery Semantics

At-most-once:   May lose messages, never duplicates
At-least-once:  May duplicate, never loses
Exactly-once:   No loss, no duplicates (hardest)

Kafka: At-least-once by default, exactly-once via transactions.

Database Idempotency

-- Safe: INSERT ... ON CONFLICT DO NOTHING
INSERT INTO payments (id, amount)
VALUES ('pay_123', 100)
ON CONFLICT (id) DO NOTHING;

-- Safe: Conditional update
UPDATE orders SET status = 'shipped'
WHERE id = 1 AND status = 'paid';

Conclusion

Idempotency keys + stored results is the universal pattern. Design every mutating API endpoint to be safely retryable.