Idempotency: What, Why and How
A user clicks checkout twice. The network drops before the response arrives. A queue worker retries because the first call timed out.
None of them are trying to break your app. They are recovering from uncertainty.
Without idempotency, uncertainty becomes duplicate orders, double charges, and repeated emails.
What It Is
An operation is idempotent when running it once or ten times leaves the system in the same state.
Idempotent:
user.language = "en"
db.commit()
Run it ten times. Language is still en.
Not idempotent:
wallet.credits += 100
db.commit()
Run it ten times. You added 1000 credits.
The client does not know if the server processed the request. Idempotency lets it retry safely.
Why It Matters
Retries are everywhere:
- Double-clicks
- Mobile clients losing responses
- Load balancers replaying connections
- Queue workers timing out and trying again
- Webhooks delivered twice
POST creates things. Call it twice, you often get two things.
That is the gap idempotency fills.
How It Works
The client sends one key per intended operation:
POST /orders
Idempotency-Key: checkout-cart-15-abc123
{"cart_id": 15, "shipping": "standard"}
The server stores the result keyed by that header. A retry with the same key and same body gets the same response — no second order.
Same key, different body? Reject it. That is not a retry. That is a mistake.
Same key + same payload → replay stored response
Same key + diff payload → 422 error
No key (when required) → 400 error
Two requests at once → one runs, other waits or gets 409
A Simple Implementation
Middleware checks the key before the handler runs.
import hashlib
import json
from fastapi import Request, Response
from fastapi.responses import JSONResponse
from redis import Redis
redis = Redis.from_url("redis://localhost:6379/0")
async def idempotency_middleware(request: Request, call_next):
if request.method not in ("POST", "PUT", "PATCH"):
return await call_next(request)
key = request.headers.get("Idempotency-Key")
if not key:
return JSONResponse({"error": "Missing Idempotency-Key"}, status_code=400)
user_id = request.state.user.id
storage_key = f"idempotency:{user_id}:{key}"
body = await request.body()
fingerprint = hashlib.sha256(
f"{request.method}{request.url.path}{body.decode()}".encode()
).hexdigest()
cached = redis.get(storage_key)
if cached:
entry = json.loads(cached)
if entry["fingerprint"] != fingerprint:
return JSONResponse(
{"error": "Key reused with different data"}, status_code=422
)
return Response(
content=entry["body"],
status_code=entry["status"],
headers={"Idempotency-Replayed": "true"},
media_type="application/json",
)
lock = redis.lock(storage_key, timeout=10)
if not lock.acquire(blocking=False):
return JSONResponse({"error": "Request in progress"}, status_code=409)
try:
response = await call_next(request)
redis.setex(
storage_key,
3600,
json.dumps({
"fingerprint": fingerprint,
"status": response.status_code,
"body": response.body.decode(),
}),
)
return response
finally:
lock.release()
Apply it to write endpoints where duplicates hurt:
@app.post("/orders")
async def create_order(order: OrderCreate, db: Session = Depends(get_db)):
...
Domain Rules Still Matter
Middleware stops duplicate HTTP execution. Your domain still needs invariants.
def create_order(db: Session, cart_id: int, user_id: int) -> Order:
with db.begin():
cart = (
db.query(Cart)
.filter_by(id=cart_id, user_id=user_id)
.with_for_update()
.one()
)
if cart.checked_out_at:
raise CartAlreadyCheckedOut()
order = Order(cart_id=cart.id, user_id=user_id, status="pending")
db.add(order)
cart.checked_out_at = datetime.utcnow()
return order
Idempotency handles retries. Transactions handle business rules. You need both.
Where to Use It
Start where duplicate execution costs money or trust:
- Checkout and payments
- Subscription changes
- Webhook handlers
- Any POST that clients retry
Skip read endpoints. Skip idempotent-by-nature updates that set absolute values.
Key Takeaways
Idempotency means safe retries — same operation, same result.
Clients send one Idempotency-Key per attempt. Server stores and replays the first response.
Same key with different data is an error, not a retry.
Protect the HTTP boundary with middleware. Protect the domain with transactions and invariants.
Networks fail. Users double-click. Design for it.