Designing Idempotent APIs for Distributed Systems

Introduction

Distributed systems fail in ways that monoliths never do. A network timeout does not mean the request failed, it means you do not know whether it succeeded. The client retries. Now you have two requests for the same operation, and if your API is not designed to handle this, you might charge a customer twice, create a duplicate order, or send the same email notification a hundred times.

Idempotency is the property that makes an API safe to call multiple times with the same input and always produce the same result. It is not a nice-to-have, in any system that uses retries (which is every distributed system worth its name), it is a correctness requirement. This post covers how to design idempotent APIs pragmatically, with patterns you can apply immediately.

Why Duplicate Requests Are Inevitable?

Three scenarios cause duplicate requests in production, and all three happen regularly:

• Network timeouts: The client sends a request, the server processes it successfully, but the response is lost in transit. The client sees a timeout and retries. The server now receives the same request twice.
• Retry logic in queues: SQS, Kafka, and EventBridge all guarantee at-least-once delivery, not exactly-once. A Lambda that consumes an SQS message and crashes mid-processing will receive the same message again on restart.
• User behavior: Users double-click submit buttons, mobile apps retry on poor network conditions, and payment SDKs have built-in retry loops. Your server will see duplicates regardless of how clean your frontend code is.

The key insight is that you cannot prevent duplicate requests from arriving, you can only ensure that processing them twice has no additional effect.

The Idempotency Key Pattern

The most widely used pattern for idempotent APIs is the idempotency key — a unique identifier the client generates per logical operation and sends with the request, typically as a header:

The server stores the idempotency key alongside the result of the operation in a durable store (typically a database or Redis). On subsequent requests with the same key, instead of processing the operation again, the server returns the stored result. The client receives an identical response whether it is the first or the tenth request.

What to Store

For each idempotency key, persist: the key itself, the HTTP status code and response body, the timestamp of the original request, and the user or tenant it belongs to (to prevent key reuse across users). A DynamoDB table with the idempotency key as the partition key and a TTL attribute for automatic expiry (typically 24 hours) is a clean, serverless-friendly implementation.

Key Generation on the Client

The client must generate a new UUID per logical operation, not per HTTP request. If a payment attempt fails with a timeout, the retry should reuse the same idempotency key. Only when the user explicitly initiates a new payment (clicking the button again after a clear failure message) should a new key be generated. This discipline is as important as the server-side implementation.

Making Operations Naturally Idempotent

Not every API needs an explicit idempotency key mechanism. Some operations are naturally idempotent if you design them correctly.

• PUT over POST: PUT /users/123 with a full user object is idempotent by definition, calling it ten times results in the same state. POST /users (which creates a new user each time) is not. Where possible, prefer PUT or PATCH with a known resource ID.
• Conditional writes: In DynamoDB, use ConditionExpressions to only write if the item does not already exist (attribute_not_exists(pk)). This turns a create operation into an idempotent upsert at the database level.
• State machine guards: Before processing a state transition (e.g., marking an order as shipped), check that the order is currently in the expected prior state (e.g., paid). If it is already shipped, return success without re-processing. This prevents duplicate state transitions regardless of how many times the request arrives.

Idempotency in Event-Driven Systems

Queue-based architectures deserve special attention because at-least-once delivery is baked into every major message broker. An SQS message that triggers a Lambda function to charge a customer must be idempotent, Lambda will automatically retry on error.

The AWS Powertools for Lambda library includes a built-in idempotency utility that uses DynamoDB to store function invocation results keyed by the SQS message ID. Wrapping your handler with this decorator means duplicate SQS messages are silently de-duplicated without any custom code:

For Kafka or EventBridge consumers, the same principle applies, use the event ID or message ID as the idempotency key and check a deduplication store before processing.

Conclusion

Idempotency is not a feature you add later, it is a design discipline you build in from the start. In any distributed system that uses retries, queues, or unreliable networks (which is all of them), duplicate requests are not edge cases. They are normal operating conditions.

The idempotency key pattern gives you a reliable, explicit mechanism for safe retries on mutation endpoints. Natural idempotency through PUT semantics, conditional writes, and state machine guards reduces the surface area where duplicates can cause harm. And purpose-built tools like AWS Lambda Powertools make idempotency in event-driven systems accessible without reinventing the wheel. Design for duplicates up front, and you will spend far less time debugging mysterious double charges in production.

Drop a query if you have any questions regarding Idempotency, and we will get back to you quickly.