16 — Idempotent Operations: Safe to Retry, Safe to Duplicate

A footnotes-grade detail was how I treated idempotency, until it bit me in production. Writing it down made the concept load-bearing: an operation is idempotent if it has the same effect whether executed once or many times. [1][2] That single property is what makes retries safe, and retries are everywhere in distributed systems.

The framing that clicked is that idempotency is the answer to a delivery problem. Networks drop messages, workers crash mid-task, queues redeliver. Most queueing systems guarantee at-least-once delivery, not exactly-once — they would rather risk a duplicate than silently drop a message, because exactly-once is expensive or impossible across geographies [2]. At-least-once means the same message may be processed two, three, five times. If the operation it triggers is not idempotent, each duplicate does the work again — a charge processed twice, a count incremented five times, a notification sent repeatedly. If the operation _is_ idempotent, duplicates are harmless.

What idempotency looks like

The cleanest way to see it is through HTTP verbs, which carry idempotency as a defined property [3]:

• GET — idempotent. Reading a resource ten times has the same effect as reading it once (no state change).
• PUT — idempotent. "Set resource X to value Y" is the same whether run once or a hundred times; the final state is Y either way.
• DELETE — idempotent. "Delete resource X" leaves the system in the same state after the first call; subsequent calls are no-ops.
• POST — not idempotent. "Create a new order" run five times creates five orders. "Increment the counter" run five times increments by five.

The pattern: operations that _set_ an absolute state are idempotent; operations that _apply a relative change_ or _create a new entity_ are not. PUT /users/42 {name: "Ave"} is idempotent. POST /users {name: "Ave"} is not — each call creates a new user.

Why this matters: at-least-once delivery

The reason idempotency is load-bearing rather than academic is that real systems deliver messages more than once. A worker crashes after side-effects but before acknowledging the message; the queue redelivers. A network blip causes a client to retry a request that actually succeeded. A geographic replication lag causes two regions to process the same event [2].

If the operation is idempotent, none of this matters — the duplicate is a harmless no-op. If it is not, every duplicate is a bug: a double charge, a duplicated record, an inflated metric. Designing operations to be idempotent lets you use an at-least-once queue without fear, which simplifies the system enormously, because exactly-once delivery is genuinely hard and often impossible across unreliable networks.

How to make an operation idempotent

The practical techniques, in rough order of how much I reach for them:

• Prefer setting absolute state over applying relative changes. PUT /balance/42 {value: 100} is idempotent; POST /balance/42/increment {by: 10} is not. Where possible, express the operation as "set to X" rather than "add X."
• Use a client-supplied idempotency key. For inherently non-idempotent operations (creating an order, charging a card), have the client generate a unique key and send it with each request. The server records the key on first success and returns the stored result on any subsequent request with the same key, so a retry returns the original response instead of creating a second order. This is how Stripe's idempotency keys work.
• Make creation idempotent by natural key. If a resource has a natural unique identifier (an order number, a transaction ID), use upsert semantics keyed on it — a duplicate create becomes an update of the same row, not a second row.

How I use this

The habit is to ask, of every operation that crosses a process or network boundary: "what happens if this runs twice?" If the answer is "nothing different," it is safe to put behind a queue or a retry loop. If the answer is "it would double-charge / double-create / double-count," I make it idempotent before it ships — usually with a client-supplied idempotency key for create/charge flows, or by switching the operation to absolute-state semantics where the domain allows. The discipline of that question, asked once per operation, is what turns at-least-once delivery from a terrifying prospect into a non-event.

References

[1] "What is an idempotent operation?," Stack Overflow. [Online]. Available: https://stackoverflow.com/questions/1077412/what-is-an-idempotent-operation

[2] "Overview of idempotent operations," Baeldung, 2023. [Online]. Available: https://www.baeldung.com/cs/idempotent-operations

[3] cs.fyi, "Everything you need to know about HTTP," 2021. [Online]. Available: https://cs.fyi/guide/http-in-depth

[4] "Idempotence — computer science meaning," Wikipedia. [Online]. Available: https://en.wikipedia.org/wiki/Idempotence#Computer_science_meaning

[5] "Exponential backoff," Wikipedia. [Online]. Available: https://en.wikipedia.org/wiki/Exponential_backoff