Surviving failure
The payment timed out. Did it go through?
Keep the identity of an operation separate from the request that happened to carry it.
An independent note, with worked examples. Watch the source lessons.
A timeout leaves a question open
A customer pays for an order. The service records the payment, then the connection breaks before the response arrives. The customer sees a spinner and presses the button again. There have been two network requests, but the customer intended one purchase.
The client cannot infer the server’s state from the missing response. The first request might never have arrived, might still be running, or might already have committed. A timeout should leave the operation’s outcome unknown until the service can resolve it.
Arpit’s payment API lesson introduces a stable idempotency key to connect those repeated requests. Hello Interview’s e-commerce mock puts the same problem inside checkout, where payment state must survive lost connections and retries.
One intended payment, one key
In our example, the client creates a random key K when the customer starts the payment. Every retry for that payment carries K. Starting a genuinely new payment creates another key. A fresh key on every HTTP attempt would make all the attempts look unrelated.
The server scopes K to the authenticated customer or tenant and records the intended operation. A fingerprint can include the order, amount and currency. If the same key arrives with different details, reject the mismatch. A key is a deduplication aid, not permission to debit an arbitrary account.
Do not deduplicate solely by amount and recipient. Two legitimate purchases can have identical fields. It is the explicit operation identity that lets the customer say “retry this purchase” without saying “make another purchase.”
Make the record and the effect agree
Consider a deliberately narrow example: a transfer between two balances in one transactional database. The operation table has a unique key on (tenant_id, idempotency_key). In one transaction, claim that key, validate the intent, write the ledger entries, and store the result.
begin transaction
insert operation(tenant, K, request_fingerprint)
validate transfer
write debit and credit ledger entries
save operation result
commit
A concurrent duplicate must lose the unique-key race or wait for the first transaction to finish, then retrieve its result. If the transaction rolls back, neither the debit nor the completed-operation record should remain. The database enforces the relationship; a separate “have I seen K?” check followed by an unprotected debit does not. Balance rules still need suitable locking or isolation for concurrent operations with different keys.
This is where a simplified diagram can mislead. Writing a flag to Redis and a debit to another database creates a crash window between the two writes. A key by itself does not guarantee exactly-once execution across arbitrary systems.
An external provider adds another boundary
A real checkout often calls a payment provider, which cannot participate in the local database transaction above. Persist a payment attempt, reuse the provider’s supported idempotency key on retries, and reconcile its final state through a status lookup or verified webhook. If the local worker crashes after the provider accepts the charge, a later worker must recover that attempt instead of inventing a new payment.
Read the provider’s exact contract. For example, Stripe documents returning the first saved status and body for a key, including some errors, comparing request parameters, and allowing keys to be removed after at least 24 hours. Your application’s retry horizon and permanent order records must account for that retention window. “Retry forever with this header” is not a durable payment protocol.
Safe repetition can still overload the server
Idempotency limits duplicate effects. It does not make retries free. If thousands of clients fail together and retry on the same schedule, they can keep a recovering service busy with the old work.
Arpit’s thundering-herd lesson explains why exponential delays still need randomness. In a toy client, the next delay could be chosen uniformly between zero and min(cap, base × 2^attempt). Set a maximum number of attempts and an overall deadline; stop when another attempt cannot finish within that deadline. The constants should come from the service’s latency and load budget.
For an interview walkthrough, crash the worker in three places: before sending, after the provider accepts, and after the local result commits. Explain which stored record lets it continue each time. That exercise is more revealing than a box labeled “retry service.”