Design retries that do not duplicate work
Treat a timeout as an uncertain outcome and make repeated requests safe for the receiving process.
An integration sends an update and the connection times out. The caller knows it did not receive a response. It does not necessarily know whether the receiving system completed the action. Retrying immediately can turn a temporary interruption into duplicate work.
This distinction matters whenever a workflow creates a record, sends a message, schedules an action, or changes a business state. Recovery needs to account for an uncertain outcome rather than assuming every unsuccessful response means nothing happened.
Give the operation a stable identity
Use an operation identifier that represents the intended action and persists across retries. The receiving side needs a way to recognize that another request refers to the same intent, rather than a new business event.
The AWS Builders’ Library discussion of idempotent APIs explains how caller-provided request identifiers can help make retries safe. The broader design question is what counts as the same operation in your process. A document identifier alone may not be sufficient when a revised document legitimately requires another update.
Record the outcome before returning it
The system responsible for deduplication should reserve or record an operation consistently with its execution. A separate “have I seen this?” check followed by an unprotected action leaves a window in which concurrent requests can both proceed.
Decide how the integration represents pending, completed, failed, and uncertain work. A second request arriving while the first is pending should not start an independent action merely because no final result has been recorded yet.
Retry only the failures that can benefit
Distinguish temporary service unavailability from an invalid record or a rejected business decision. Waiting may help a rate-limited service. It will not supply a missing approval or repair an invalid supplier identifier.
Use bounded retries, increasing delays where appropriate, and a limit that ends in a visible recovery path. Avoid having several layers retry the same operation independently. Their combined behavior can generate far more requests than any one component appears to permit.
Keep compensation separate from reversal
When a process has already changed several systems, recovery may require a compensating action. It may not be possible or correct to restore every system to an earlier state, especially if other work has happened since.
Microsoft’s compensating transaction pattern describes why recovery can require business-specific actions that account for concurrent changes. Name who may authorize those actions, and record the relationship between the original operation and the compensation.
Rehearse the uncomfortable cases
Test a request whose response is lost after the destination accepts it. Send concurrent retries. Change the payload while retaining the same operation identifier. Restart a worker while an action is pending. Inspect what the team can see and how it can complete the work manually.
The goal is a recoverable process. People should be able to determine what was attempted, what is known to have completed, and what needs a decision before anything is repeated.