The Second Attempt Problem: Retrying AI Without Duplicating Actions

A procurement agent submits a purchase order to an ERP. The tool call waits ten seconds and times out. The orchestrator assumes failure, switches provider, reconstructs the command, and submits it again. The first call had already committed; only its response was lost. The business now has two purchase orders, two approval trails, and an investigation caused by a recovery mechanism.

Each component behaved plausibly: timeout, retry, failover, and two valid ERP commands. What the system lacked was a shared definition of one business intent.

This field guide is about the decision that follows an uncertain attempt: should the system retry, hedge, fail over, reconcile, or stop? Its central rule is simple: a transport failure is not a business outcome. One logical intent must own every attempt, and any operation that may change the world must become identifiable and reconcilable before it becomes retryable.

A timeout creates an unknown state, not a clean failure

Callers observe only part of a distributed operation. A timeout may mean the request never left the client, reached a gateway but not the service, completed inference but lost the response, invoked a tool that committed, or is still running after the caller stopped waiting. These states demand different actions, yet a generic exception often compresses all of them into failed.

That compression is especially dangerous in AI workflows because one visible turn can contain several operations:

The first design task is therefore not choosing a backoff formula. It is drawing the action boundary and naming what can happen on each side of it.

Separate the logical intent from its attempts

A user asks for one outcome: “Create the approved order.” The system may make several network calls while pursuing it. Model the two levels separately:

• intent_id identifies the single user- or policy-authorized outcome;
• attempt_id identifies one execution try inside that intent;
• provider_request_id records a vendor or infrastructure call;
• action_key is the stable idempotency identifier presented to the mutating service;
• receipt_id identifies the authoritative committed result.

Changing provider, region, model, process, or worker creates a new attempt, not a new intent. A browser refresh, queue redelivery, workflow replay, or operator retry must recover the same durable intent. This extends the principles in durable AI workflows: persistence is useful only when replay preserves meaning.

The intent owns the deadline, authorization, canonical parameters, attempt budget, and final disposition. Attempts cannot expand them.

What the protocols establish—and what this guide adds

Several verified sources define the boundary. RFC 9110 says a method is idempotent when repeated identical requests have the same intended server effect as one. It warns against automatically retrying non-idempotent requests unless their semantics permit it or non-application is known, and defines Retry-After, including with 503 Service Unavailable.

The AWS Builders’ Library connects timeouts, capped retries, backoff, jitter, and throttling. Its guide to idempotent APIs favors caller-provided intent identifiers over duplicate guesses and ties the deduplication record atomically to the mutation.

Google SRE shows how cascading retries amplify overload—including 64 backend attempts from retries at three layers. gRPC defines bounded overlapping attempts with shared deadlines, pushback, and throttling. Stripe demonstrates stable keys that replay the first result and reject changed parameters.

The architecture below is ZharfAI analysis derived from those sources. It is not a claim that HTTP, AWS, Google, gRPC, or Stripe defines one universal AI retry standard. In particular, every downstream API has its own idempotency scope, retention window, error caching, and reconciliation semantics; copy the principle, not another service’s undocumented assumptions.

Use a state machine that makes uncertainty visible

Give the logical intent an explicit state machine:

new → prepared → authorized → executing → committed

Add terminal states for rejected, cancelled, and expired, plus one state teams often omit: indeterminate. An attempt becomes indeterminate when the caller cannot prove whether a consequential effect occurred. It must not be relabelled failed merely to keep a dashboard simple.

Only the intent coordinator may move from authorized to executing. It records the attempt before dispatch, passes the same action key to every allowed retry, and accepts one authoritative receipt. When a response is lost, it enters indeterminate, queries the system of record, consumes a webhook or event, or sends the case to manual reconciliation. A new attempt is allowed only after the contract says replay is safe.

Cancellation is an observation, not a rollback: stopping a stream cannot prove that a tool or transaction did nothing.

Choose the recovery by operation class

Use a decision table before writing retry code:

“Retryable” is not an intrinsic property of an error code. It is the intersection of failure evidence, operation semantics, remaining deadline, retry budget, and consequence.

Build an idempotency envelope around business intent

An idempotency key should represent a caller’s declared intent, not a random transport attempt and not merely a hash of prompt text. Store an envelope beside it:

Bind release_id to the AI release passport; a provider or schema change must not broaden the approved command.

The receiver should atomically claim the action key and apply the mutation, or provide an equivalent transactional contract. It should return the prior semantic result for a replay, reject the same key with different canonical parameters, and retain deduplication knowledge longer than any plausible delayed request. Do not put personal or secret data inside visible keys.

If the receiver cannot support idempotency, the caller needs a narrower pattern: conditional creation with a unique business reference, compare-and-set against a version, a reservation followed by confirm, serialized execution, or a status lookup by client reference. If no safe replay or reliable reconciliation exists, consequential automatic retry is not available.

Give one layer the retry budget

Retries consume capacity and money. Set one end-to-end deadline and let one coordinator own the policy. Lower layers may expose failure detail and server pushback, but must not multiply attempts behind its count.

Reserve time for confirmation. A 20-second deadline spent almost entirely regenerating leaves no time to verify the action. A low-consequence read might allow one retry; an unprotected mutation allows none. Use capped exponential backoff with jitter, honor explicit waits, and never reset the exhausted budget by switching provider.

This budget complements inference latency engineering. Tail latency cannot be improved by creating enough duplicate work to overload the service that is already slow.

Hedge reads, never business side effects

Hedging sends a second attempt before the first has failed. It can reduce tail latency for a safe read, but it deliberately increases concurrent load. Use it only for operations that remain harmless when both copies run to completion: immutable retrieval, health queries, or pure inference whose output is not itself an action.

Even then, “first answer wins” needs qualification for AI. The fastest fluent answer is not necessarily the best supported answer. Preserve a shared deadline, cap overlapping attempts, cancel losers, record their cost, and pass the winning output through the same evidence and safety checks. Do not hedge a tool-capable agent unless the action plane is physically disabled for every hedge.

Models may interpret instructions, schemas, tools, and safety differently. Apply the eligibility controls in model routing; failover must not become an unreviewed release or privilege escalation.

Make the action boundary transactional

A robust agentic workflow separates thought from commitment:

1. Prepare: retrieve current evidence and generate a proposed command without mutation.
2. Authorize: validate policy, permissions, limits, schema, freshness, and any human approval; freeze canonical arguments.
3. Record: write the authorized intent and an outbox item atomically in the application system of record.
4. Dispatch: send the outbox item with the stable action key; store the downstream request and receipt identifiers.
5. Reconcile: confirm the authoritative business state, then mark the intent committed and notify the user once.

Queue redelivery replays the outbox item, not the reasoning turn. A webhook is deduplicated before it moves state. The model may explain an indeterminate result, but it does not decide that an absent response means an absent effect. Tool authorization remains independent, as described in AI tool permission security.

This is not magical “exactly once” delivery. It combines at-least-once transport, an idempotent effect, a durable receipt, and reconciliation.

Worked example: one bilingual procurement instruction

An employee requests an approved order in Persian; a supervisor later opens it in English. Both views share one intent_id: locale changes presentation, not business identity.

The assistant retrieves the approved requisition and current supplier record, then produces a structured proposal. Policy code verifies the approver, amount, cost centre, supplier status, and current purchasing rule. After authorization, the application freezes the ERP command and writes an outbox row with action key po:<tenant>:<intent>.

The ERP accepts the command but the response times out. The dispatcher records the attempt as indeterminate. It does not ask another model to rebuild the order. It queries the ERP by the stable external reference. If the purchase order exists, the workflow stores its receipt and completes the original intent. If it does not exist and the ERP contract guarantees idempotent creation for that key, the dispatcher retries the same canonical command with the same key. If neither fact can be established, the case goes to reconciliation with all attempts visible.

A fallback may summarize status, but cannot create another authorization, change the order, or mint a new key. The user receives one explicit outcome.

Test the failure path the demo never shows

Run fault injection before launch:

• drop the response after the downstream service commits;
• delay the first attempt until after the retry returns;
• redeliver the same queue message and webhook several times;
• return overload to every retrying layer simultaneously;
• ignore client cancellation and let the losing hedge finish;
• expire authorization while an attempt is in flight;
• replay the same intent from Persian and English interfaces;
• reuse an action key with changed parameters and confirm rejection.

Measure logical outcomes, not just RPC success:

• attempt amplification: total attempts per logical intent;
• duplicate-effect rate: repeated business actions per committed intent;
• indeterminate rate and age: unresolved outcomes and time to reconciliation;
• failover divergence: materially different proposals across eligible releases;
• late-result rate: attempts completing after deadline or cancellation;
• goodput: useful, policy-valid outcomes per unit of constrained capacity.

Slice these measures by provider, release, tool, operation, tenant, language, and consequence tier. Preserve intent, attempt, policy, action-key, and receipt linkage in an audit-ready evidence trail, without logging unnecessary protected content.

Failure modes that look like resilience

• New idempotency key on every retry: every attempt appears unique, so deduplication cannot work.
• Same key for a changed command: a corrected amount or recipient is a new intent, not a replay.
• Retry at every layer: SDK, gateway, workflow, and UI multiply load while each reports a small local limit.
• Hedging tool-capable agents: both paths may cross the action boundary before a winner is selected.
• Failover without release parity: recovery silently changes policy, tool access, evidence requirements, or schema.
• LLM-generated deduplication: a model guesses whether two actions are “the same” instead of enforcing canonical business identity.
• Retention shorter than replay: a late attempt arrives after the receiver forgot the key and becomes a fresh action.

The acceptance gate

Do not enable automatic retry or failover for an AI workflow until the team can answer yes to these questions:

1. Is the logical intent distinct from every execution attempt?
2. Is the action boundary explicit and physically disabled during proposal generation?
3. Can each error be classified as permanent, transient, overloaded, or indeterminate?
4. Does one layer own the end-to-end deadline and retry budget?
5. Do mutating services accept a stable action key or provide a safe conditional alternative?
6. Can the system locate the authoritative result without repeating the mutation?
7. Are provider fallbacks equivalent for policy, schema, permissions, and evaluated quality?
8. Are delayed results, duplicate events, cancellation, and multilingual replay tested?
9. Can operators pause retries, inspect every attempt, and reconcile unresolved outcomes?
10. Do alerts use duplicate effects, amplification, indeterminate age, and goodput—not only availability?

Revisit the gate when a provider changes retry behavior, an SDK adds automatic attempts, a tool gains a new side effect, the action-key retention window changes, or a model/router release moves the action boundary. The safest second attempt is not the one that runs fastest. It is the one that can prove it still represents the first intent and cannot create a second consequence.

Source Notes — reviewed August 7, 2026

• RFC 9110: HTTP Semantics — Internet Standard published in June 2022; source for safe and idempotent method semantics, automatic-retry cautions, Retry-After, and 503 behavior.
• AWS Builders’ Library: Timeouts, retries, and backoff with jitter — living Amazon engineering guidance reviewed on the publication date; source for timeout selection, capped retry, exponential backoff, jitter, and retry throttling.
• AWS Builders’ Library: Making retries safe with idempotent APIs — Amazon engineering guidance by Malcolm Featonby, publicly announced in January 2021; source for caller-declared intent identifiers, semantic replay, atomicity, late requests, and parameter mismatch.
• Google SRE Book, Chapter 22: Addressing Cascading Failures — online chapter from Google’s 2016 SRE book, reviewed on the publication date; source for retry amplification, randomized backoff, single-layer reasoning, retry budgets, load shedding, and overload recovery.
• gRPC: Request Hedging — official guide last modified October 3, 2023; source for overlapping attempts, per-method limits, shared deadlines, cancellation, pushback, and throttling.
• Stripe API Reference: Idempotent requests — living API contract reviewed on the publication date; a concrete example of stable keys, saved results, parameter comparison, retention boundaries, and the warning not to place sensitive data in keys.