Blog
An AI API timeout is an unknown outcome, not a failed action
· agents, engineering, reliability, analysis
How to separate generation retries from tool execution, recover after an interrupted stream, and test duplicate effects before shipping an agent.
An agent asks a service to create a support ticket. The service creates it, but the connection closes before the acknowledgement reaches the agent. The agent sees an error, repeats the request, and creates a second ticket.
The model may have followed its instructions perfectly. The application lost the distinction between a failed connection and a failed action.
The same ambiguity appears one step earlier in an AI API call. A generation can begin or finish upstream while the client receives no complete result. Repeating the call can create another billable generation. That does not, by itself, prove that one request was billed twice. It may mean the application submitted two independent requests for one intended operation.
Three events that need separate records
For an agent that can change external state, keep three questions separate:
Boundary What success establishes What it does not establish
Model response A complete model result reached the application A proposed tool action actually happened
Tool execution The destination accepted or completed the action The user received the acknowledgement
User acknowledgement The application reported the result Every retry or side effect was accounted for
A single green “agent succeeded” badge cannot explain which boundary failed. An operation identifier should connect the records without making them the same record. A generation attempt may have its own request ID; a business action should retain a stable operation ID across recovery attempts.
For example, create-ticket:case-123 identifies the user's intent. It should not be regenerated merely because a new model response proposed the action again. A fresh operation ID on every retry defeats deduplication.
A timeout belongs in the unknown state
HTTP already makes this distinction. RFC 9110, section 9.2.2 says automatic retries of non-idempotent requests require knowledge that the operation is effectively idempotent or that the original request was never applied. Receiving