Circuit Breakers for AI Agents

An AI agent calls an external API and gets a timeout. It tries again, gets another timeout, and continues the cycle. Other agents encounter the same problem, adding traffic to a service that may already be overloaded. Retries are useful when a disruption is brief. When a failure persists, each attempt wastes resources and adds latency. A circuit breaker gives the runtime a way to stop calling a service that is unlikely to recover immediately. After failures cross a defined threshold, the circuit opens and blocks further attempts for a limited time.

How the Circuit Breaker Works

Circuit breakers are commonly described using three states:

Opening a circuit only blocks new calls routed through that breaker. It does not cancel work already in progress or stop an agent’s reasoning loop. Cancellation, concurrency limits, and workflow budgets remain separate controls.

Circuit breakers and retries also serve different purposes. A retry assumes another attempt might succeed soon. A circuit breaker recognizes when further attempts are becoming harmful or wasteful. Retries still need attempt limits, backoff, deadlines, and random jitter to reduce synchronized traffic spikes.

Deciding What Should Open the Circuit

Traditional breakers often monitor timeouts, connection failures, server errors, and slow responses. An AI runtime may also watch for repeated schema failures, invalid tool arguments, malformed retrieval responses, throttling, provider capacity problems, or a sudden decline detected by an evaluator.

These signals need careful interpretation. An unusable result does not always mean the service is unhealthy. An empty retrieval result may be valid, and an authorization rejection may show that access controls are working correctly. A validation error may point to a defective request rather than a failing service.

The runtime should classify failures before changing circuit state. Permanent errors can fail immediately and temporary faults may qualify for bounded retries. Request defects, permission problems, and quality regressions may need correction or review because waiting alone will not resolve them.

Scope matters too. A breaker that is too broad can stop healthy work, while one that is too narrow may allow many agents to keep pressuring the same service. A circuit might apply to one model deployment, tool endpoint, customer account, workflow, region, or operation. Thresholds should reflect traffic volume, failure type, and operational impact. Five failures among ten requests signal something different from five among thousands.

What Happens While the Circuit Is Open

Opening the circuit protects the dependency, but the workflow still needs an outcome. The runtime might return a limited response, use a verified cached result, route the request to an approved secondary provider, queue the task, or escalate it to a person.

Fallbacks need the same scrutiny as the primary path. A cheaper model may lack a required capability, cached information may be stale, and a secondary tool may follow different permission or retention rules.

The runtime must also handle uncertain outcomes carefully. If a tool times out after sending a payment or creating a record, the breaker cannot tell whether the action succeeded. Recovery may require service-enforced idempotency, durable action records, status checks, or reconciliation. A local record alone cannot prevent duplicate effects in an external system.

Observability and Recovery

Teams should record circuit transitions with the triggering dependency, failure category, affected workflows, and fallback behavior. Useful metrics include blocked requests, half-open probe results, recovery time, and tasks routed to queues or reviewers.

Recovery should happen gradually. Releasing every waiting task at once can overwhelm a service that has only partly recovered. Limited probes, controlled traffic increases, queue management, and concurrency caps reduce that risk.

AI agents can generate new requests faster than a struggling dependency can recover. Combined with careful error classification, bounded retries, safe fallbacks, and gradual recovery, circuit breakers help contain repeated failures without immediately recreating the same problem.