Distributed systems
Circuit Breaker
When a downstream dependency is failing, a circuit breaker stops the bleeding — it fails fast instead of piling retries onto a service that's already down.
✗ The problem
→→→
Retrying a sick service makes it sicker
A downstream call starts failing. The caller keeps retrying — every retry blocks a thread waiting on a service that won't answer.
while (true) {
try {
return callDownstream();
} catch (e) {
// retry immediately —
// piles more load on a
// service that's already down
}
}
Caller
retrying…
Service
down, overloaded
Threads pile up waiting on a dead dependency. The caller's own thread pool
exhausts, and the failure cascades upstream to every service that depends on it.
✓ How it works
↓ failure threshold reached
↓ cooldown elapsed
↑ success → CLOSED · failure → OPEN
Three states, one wrapper around the call
The breaker sits in front of the call and tracks its own health, independent of the caller.
class CircuitBreaker {
call(fn) {
if (this.state === 'OPEN')
throw new Error('fail fast');
try {
const r = fn();
this.onSuccess();
return r;
} catch (e) {
this.onFailure(); // may trip OPEN
throw e;
}
}
}
CLOSED
calls flow, count failures
OPEN
fail fast, no call made
HALF-OPEN
allow one trial call
✓ See it live
Trip it, then watch it recover
Fail 3 calls in a row to trip the breaker OPEN — further calls fail fast with no call made. Then cooldown to HALF-OPEN and send one trial call.
CLOSED
calls flow
OPEN
fail fast
HALF-OPEN
trial call
State: CLOSED • Failures: 0 / 3
✓ Takeaway
Fail fast, recover gracefully
- Fail fast: once tripped, callers get an immediate error instead of waiting on a timeout.
- Give the dependency time to recover: no load reaches it while OPEN.
- Prevent cascades: the caller's own threads and resources stay free for other work.
- Pair it up: combine with retries, timeouts, and fallbacks for full resilience.
- You already use it: Netflix Hystrix, Resilience4j, Polly.
🎯 Combines / relates: it's the State pattern applied at the system boundary (a Proxy around the call), and teams with rate limiting for resilience.
Back to all topics →