Resilience Questions

The scenario arrives as a complaint: "The state licensing API we depend on is flaky — slow some afternoons, down for minutes at a time. Users see spinners and 500s. What do you do?" The rubric behind it has four boxes — timeouts, retries with backoff, circuit breakers, idempotency — and one meta-box: do you understand these protect your app from their outage, not the other way around.

Timeouts first, always

Start here, because everything else is meaningless without it: "First I'd make sure every outbound call has an explicit timeout." HttpClient defaults to 100 seconds — say that number; it signals real-world experience. A request thread pinned for 100 seconds per call is how a dependency's slowness becomes your thread-pool starvation and your outage. Pick a timeout from the dependency's observed p99, not from vibes, and mention cancellation tokens flowing through the call chain so abandoned requests actually stop.

Retries — the part everyone gets half right

Retry transient failures only: timeouts, 408, 429, 5xx, connection resets. Never retry 400 or 401 — the same request will fail the same way. The words the interviewer is listening for: exponential backoff with jitter and a small, bounded attempt count (2–3). Explain jitter in one sentence: if a thousand clients retry on the same schedule, they stampede the recovering service in synchronized waves. Name the tool — Polly, via Microsoft.Extensions.Http.Resilience (AddStandardResilienceHandler() gives you the whole pipeline) — but make clear you understand the policy, not just the NuGet package.

Then the key follow-up: "What if the retried call was a POST that creates a payment record?" This is your idempotency key cue: the client sends a unique key per logical operation, the server stores it and returns the original result on replay. Retries are only safe when the operation is idempotent — by HTTP semantics or by engineering.

Circuit breakers and failing gracefully

When the dependency is down, retries make it worse. A circuit breaker watches the failure rate, opens after a threshold (fail fast, no calls), then half-opens to probe recovery. The half of the answer that actually differentiates candidates: what does the user see while the circuit is open? A cached last-known verification result, a "verification pending" status that a background job resolves later, a degraded read-only view — pick one and say it. Failing fast is a tool; graceful degradation is the goal.

Answers that fall flat

• Infinite or unbounded retries; retrying non-transient errors; no backoff.
• No timeout story — or "we'd just increase the timeout."
• Treating Polly as magic: naming the package but unable to explain open/half-open states.

Practice prompts

• Answer the flaky-API question in three minutes, hitting all four boxes in order.
• Explain jitter and the retry stampede to a non-engineer.
• Design the user experience for "licensing API down for 10 minutes" in a records-lookup screen.