State, Caching, and Config

This lesson covers the operational cluster — where "works on one box" answers go to die. The interviewer's real question underneath all of it: have you run this stuff behind a load balancer?

Statelessness — the scale-out question

"We add a second server behind a load balancer. What breaks?" Inventory the hidden state out loud: in-memory session (user bounces between servers and loses their session — fix with a distributed session store or, better, less session), IMemoryCache (each server now has its own, so users see different data depending on which box they hit), local file uploads, in-process background queues, and — the sneaky one — Data Protection keys: if servers don't share a key ring, auth cookies issued by one server fail validation on the other, producing "random logouts." Naming the Data Protection issue is a strong senior marker because almost everyone has been bitten by it and almost no one can explain it.

IMemoryCache vs distributed cache — the scenario

"Reference data (county codes, review rubrics) is queried on every request. Cache it?" Structure: yes, then choose the tier by coherence needs. IMemoryCache is in-process — nanosecond reads, no serialization, but per-server copies that can disagree after an update. A distributed cache (Redis via IDistributedCache) is shared and survives restarts, but adds a network hop and serialization. For rarely-changing reference data, per-server memory cache with a modest absolute expiration is fine — brief inconsistency is acceptable. For anything correctness-sensitive (a lock on a transcript under review), memory cache is wrong. Two depth points to volunteer: cache invalidation on write beats waiting for expiry when users edit the data, and a stampede (hot key expires, hundreds of requests regenerate it at once) is why you cap concurrency on the rebuild. Trade-off language throughout — this question is graded almost entirely on it.

Options pattern and environment config

"How does configuration work in your apps?" The layered model: appsettings.json → appsettings.{Environment}.json → environment variables → (in Azure) Key Vault or App Configuration, later sources overriding earlier. Bind sections to typed classes via the options pattern — services.Configure<ReviewOptions>(config.GetSection("Review")), inject IOptions<ReviewOptions> — and mention IOptionsSnapshot when per-request reload matters. Validation on startup (ValidateDataAnnotations().ValidateOnStart()) turns a missing setting into a boot failure instead of a 3 a.m. runtime one.

"How do you handle secrets?"

This is a fail-fast question in public sector. The floor: secrets never go in source control — no connection strings in appsettings.json beyond local dev. Local: user secrets (dotnet user-secrets). Deployed: Azure Key Vault or pipeline-injected environment variables, ideally with managed identity so there's no credential to leak at all. Red flags that end interviews: "we keep them in appsettings but the repo is private," or not knowing where production secrets currently live in your own app.

Practice prompts:

1. Answer the load-balancer question for your real app — enumerate its actual hidden state.
2. "Cache the AI analysis results per transcript?" Decide the tier, the key, the expiry, and the invalidation trigger, aloud.
3. Trace one secret in your current project from developer laptop to production, naming every place it lives.