Scaling and Operations

A system that works on your laptop but can't be deployed, scaled, or recovered is a liability, not an asset. The final concern of an enterprise AI architecture is operations: how the app runs in production, how it grows under load, and how it survives the bad day. The good news is that the layered, interface-driven design from earlier lessons makes the app naturally easy to operate — most of the work is following a few well-worn patterns.

Scale by staying stateless

The single most important property for scaling is a stateless application tier. The web app holds no session in memory, no per-user files on disk, no cached embeddings that only one instance knows about. Every piece of durable state lives in SQL Server — business data, vectors, the outbox, audit logs.

When state lives entirely in the database, scaling is mechanical:

• Scale out, not up — run two or more identical app instances behind a load balancer and add more when traffic grows. Any instance can serve any request.
• Separate the tiers — the app and the database scale independently. CPU-bound inference pressure on the app never competes with the database's memory budget.
• Isolate the ingestion worker — run it as its own process so a heavy re-index batch can't starve user-facing requests.

LyraLearn deploys exactly this way: stateless ASP.NET Core containers, one SQL Server, one ingestion worker, all horizontally scalable.

Deploy with confidence: containers, push-to-deploy, health checks

Package the app as a container so the same image runs identically on a laptop, in CI, and on the VPS. Wire push-to-deploy: a push to the main branch builds the image, runs tests, and rolls it out automatically — no manual SSH, no snowflake servers.

The rollout must be verified, which is what health endpoints are for:

• Liveness (/health/live) — is the process up? If not, the orchestrator restarts it.
• Readiness (/health/ready) — can it actually serve traffic? It checks the SQL Server connection and that the inference provider responds. A new instance receives traffic only after readiness passes, so a broken deploy never takes users down.

Plan for the bad day: backups and runbooks

Because SQL Server is the only place state lives, your operational story collapses to one thing: protect the database. Take automated backups, verify them by periodically restoring to a scratch instance (an untested backup is a hope, not a backup), and know your recovery-time and recovery-point objectives.

Finally, write runbooks — short, concrete documents for the predictable incidents: the ingestion worker is backed up, the inference provider is timing out, readiness is failing after deploy. A runbook turns a 2 a.m. panic into a checklist. Combined with the audit and evaluation records from earlier lessons, the operator always has both the what's broken and the what do I do — which is the difference between a demo and a system you can run for years.