Deterministic vs Probabilistic Systems
The most important mental shift for an AI architect is moving from deterministic thinking to probabilistic thinking. Traditional software is deterministic: the same input produces the same output, every time, and you can reason about correctness with tests and types. AI systems built on large language models are probabilistic: the same prompt can produce different answers, and any answer can be subtly or completely wrong.
What changes when behavior is probabilistic
- Correctness becomes a distribution, not a guarantee. You design for "usually right, and safe when wrong," not "always right."
- Tests change shape. You still unit-test the deterministic glue (retrieval, validation, routing), but you evaluate the model with example sets and quality metrics rather than asserting exact strings.
- Errors are silent. A wrong answer looks exactly like a right one. Nothing throws. This is why observability and grounding are not optional.
Designing around uncertainty
Good AI architecture surrounds a probabilistic core with deterministic guardrails:
- Constrain the input — give the model only the context it needs (retrieved, scoped).
- Constrain the output — force structured output (JSON schemas) where possible, so a malformed answer is rejected instead of stored.
- Measure confidence — grade how well the retrieved evidence supports an answer; if it's weak, hedge; if there's none, refuse.
- Log everything — provider, model, tokens, latency, and whether the system refused.
A concrete example
When LyraLearn's AI Tutor answers a question, it does not ask the model "what do you know about X?" It first retrieves the relevant lesson passages, grades how strongly they support an answer, and only then asks the model to answer using only those passages, with citations. If nothing relevant is retrieved, it refuses rather than inventing an answer. That is deterministic scaffolding around a probabilistic core — the essence of AI architecture.