Classical ML in One Lesson
"AI" postings say "machine-learning components," and interviewers ask "when would you NOT use an LLM?" — a question designed to catch candidates whose AI knowledge begins and ends with chat models. Classical machine learning is decades older than LLMs, runs most of the world's production ML, and is often the right answer in government systems precisely because it's cheap, explainable, and auditable. Here's the literacy you need — no math.
The core idea
Classical supervised learning: you have historical rows with known outcomes (features → label), you train a model to map features to outcomes, and it predicts outcomes for new rows. Two flavors:
- Regression — predict a number. How many days will this application take to process?
- Classification — predict a category. Is this transaction fraudulent? Will this application need specialist review?
The models to be able to name:
- Linear/logistic regression — the simple, fast, deeply explainable baseline; every feature gets a weight you can read ("each prior application adds 1.3 days"). Start here; it's often embarrassingly competitive.
- Decision trees and gradient-boosted trees (XGBoost/LightGBM) — the workhorses of structured-data ML. On tabular data (rows and columns — most government data), boosted trees routinely beat neural networks, train in minutes on a laptop, and cost approximately nothing to run.
- Clustering (unsupervised) — no labels; find natural groupings. Do our batch-failure patterns fall into families?
- Anomaly detection — learn "normal," flag departures. This job's runtime is four standard deviations off; alert. (If you've written threshold-based monitoring, you've built this by hand.)
Evaluation vocabulary that must be automatic: precision (of the things flagged, how many were right), recall (of the things that should be flagged, how many we caught), the precision-recall trade-off, and training vs. test data — you always evaluate on data the model never saw, or you're grading it on memorization.
When classical beats the LLM
The decision rule: LLMs are for language; classical ML is for structured data.
| Situation | Right tool | Why it wins | |---|---|---| | Predict processing time from application attributes | Regression / boosted trees | Numeric prediction from tabular features; explainable weights; costs ~nothing per prediction | | Flag applications likely to need specialist review | Classification | Trained on your history; per-feature explanations satisfy auditors; millisecond inference | | Detect abnormal batch-job behavior | Anomaly detection | Learns your baseline; no labels needed; runs continuously for free | | Read free-text course titles and judge equivalence | LLM (grounded) | Genuine language understanding; no training set of labeled pairs exists | | Summarize a failure log against runbooks | LLM | Unstructured text in, prose out |
Cost and explainability are the interview clinchers. A boosted-tree prediction costs microseconds of CPU; an LLM call costs real money and hundreds of milliseconds — at volume, that's the whole business case. And when an auditor asks "why was this application flagged?", a tree model gives you feature importances; an LLM gives you a paragraph you can't fully trace. In public-sector decision support, that difference is often decisive.
The answer to memorize
"LLMs earn their cost on language — unstructured documents, equivalence judgments, drafting. For structured data — predictions and classifications from rows and columns — classical models win on cost, speed, and explainability: regression for numbers, boosted trees for tabular classification, anomaly detection for monitoring. In a government context I'd reach for classical first wherever the input is structured, because explainable models are easier to govern. And for a lot of 'is this over the threshold' logic, the honest answer is: that's not ML at all — that's a rule, and rules are even cheaper."
That last clause — knowing when it's neither an LLM nor ML but a plain rule — is the architect flex. Three tiers: rules where logic is known, classical ML where patterns hide in structured history, LLMs where language lives.