Vocabulary: Classical Machine Learning

Supervised learning — training on historical rows where the outcome (label) is known, so the model learns to predict outcomes for new rows. Lives in: classical ML; the family covering most government prediction problems. Say it: "Five years of history means the labels already exist — that's supervised learning, not anomaly detection."

Unsupervised learning — finding structure in data with no labels: clustering, anomaly detection. Lives in: classical ML. Say it: "No labels? Then we're in unsupervised territory — grouping or flagging the unusual, not predicting a known outcome."

Label — the known outcome attached to a historical record: which applications needed specialist review, how long each took. Lives in: supervised learning; the thing you're teaching the model to predict. Say it: "The label quality caps the model quality — mislabeled history trains a confidently wrong model."

Feature — an input variable the model uses: application type, unit count, submission date. Lives in: the model's input side. Say it: "Feature engineering — choosing and shaping the inputs — usually moves accuracy more than model choice."

Regression — predicting a number: processing days, expected workload. Lives in: supervised learning. Say it: "Predict-how-long is regression; linear regression is the explainable baseline, boosted trees the workhorse."

Classification — predicting a category: needs-review or not, complete or incomplete. Lives in: supervised learning. Say it: "Flagging applications is binary classification — and then precision and recall are the whole conversation."

Gradient-boosted trees — ensembles of decision trees (XGBoost, LightGBM); the workhorse for tabular data, routinely beating neural networks on rows-and-columns problems, cheap to train and run. Lives in: classical ML model choice. Say it: "For tabular data I reach for gradient-boosted trees before anything neural — better accuracy, millisecond inference, explainable."

Clustering — grouping records by similarity without labels. Lives in: unsupervised learning. Say it: "Clustering our failure records could reveal families of root causes we haven't named yet."

Anomaly detection — learning "normal" and flagging departures; needs no labels. Lives in: unsupervised learning; monitoring's ML upgrade. Say it: "A job that succeeds in two seconds when it normally takes five minutes is an anomaly — success status can lie, baselines don't."

Train/test split — evaluating only on data the model never saw; for time-series, hold out the most recent period, because you predict the future from the past. Lives in: evaluation discipline; the first question to any vendor. Say it: "Show me performance on a held-out recent period — accuracy on training data just measures memorization."

Precision — of the items the model flagged, the fraction that were actually right. Your staff's wasted-work rate. Lives in: classification evaluation. Say it: "Low precision means analysts chase false alarms until they stop trusting the tool."

Recall — of the items that truly should be flagged, the fraction the model caught. Your risk-exposure rate. Lives in: classification evaluation; precision's counterpart. Say it: "For an eligibility system the recall-side failure is the dangerous one — so we tune to keep false qualifications near zero and accept more 'needs human review.'"

False positive / false negative — flagged-but-wrong versus missed-but-real. Decide which is worse before building; it determines every threshold. Lives in: evaluation and risk framing. Say it: "Step one is naming the worst failure direction — the metrics follow from that, not the other way around."

Overfitting — a model that memorized its training data and fails on new data; the disease train/test splits exist to detect. Lives in: evaluation discipline. Say it: "Great training accuracy plus poor test accuracy equals overfitting — it learned the past, not the pattern."

Explainability / feature importance — the ability to say why the model decided: which features drove a prediction. Trees give this nearly free; deep models mostly don't. Lives in: responsible AI meets model choice; decisive in government. Say it: "When the auditor asks why application X was flagged, a tree model gives feature importances; that's often the deciding argument for classical ML."

Baseline — the simplest credible approach (a rule, an average, linear regression) that any fancier model must beat to justify its complexity. Lives in: evaluation discipline; the honesty check. Say it: "First question for any proposal: what's the baseline, and by how much does the model beat it?"

Rules vs. ML vs. LLM — the three tiers — deterministic rules where logic is known; classical ML where patterns hide in structured history; LLMs where language lives. Lives in: architecture judgment — the decision rule above all tool choices. Say it: "Rules where the logic is known, classical ML for structured history, LLMs for language — and a lot of 'is it over the threshold' logic is honestly just a rule."