Matching Coursework to SMRs

The intellectual core of the service: given clean coursework rows, propose which courses provide evidence toward which Subject Matter Requirement domains. No single technique does this well alone, so the design is a hybrid pipeline — rules, then semantics, then language — where each stage only handles what the previous one couldn't.

Stage 1: deterministic rules

Some matches are certain and should never burn an AI call. The Commission (and years of analyst practice) can enumerate them: articulation agreements, known course-code patterns (MATH 1xx at CSU campuses → mathematics content domains), and previously human-confirmed pairs of (institution, course code) → SMR. Encode these as data-driven rules in SQL Server, keyed to the methodology version. Rule hits produce findings with confidence 1.0 and evidence "matched rule R-041." Fast, free, explainable — and they establish the shape every later stage must also produce.

Stage 2: embeddings for semantic similarity

For the long tail, compare meanings: embed each course title + catalog description and each SMR domain description as vectors, and rank candidate pairs by cosine similarity. From .NET this is an HTTP call to an embeddings API (Azure OpenAI on this stack); store vectors in SQL Server (a varbinary column and a simple similarity computation is fine at this scale — you're comparing one transcript against a few hundred SMR nodes, not doing web-scale search).

Embeddings catch what rules can't: "Statistical Reasoning in Everyday Life" lands near the statistics domain without anyone writing a rule. But similarity is not evidence — a high score is a candidate, not a finding.

Stage 3: LLM-assisted rationale

Top candidates go to an LLM with a tightly-scoped prompt: the course description, the SMR domain text, and an instruction to judge alignment and explain why, returning structured JSON:

{ "aligned": "partial", "confidence": 0.72,
  "evidence": "Covers descriptive statistics and sampling;
   no inferential methods, which SMR 4.2 requires." }

Deserialize into your AlignmentFinding, validate the schema, and reject malformed output (fail-closed — a finding that can't be parsed is a finding that doesn't exist). The LLM's value is the rationale: analysts review evidence far faster than they research from scratch.

Every finding carries evidence and confidence

Whatever stage produced it, a finding persists the same fields: source stage (rule / embedding / LLM), confidence, evidence text, and the input snapshot it was computed from. Below-threshold findings surface as "needs review," never pre-accepted. This uniformity is what makes the workbench UI (next lesson) simple — the analyst sees one list, sorted by confidence, regardless of machinery.

Consistency via the versioned methodology

The whole pipeline — rule set, embedding model, prompt text, thresholds — is part of the methodology version from Module 10. Change any of it and you cut a new version; in-flight submissions finish on the version they started with. That's what lets the Commission say the magic sentence: every transcript is evaluated by the same method, and we can name the method.