The Eligibility-Checker Design Round

The signature design exercise for a government AI role: "Members of the public should be able to upload a document — say, a transcript — and learn what they qualify for. Design it." This shape (document in, eligibility guidance out) recurs across licensing bodies, benefits agencies, and admissions. It's a perfect architect question because it layers document AI, deterministic rules, responsible-AI controls, and enterprise integration. Here's a worked answer, in the order you should say it.

Step 0 — restate and classify (30 seconds)

"We're building public-facing, consequential guidance from sensitive documents. Transcripts are federally protected education records, and the output shapes a person's licensing decisions — so this is a high-risk AI use case: privacy-first architecture, advisory-only output, human review, and measurable accuracy before launch." You've just scored the responsible-AI line before drawing a single box.

The pipeline, six boxes

1. Intake. Upload (PDF or image), malware scanning, encrypted storage, and an explicit retention policy decided with the records officer — how long uploads live and who can see them is a design decision, not an ops afterthought.

2. Extraction. OCR + document parsing into structured data with per-field confidence: institution, degrees, terms, courses, units, GPA. In an Azure shop, this is Azure AI Document Intelligence territory. Two facts to say out loud: transcripts have wildly inconsistent formats across thousands of institutions, so extraction is its own testable component with its own accuracy metrics; and low-confidence extraction must degrade gracefully — ask the user to confirm fields, or route to staff — never silently guess.

3. Deterministic matching first. Eligibility requirements become versioned, reviewable data — rules the subject-matter experts can read and correct — evaluated by a plain rules engine: degree present, unit thresholds, required examinations. No model where a rule suffices: cheaper, explainable, testable, and it never hallucinates. (This is usually the largest and most valuable work in the whole system — codifying requirements that today live in PDFs and experts' heads.)

4. AI where it earns its place. The genuinely fuzzy judgment: is "Human Development, Birth to Adolescence" equivalent to the required "Child Development" course? Use a language model grounded in the authoritative requirements corpus, with citations to the specific requirement relied on — and a relevance threshold below which the system says "needs human review" instead of improvising. Watch for retrieval over-matching (confidently citing a related-but-wrong source); scope the corpus to authoritative versioned documents and build eval questions that specifically try to trigger wrong-source answers.

5. Advisory output + human loop. Wording is architecture: "appears to meet," "may meet — verification required," "does not appear to meet," never "you are qualified." Every claim carries its citation. Low-confidence or contested results route to staff with the assembled context attached, so review starts warm. Users are told AI was involved and how to reach a human. Plan review capacity with the program area — an escalation path that lands on nobody's desk is a fiction.

6. Evaluation and monitoring. Before launch: a gold set built from historical staff determinations; measure extraction accuracy and matching precision/recall separately; the headline metric is false qualifications ≈ zero (accept more false "needs review" as the price). Run shadow mode first — the system silently processes cases staff are already deciding, and you measure agreement before anyone outside sees an answer. After launch: sampled human review, override rates, drift watch (new transcript formats, requirement changes), cost per document.

Cross-cutting lines that score

• Model routing follows data classification: protected documents argue for the department's government-cloud tenancy with contractual data-protection terms, or local inference; the public requirements corpus can use commodity services.
• Integration, not an island: the checker reads requirement data from (or alongside) the systems of record, publishes through the existing web platform's auth and accessibility standards, and logs to the enterprise audit stack.
• Accessibility and language access are launch criteria for public-facing state services, not enhancements.

Questions to ask the panel

Asking two of these marks you as the architect in the room:

1. "Is the output legally advisory, or could it create reliance or appeal exposure? Who owns that wording?"
2. "How much of the requirements corpus is already structured data versus PDFs and institutional knowledge? That codification effort usually dwarfs the AI work."
3. "What's the human-review capacity if we route low-confidence cases to analysts?"
4. "What's our approved procurement path for AI services — existing cloud tenancy, or do we need a new agreement?"

The close

"Phase one is the deterministic checker with structured extraction — that alone is a better public service than a PDF. AI-assisted equivalence comes second, shadow-tested against staff decisions. The system tells people what it knows, cites its sources, and hands the hard calls to humans." That's the whole design philosophy in three sentences — end there.