Reporting and Continuous Improvement

The service is live, monitored, and processing submissions. The last discipline is turning its data exhaust into two things: reports that stakeholders actually use, and a feedback loop that makes the methodology itself better over time.

Operational reporting

Operational reports answer "is the service keeping its promises this week?" — throughput (submissions received, determinations issued), turnaround time against the published service level, queue depth by stage, and per-analyst workload for the review manager. These are the same metrics as the ops dashboards from Lesson 3, but aggregated and scheduled: a weekly summary to Commission management beats a dashboard nobody opens.

Serve them from the admin area with Kendo grids and charts over dedicated reporting queries — read-only, denormalized views or a nightly summary table, so a heavy report can never lock the tables analysts are working in. If leadership lives in Power BI, point it at those same views rather than hand-building every chart in MVC.

Program-level reporting

The more valuable layer is what only a statewide service can see:

• Gap patterns by subject — if 40% of candidates statewide show the same SMR domain gap, that's curriculum intelligence no single EPP could produce. It tells the Commission where preparation programs are thin and tells EPPs what to strengthen.
• Override rates and reason codes over time — the AI-quality trend from Module 11, now reported per methodology version: did version 3.2 actually reduce WrongDomain overrides?
• Consistency measures — spread of outcomes across analysts for similar coursework, the metric behind the service's core promise of uniform evaluation.

Every program-level report is aggregate and de-identified. The moment a report slices thin enough that "mathematics gaps at small EPP X in 2026" identifies a person, the PII rules apply. Design minimum-cell-size rules into the queries, not into reviewer vigilance.

Stakeholder exports

Different consumers, different shapes: EPPs get CSV/Excel exports of their own candidates' statuses and outcomes (row-level security enforced in the query — an EPP's export can only ever contain its own rows); the Commission gets quarterly aggregate packs; candidates already have the portable report from Module 10. Build exports on the same reporting views as the screens so a number in an export always matches the number on the dashboard — mismatched figures destroy trust faster than missing figures.

Closing the loop

This is where the whole design pays off. The versioned methodology (Module 10), the evidence- bearing findings (Module 11), and the override telemetry (Lessons 3–4) converge into a governance cycle: reports surface where the AI or the rules underperform → the Commission reviews proposed rule, threshold, or prompt changes → changes ship as a new methodology version through the pipeline from Lessons 1–2 → the next reporting period measures whether they worked. Past determinations stay explainable under their original versions; future ones get better. That loop — measured, versioned, human-governed improvement — is the difference between software that merely automates a process and a service a state commission can stand behind for a decade.