California AI Policy in One Lesson

For a state AI architect role, policy literacy is the differentiator almost no candidate brings. Engineers talk about models; architects in government talk about models inside a policy framework. This lesson gives you the working vocabulary — enough to answer "how would you make sure our use of AI is appropriate?" like someone who has read the actual documents.

Executive Order N-12-23 — where California's GenAI posture comes from

In September 2023, the Governor signed Executive Order N-12-23, directing state entities to study and prepare for generative AI. Its practical offspring shape everything a state department does with AI:

• A risk-benefit analysis of GenAI for state government, including guidelines for procurement and required use analysis.
• State GenAI guidelines and toolkits issued through the operational agencies — the Government Operations Agency, the California Department of Technology (CDT), and CalHR — covering procurement, acceptable use, and training.
• Inventories and reporting: departments are expected to know where GenAI is used in their operations, especially anywhere it touches decisions about members of the public.
• Pilots and sandboxes rather than uncontrolled adoption: try GenAI in controlled environments with evaluation criteria before production use.

What you say in the room: "Any AI feature I design for the Commission starts from the state's GenAI guidance — CDT's guidelines under Executive Order N-12-23 — which means a documented use case, a risk assessment keyed to whether the system affects the public, human review of consequential outputs, and the system showing up in our AI inventory rather than running as shadow IT."

The GenAI risk lens: does it touch the public?

California's guidance consistently draws the line at consequential decisions about people. An internal tool that summarizes logs for engineers is low-risk. A system that tells a member of the public what credential they qualify for is high-risk by definition — it shapes a decision about a person's livelihood. For high-risk uses, expect to say:

• Advisory, not determinative — the AI informs; an authorized human (or a deterministic, auditable rule) decides. Output wording matters: "appears to meet" is a different legal object than "you are qualified."
• Human review pathways — low-confidence or high-stakes outputs route to staff.
• Transparency — users should know AI was involved and how to reach a human.
• Auditability — you can reconstruct why the system said what it said (grounding, citations to source requirements, logged context).

NIST AI Risk Management Framework — the shared vocabulary

The NIST AI RMF 1.0 (January 2023) is the closest thing U.S. public sector has to a common AI-governance language, and California's guidance rhymes with it. Its four functions are worth knowing cold:

• Govern — the organizational layer: policies, roles, accountability, inventory. Who owns each AI system, who can turn it off, what's the review cadence.
• Map — establish context per use case: purpose, users, data sensitivity, what could go wrong for whom. (Your "does it touch the public?" triage lives here.)
• Measure — evaluate: accuracy against a gold set, bias where relevant, robustness, monitoring for drift. If you can't measure it, you can't claim it's safe.
• Manage — act on what you measured: mitigations, human-in-the-loop, incident response, the decision to not deploy.

You don't need to be an auditor. You need one fluent sentence: "I'd frame it with the NIST AI RMF — map the use case and its risks first, define how we'll measure accuracy and failure modes before launch, and manage with human review, monitoring, and a kill switch — under whatever governance structure the department already has."

Data classification: the non-negotiable layer

Government AI conversations end up at data. Know your categories and say them unprompted:

• Public records — e.g., published credential requirements. Safe RAG corpus material.
• Personally identifiable information (PII) — names, identifiers. Minimize what enters any prompt; never send it to services without an approved agreement.
• Federally protected education records — transcripts and student records carry FERPA-class obligations; background-check data carries its own strict handling rules. These shape architecture: where inference runs, what leaves the network, retention.

The architect-level move is connecting classification to model routing: sensitive-data workloads argue for enterprise agreements with contractual data-protection terms (e.g., a government cloud tenant where prompts aren't used for training) or for local/on-premises inference; public-data workloads can use commodity cloud AI. Saying "the data classification decides where the model runs" in an interview is worth more than naming ten frameworks.

Procurement reality

One more sentence that lands well: state entities generally can't just buy AI SaaS on a credit card. GenAI procurement runs through CDT-influenced processes with required risk disclosures. Architects who design around approved channels — the department's existing cloud tenancy, already-contracted platforms — ship; architects who design around whatever API was on Hacker News last week stall in procurement. Framing a design as "built on services we can actually procure" is public-sector judgment, and panels hear it rarely enough to remember it.