Metrics That Matter

1. What does a rising refusal rate typically indicate?

An increase in user satisfaction A potential content gap Improvement in model performance Reduction in latency

2. Why is p95 latency an important metric to track per provider and model?

It averages all users' experiences It captures the best possible performance It reflects the experience of the slowest users It prevents data over-extraction

3. Which of the following describes the purpose of tracking token spend per feature?

To measure model accuracy To identify high-cost features To determine user engagement To understand model grounding

4. What does a shift in the local-vs-cloud mix indicate?

Improved AI performance User satisfaction increase Potential cost and latency implications Reduced token spending

Continue: Evaluation Sets and Regression Testing → Review this lesson Retake quiz