Capacity, Latency, and SLOs

1. What is the recommended latency budget for an autocomplete hint in an AI application?

~100 ms ~300 ms ~1 second ~3 seconds

2. What is a key strategy to improve perceived speed in a chat application?

Increase the model's processing power Use additional data storage Implement streaming responses Optimize database queries

3. How should developers respond to hitting rate limits from a model provider?

Ignore the limits and retry immediately Implement exponential backoff with jitter Send requests continuously until succeeded Pause all operations temporarily

4. What is the purpose of establishing SLOs in AI features?

To enable faster transaction processing To maintain system uptime To formalize reliability standards and error budgets To enhance user interface design

Continue: How to Design an AI System → Review this lesson Retake quiz