Evaluation foundations
Evaluation suite
A coordinated collection of evals covering different capabilities and risks.
Definition
An evaluation suite groups tests for quality, safety, controls, cost, latency and operational behaviour into a repeatable assessment.
Why it matters
No single score captures production fitness; a suite makes trade-offs and critical failure modes visible.
Related concepts
- Eval
A structured test of whether an AI system meets a requirement.
- Critical-error rate
The share of cases containing a failure with severe consequences.
- Operational fitness
Whether a system works reliably within real service constraints.