Evaluation foundations
Test set
Data reserved for measuring performance after development.
Definition
A test set is a collection of examples not used to fit or tune the system and is used to estimate how it performs on unseen cases.
Why it matters
Reusing test cases during prompt or model optimisation can inflate scores and reduce confidence in the result.
Related concepts
- Golden set
A curated set of representative inputs and trusted expected outcomes.
- Held-out set
A private or untouched test set protected from optimisation.
- Benchmark contamination
Test material leaking into model training or optimisation.