Risk & governance
Benchmark contamination
Test material leaking into model training or optimisation.
Also known as: data contamination
Definition
Contamination occurs when a model or development team has seen benchmark examples or close variants before the final evaluation.
Why it matters
Scores can reflect memorisation or benchmark-specific tuning rather than general capability.
Related concepts
- Test set
Data reserved for measuring performance after development.
- Held-out set
A private or untouched test set protected from optimisation.
- Leaderboard
A ranked table of systems evaluated under stated conditions.