Scoring & evidence
Pass@k
Whether at least one of k attempts successfully completes a task.
Also known as: pass at k, Pass@1
Definition
Pass@k estimates the probability of obtaining a passing result when a system is allowed multiple independent attempts.
Why it matters
Pass@k can overstate real-world reliability if production users receive only one answer; compare it with Pass@1.
Related concepts
- Leaderboard
A ranked table of systems evaluated under stated conditions.
- Reproducibility
The ability to repeat an evaluation under documented conditions.