Inference & optimisation
Top-p sampling
Sampling only from the smallest token set containing a chosen probability mass.
Also known as: nucleus sampling
Definition
Top-p, or nucleus sampling, limits generation choices to tokens whose cumulative probability reaches a configured threshold.
Why it matters
It changes output diversity and should be fixed and recorded when comparing models or rerunning regulated evaluations.
Related concepts
- Temperature
A setting controlling randomness in a model's token selection.
- Sampling
Selecting generated tokens from the probability distribution produced by a model.
- Reproducibility
The ability to repeat an evaluation under documented conditions.