Inference & optimisation
Sampling
Selecting generated tokens from the probability distribution produced by a model.
Definition
Sampling methods choose among likely next tokens using controls such as temperature, top-p or deterministic decoding.
Why it matters
Sampling settings influence variability and can make identical requests produce different financial or control outcomes.
Related concepts
- Temperature
A setting controlling randomness in a model's token selection.
- Top-p sampling
Sampling only from the smallest token set containing a chosen probability mass.
- Reproducibility
The ability to repeat an evaluation under documented conditions.