Models & architecture
Mixture of experts
An architecture that activates selected specialist sub-networks for each input.
Also known as: MoE
Definition
A mixture-of-experts model routes each token or request through a subset of expert components rather than activating every parameter.
Why it matters
It can provide high capability with lower inference compute, while introducing routing complexity and less transparent capacity comparisons.
Related concepts
- Model parameters
The learned numerical values that determine a model's behaviour.
- Inference
Running a trained model to produce a prediction, answer or action.
- Model routing
Selecting different models based on task, risk, cost or availability.