Models & architecture
Transformer
The neural-network architecture underlying most modern language models.
Definition
A transformer uses attention mechanisms to process relationships between tokens in parallel and build contextual representations.
Why it matters
Transformers enable modern LLMs, but architecture labels alone say little about accuracy, cost, controls or use-case fitness.
Related concepts
- Large language model
A foundation model trained to understand and generate sequences of language tokens.
- Attention
A mechanism that weights which parts of an input matter for each output.
- Context window
The maximum information a model can process in one interaction.