Training & post-training
Pretraining
Initial large-scale training that gives a foundation model broad capability.
Definition
Pretraining exposes a model to large datasets and teaches it general statistical patterns, often through next-token or reconstruction objectives.
Why it matters
Banks rarely control pretraining data, so provenance, intellectual-property, bias and memorisation remain vendor-assurance questions.
Related concepts
- Foundation model
A large pre-trained model adaptable to many downstream tasks.
- Model training
The process of adjusting model parameters using examples and an objective.
- Model provenance
Traceable information about where a model, its data and changes came from.