AI
Training
The process of adjusting a model's parameters by iterating over data, producing the weights that define the model's behaviour.
Pretraining on a large corpus gives general capability; fine-tuning on a narrow dataset specialises it. Training cost is measured in GPU-hours and is the dominant expense for foundation models - GPT-4 class training runs are estimated at tens of millions of dollars. The dataset determines what the model knows and what biases it carries.