Fine-tuning

Continuing training on a narrower dataset to specialise a pretrained model for a particular task or domain.

The cost is much lower than pretraining because the model already has general capability. The risk is catastrophic forgetting, where the model loses prior knowledge. LoRA and similar methods reduce cost further by training a small set of adapter weights rather than the full model.