AI
Distillation
Training a smaller student model to reproduce the outputs of a larger teacher model, transferring capability at lower inference cost.
The student does not match the teacher on all tasks but can match or exceed it on the narrow distribution it is distilled on. This makes distillation a deployment strategy: train or buy a large model, then distill for production. The licensing question is whether the teacher's terms permit using its outputs as training data.