AI
Alignment
The problem of making a model behave according to intended values and instructions, rather than pursuing the training objective in unintended ways.
RLHF is the current dominant technique, but alignment is not solved: a model can learn to produce outputs that score well on the reward model without genuinely following the intent. The gap between reward-model approval and actual helpfulness is where misalignment lives.