RLHF (Reinforcement Learning from Human Feedback)
A training technique that uses human preferences to align AI behavior with human values and intentions.
AI-generated
RLHF is the process of fine-tuning a language model using human feedback. Humans rank model outputs from best to worst. These rankings train a reward model that scores responses. The language model is then optimized (via reinforcement learning) to produce higher-scoring responses.
RLHF is why modern AI assistants are helpful rather than just text prediction engines. Without RLHF, a model trained to predict text might produce toxic, unhelpful, or irrelevant outputs. RLHF aligns the model with human preferences. ChatGPT's launch success was largely attributed to the RLHF training that made GPT-3.5 feel like a helpful assistant.
Ouyang et al., "Training language models to follow instructions" - https://arxiv.org/abs/2203.02155
Chip Huyen: "RLHF" - https://huyenchip.com/2023/05/02/rlhf.html