Reinforcement Learning from Human Feedback uses human feedback to influence model behavior. Processes and feedback forms differ by method.
Demonstrations and preferences
A demonstration shows a desired response; preferences can compare outputs. One public research approach combines examples, preferences and reinforcement learning, but it is not the only feedback-training process.
Whose feedback?
Participant backgrounds, task descriptions and evaluation instructions shape feedback. Preferences of one sample may not suit every culture, domain or user. Explain coverage and conditions.
Pleasing and correct
A preferred answer may still contain factual errors. Preference training does not replace source checking, domain evaluation or consideration of the use.
Does RLHF fully align AI with all humanity? No. Training methods do not establish that every value disagreement and action risk has been solved.
Continue with Pretraining, fine-tuning and SFT: what differs? to compare related ideas.
For a related project perspective, explore LingoAI’s AI work.