DPO vs RLHF: which should you use to align a model?
Start with DPO: same preference labels, no reward model or sampling loop. Switch to RLHF with PPO when you need online exploration. Full cost math.
dpo
2 posts with the tag “dpo”
PPO vs DPO vs GRPO: which RL method should you use?
Pick by the signal you have: preference pairs mean DPO, a checker you can run in code means GRPO, a learned reward model means PPO. Memory and cost math inside.