What compute do you need for RL training of LLM agents?
Agent RL needs GPUs for generation and training plus one isolated sandbox per live rollout. Cost math showing GPU wait time, not sandboxes, drives the bill.
grpo
3 posts with the tag “grpo”
PPO vs DPO vs GRPO: which RL method should you use?
Pick by the signal you have: preference pairs mean DPO, a checker you can run in code means GRPO, a learned reward model means PPO. Memory and cost math inside.
What is GRPO, and how do you run GRPO training on GPUs?
GRPO trains an LLM by scoring groups of sampled answers against each other, with no critic model. How it works, memory math for 8B, and how to run it.