Walk me through the infrastructure for a GRPO training...

The infrastructure for GRPO at scale necessitates a clear separation between rollout workers and trainer workers. Key architectural components include one-way weight broadcasting per outer iteration and the ability to reuse rollouts by recomputing log-probabilities for inner steps. Reward evaluation should be decoupled from the generation process. Engineers must also account for vLLM integration and decide between teardown-vs-hot-update strategies for worker management.