← All questions
Training & Fine-tuningHard· Reinforcement Learning· GRPO· Mathematics

Is dividing the group advantage by the standard deviation in GRPO mathematically valid?

While subtracting the group mean is a valid baseline, dividing by the standard deviation (σ) is problematic. When σ approaches zero, gradients explode. This normalization gives disproportionate weight to problems at the extremes of the difficulty spectrum, and analysis shows that removing this normalization can lead to equal or better performance.

Code playground (Python)