Z-loss controls the output softmax normalizer. But cros...

While cross-entropy backpropagates through the softmax, it primarily pushes the correct logit up relative to others, without constraining the absolute scale of the logit vector. Z-loss provides additional information by penalizing the scale directly. Because the two losses operate on orthogonal degrees of freedom in logit space—specifically direction versus magnitude—they are complementary rather than redundant, helping to prevent numerical instability even when cross-entropy is minimized.