← All questions
EvaluationHard· NDCG· Learning to Rank· Loss Functions

You have query–document pairs graded 0 to 4 by humans,...

The core problem is that nDCG is a function of the relative ordering (argsort) of the scores rather than their absolute values. Because of this, a model can achieve a perfect nDCG of 1.000 while still maintaining a high mean squared error. Consequently, the training loss can decrease significantly without producing any meaningful improvement in the actual ranking quality, as the model is optimizing for absolute values that do not impact the final metric.

Code playground (Python)