← All questions
EvaluationHard· Ranking Metrics· Pairwise Learning· Loss Functions

Our pairwise reranker’s validation pair accuracy climbe...

A strong answer refuses to treat the two numbers as measuring the same thing: the pairwise objective weights every inversion equally, and nDCG@10 weights inversions by a discount that falls 426× between rank 1 and rank 50, so the model can be winning thousands of pairs it is not paid for. It then names the fix - multiply each pair’s gradient by the absolute change in NDCG - and the diagnostic that confirms it: histogram the newly corrected pairs by rank and check whether any of them cross rank 10. A weak answer proposes a larger model or more pairs, which are responses to underfitting; the model is not underfitting, it is fitting the wrong thing.

Code playground (Python)