← All questions
EvaluationHard· Stakeholder Management· Metrics· Graded Relevance

Search wants to standardize the team’s dashboards on MR...

Using MRR for decision-making requires binarizing graded relevance judgments, which discards the valuable signal collected by the relevance team. Instead, use nDCG@k for technical decision-making as it respects the graded nature of the data. To satisfy stakeholder needs, maintain a binarized MRR dashboard for high-level summaries, ensuring the binarization threshold is clearly documented to prevent drift between the two metrics.

Code playground (Python)