Your dense retriever beats BM25 by 19 points. Should yo...

The aggregate win of a dense retriever often hides a tail where its performance is near-zero, specifically for rare terms not well-represented in training. You should segment your evaluation set by rare-term frequency to identify the specific traffic at risk. A hybrid approach is necessary to cover the failure modes of dense retrieval.