← All questions
EvaluationMedium· A/B Testing· Experimentation· Metrics

Your team ships a new reranker and macro nDCG improves...

Relying on a single aggregate number is insufficient to establish causality. By defining slices based on the expected influence of the reranker, you create a framework to verify that the observed improvements are actually coming from the component in question rather than external noise or unrelated system changes.

Code playground (Python)