179 questions
No questions match those filters.
← All questions
EvaluationHard· Statistics· Experimentation· Metrics
Your target bucket improved by 15 points on n = 100, but your control bucket - queries the new component should never touch - also moved by 4 points. Do you ship?
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansIt is critical to distinguish between signal and noise. When the control bucket shows movement, it suggests that the observed changes might be due to sampling variance. Performing a statistical check on the standard error helps prevent making decisions based on unreliable data, highlighting the need for more samples to reach a definitive conclusion.