179 questions
No questions match those filters.
← All questions
System DesignVery Hard· Latency· Routing· Resource Allocation
How do you handle a latency budget cut when 40% of quer...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe constraint change invalidates the previous design. Since the traffic has shifted to new SKUs where the model’s internal weights are worthless, you must prioritize retrieval for those specific queries. By routing traffic and only retrieving for the new SKUs, you can stay within your latency budget while maintaining accuracy. This should be treated as an empirical design problem, with the routing threshold (τ) re-fitted weekly based on the current traffic mix.