179 questions
No questions match those filters.
← All questions
RAGVery Hard· Latency· RL· SLO· Optimization
Research wants a learned per-query stopper. Platform wants a hard cap of two rounds, because p95 is a contractual SLO and variance is the enemy. Adjudicate.
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansA fixed cap optimizes the tail of the distribution, while a learned stopper optimizes the mean but may widen the tail. If more than 15% of traffic requires three or more hops, a learned stopper will not help the p95. Furthermore, learned stoppers introduce OOD risks. The best approach is to use the data from an offline sweep to determine the optimal static cap and use prefix caching to handle deeper queries without violating the SLO.