← All questions
System DesignVery Hard· Reranking· Latency· Query Routing

Search wants depth 1,000 for long-tail queries. Platform wants 50 to hold p99. Both have data. Adjudicate.

Find the variable they disagree about, which is the slope of their recall curve on the long-tail slice. If long-tail queries are served by an exact-match-heavy sparse path, their slope is BM25-like and depth buys significant points, meaning Search is right for that slice. If the pipeline is document-bound rather than model-bound, both proposals are aimed at the wrong stage. Price the alternative: at low epsilon, depth 1,000 is only safe if the number of candidates is large, so hybrid fusion is often a more efficient fix.

Code playground (Python)