Where do you look when the p95 time to first token is 1...

When addressing latency issues, it is critical to avoid premature optimization. Instead, decompose the latency budget across the five core inference decisions. Recognize that prefill latency scales as 2PT, whereas search latency scales sublinearly in N. Formulate a hypothesis about which decision is contributing the most to the latency mass, then define what evidence would falsify that hypothesis before proceeding to measurement.