Your p95 time to first token is 1.1 seconds against an...

To debug a TTFT latency breach (1.1s vs an 800ms SLO), avoid making blind optimizations. Decompose the end-to-end latency budget into its constituent components: network transit, query embedding generation, vector store retrieval, reranking, and model prefill computation.

Vector search scales sublinearly with database size ($N$), whereas prefill computation scales linearly or quadratically with prompt token length ($P$). In typical setups, vector search should account for less than 15% of the latency budget. If measurement reveals retrieval exceeds this baseline, the vector index is misconfigured (e.g., missing index, unoptimized HNSW settings). If retrieval is normal, the bulk of TTFT latency resides in prefill, requiring reductions in context length, context pruning, prompt compression, or model acceleration (e.g., flash attention, KV caching).