← All questions
Retrieval & EmbeddingsHard· IVF Index· Latency· Performance Debugging

Your IVF index holds p50 at 5 ms but p99 at 60 ms, at f...

The work per query is the sum of the probed list lengths, which varies because k-means on a clustered manifold does not produce equal-sized cells. To diagnose, pull the list-length histogram to confirm if slow queries hit oversized cells. Solutions include splitting oversized cells, retraining with more centroids, or capping the per-query scan budget. Increasing nlist alone is insufficient as it shifts costs to the first stage of the search.

Code playground (Python)