Recall@10 dropped after migrating from HNSW to IVF-PQ....

To diagnose the drop, you must separate the two potential sources of error: routing (the IVF component) and quantization (the PQ component). By setting nprobe equal to nlist on a small sample, you force an exhaustive search. If recall improves, the problem was that the query was not reaching the correct cells, which can be fixed by increasing nprobe or making cells coarser. If recall does not improve, the issue is the lossy nature of Product Quantization, which can be addressed by increasing the number of sub-quantizers (m) or adding an exact rescoring stage.