Your flat index is 30.7 GB and you have 1 GB of RAM for it. Same embedding model, same dimension. What do you do?

First, convert the memory budget into bytes per vector. With 1 GB of RAM for 10 million vectors, you have a budget of 100 bytes per vector. Using Product Quantization (PQ) with k=256, you need to select a number of segments ‘m’ such that m ≤ 100. For a 768-dimension embedding, choosing m=96 allows the index to fit within 960 MB. It is critical to include a rescoring stage to maintain retrieval accuracy after the initial approximate search.