← All questions
System DesignHard· Retrieval· Latency· Data Processing

Our corpus is 50 GB of PDFs and we want sub-second answ...

At 50 GB, you are looking at approximately 42 million chunks, which exceeds the capacity for a single HNSW node at FP32. IVF-PQ is a more viable candidate for memory constraints. However, the real design risk is the generative decode time, which will consume most of the sub-second budget, making answer length and model size critical factors.

Code playground (Python)