← All questions
System DesignVery Hard· Generative Retrieval· RAG· Scalability

Adjudicate the debate between using RAG for small models versus generative retrieval for large models, and explain what changes if the corpus grows from 10^6 to 10^8 chunks.

The debate is resolved by the three-store split: the payload, the index, and the model. RAG is effective because it keeps the payload outside the weights. Generative retrieval moves the index inside, which increases the cost of the write path. If the corpus grows to 10^8, the query path remains manageable due to logarithmic identifier growth, but the parametric capacity falls, and the rebuild time increases from minutes to days. The optimal design uses the parametric path only as a candidate generator, not a replacement for retrieval.

Code playground (Python)