Can you replace a 70B generator with a 7B model plus a vector store?

To evaluate this, split the problem: retrieval provides factual content, but the generator must still possess the reasoning competence to process the retrieved information. A simple experiment is to provide the gold chunk to the 7B model on failing queries to see if it succeeds. Note that per-query FLOPs often rise when switching to a smaller model with a large datastore, as the overhead of processing retrieved context can exceed the savings from a smaller parameter count. The primary benefits are memory reduction and lower update costs, not necessarily lower compute.