RAG relies on in-context learning to make use of retrie...

In-context learning lets a model make use of new information placed in its prompt at inference time, with no weight updates at all — the “learning” is entirely a property of the forward pass, gone the moment the context window is cleared. That’s exactly the property RAG needs: retrieved passages are different for every single query, so the only way to make a frozen model use them is if it can genuinely learn from context on the fly. RAG is only possible because ICL exists — you obviously can’t run a fine-tuning job for every incoming request.

SFT does the opposite: it permanently updates the model’s weights based on training examples, so the resulting behavior persists across every future request without needing to be re-supplied each time. That makes it right for teaching stable, general behavior — format, tone, instruction-following, broadly applicable skills — but wrong for information that’s specific to a single query and would need a fresh training run to inject, which is far too slow and expensive to do live. The two are complementary: SFT shapes how the model behaves in general; ICL (via RAG or otherwise) is how it incorporates what’s true right now.