179 questions
No questions match those filters.
← All questions
Training & Fine-tuningMedium· RAG vs Fine-tuning· Catastrophic Forgetting
Why is fine-tuning on documentation a poor substitute f...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe update objective in fine-tuning lacks a term for the old distribution, leading to catastrophic forgetting. Furthermore, the compute cost of updating weights (6Ngen FLOPs) is vastly higher than updating a retrieval index (2Nemb). While fine-tuning is excellent for controlling the model’s tone and format, it is not an effective mechanism for keeping knowledge current.