179 questions
No questions match those filters.
← All questions
Inference & ServingHard· Memory Management· Fine-tuning· PEFT
Walk through the memory requirements for fine-tuning a...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansFull fine-tuning requires 12 bytes per parameter for Adam, totaling 84 GB, which exceeds the 24 GB limit. By freezing the base model, you reduce the footprint to 14 GB. Adding adapter state at r=16 adds ~0.1 GB, and activations at a sequence length of 4,096 add ~1 GB, totaling 15.2 GB. The critical constraint is that activations scale with b * s, and RAG sequences are inherently long due to retrieved documents, which can quickly exhaust remaining memory.