179 questions
No questions match those filters.
← All questions
Training & Fine-tuningMedium· Memory Management· Fine-tuning· Hardware Constraints
Can we fine-tune a 27B model on one 80GB GPU?
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansA 27B model requires 108GB just for the forward pass at fp32, which already exceeds the 80GB capacity of a single GPU. When accounting for training with the Adam optimizer, the memory requirement jumps to approximately 432GB. To make this feasible, one must employ specific levers: using bf16 precision halves the base memory requirement, LoRA significantly reduces the memory needed for parameter updates, and activation checkpointing trades off compute cycles to reduce activation memory usage by approximately 33%.