How do you get the accuracy back after structured pruni...

Structured pruning inherently leaves a model with lower accuracy than the original. Knowledge distillation is used to recover this performance by training the student to match the teacher’s full probability distribution over the vocabulary using forward KL divergence, often mixed with a small cross-entropy term. This process is significantly more efficient than training from scratch because the pruned model’s weights are already near the target loss, meaning the training process only needs to correct a small perturbation.