← All questions
Training & Fine-tuningMedium· SFT· Instruction Tuning· Model Behavior

What does SFT actually change in a pre-trained model?

Supervised Fine-Tuning (SFT) primarily modifies the surface properties of the output distribution, including format, register, and task framing. It does not add new world knowledge to the model. Instead, it acts as an installation of routing mechanisms that trigger the latent capabilities already existing within the pre-trained base model. This effect is significant even with small datasets, where quality is more important than quantity.

Code playground (Python)