How would you approach SFT if your instruction data was...

When using a stronger teacher model, there is a risk of distribution mismatch: the teacher’s responses may require reasoning or knowledge the student model does not possess. If the student is forced to produce tokens it cannot faithfully generate from its own weights, it learns to imitate the structure of the response while hallucinating the content. The mitigation strategy is to filter the teacher-generated data, conditioning the SFT examples on whether the student model, when probed, already demonstrates the underlying knowledge required to support the response.