A model fine-tuned on our data produces outputs that ar...

To fix overly long outputs, you should audit your SFT data for length distribution and potential human or AI curation bias. The most effective solution is to explicitly add short-answer examples to the training set. If you are using preference data to filter SFT examples, ensure that length is controlled when computing preference scores. Relying on a system prompt to enforce brevity is merely a band-aid and should be avoided.