← All questions
Training & Fine-tuningMedium· Budgeting· Data Strategy· SFT

You have a limited annotation budget of $50,000. How do...

For a 7B model SFT run, you should leverage model-generated data from a strong API model to achieve broad coverage, as this can provide 500,000 to 1,000,000 examples at near-zero marginal cost. The limited human budget should be reserved for high-value tasks, such as reviewing 5,000 to 10,000 borderline and safety-critical examples. Finally, any remaining gaps in structured-task coverage should be addressed by incorporating aggregated NLP data.

Code playground (Python)