← All questions
LLM FoundationsMedium· Scaling Laws· Data Mixture

If you changed the training data mixture—say, you doubl...

Changing the data mixture primarily shifts the offset of the scaling curve rather than the slope α, as the slope represents the intrinsic difficulty of the learning task. Increasing the fraction of code will lower the loss floor for code-related benchmarks but may raise it for other domains. This offset shift can be measured and extrapolated reliably by training smaller models.

Code playground (Python)