Kaplan and Chinchilla disagree by more than a factor of...

The difference between the Kaplan and Chinchilla scaling laws is often misunderstood as a fundamental change in data or architecture. However, the primary drivers of the discrepancy were procedural.

First, previous measurements were biased downward because the learning-rate schedule was not properly matched to the total length of the training run. Second, the accounting of model size was adjusted to include embedding parameters, which were previously excluded. These corrections led to a more accurate understanding of how model performance scales with compute, data, and parameter count.