Why is the empirical scaling exponent so much smaller t...

The -1/√n rate is the standard non-parametric rate for estimating a scalar or fitting a one-dimensional function. Language modeling, however, is a high-dimensional task where the exponent is -1/D. Because language modeling has very high effective dimensionality, the slope of the scaling curve is significantly shallower than the -1/√n rate.