How do you adjudicate between a Chinchilla-optimal 77B...

When faced with a conflict between research preferences for larger, compute-optimal models and infrastructure constraints, it is essential to move the conversation from training efficiency to operational economics. While the 77B model is indeed the better model at equal training compute, the scale of deployment changes the priority. At a volume of 10^10 tokens per day, the model will generate approximately 3.7 trillion tokens per year. At this scale, the inference and serving costs will significantly outweigh the initial training investment, making the smaller 8B model a more viable choice for the business.