If you had a fixed compute budget, how would you split...

The standard approach is to follow the Chinchilla optimal ratio of approximately 20 tokens per parameter. However, engineering judgment requires adjusting this based on the intended use case. If the model will be deployed at high request volume, the lifetime inference savings of a smaller model often outweigh the initial training costs, justifying an overtrained, smaller architecture.