179 questions
No questions match those filters.
← All questions
Inference & ServingHard· Scaling Laws· Cost Optimization
How does the IsoFLOP result change if you account for i...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansThe standard Chinchilla result minimizes training loss per FLOP, ignoring the lifecycle cost of the model. If a model is expected to serve R inference requests, the total cost becomes 6ND + 2NR. When fixing the total tokens processed, the cost-minimizing parameter count N is smaller than the compute-optimal N. Consequently, production models are often trained on significantly higher token-to-parameter ratios (100-2,000+ tokens per parameter) to optimize for total cost rather than just training efficiency.