179 questions
No questions match those filters.
← All questions
System DesignVery Hard· Retrieval-Augmented Generation· Model Sizing· Inference Optimization
Our research lead wants the Chinchilla-optimal 77 B bec...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansConcede that the 77B is better at equal training compute, but point out that ‘optimal’ is being calculated on the wrong integral given the massive inference volume. The decision hinges on whether the extra parameters provide value beyond what the retrieval index already supplies. Propose an experiment at small scale to see if the gap is in recall (which retrieval fixes) or multi-step reasoning (which might justify the larger model).