179 questions
No questions match those filters.
← All questions
System DesignVery Hard· Long Context· KV Cache· Architecture
You are designing a new model targeting 128K context se...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansBegin by defining your memory budget (e.g., 80GB per GPU, with a portion reserved for weights). Calculate the per-token KV footprint for MHA, GQA, and MLA at your target context length and head dimensions. For 128K context, even GQA may exceed VRAM limits, whereas MLA can reduce the footprint to a fraction of the budget. If the model is intended for high-scale deployment, the engineering investment in custom kernels for MLA is justified by the long-term serving economics and its superior scaling with sequence length.