179 questions
No questions match those filters.
← All questions
Inference & ServingMedium· Attention Mechanisms· KV Cache
What is the difference between MQA, GQA, and MHA? When...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansMulti-Head Attention (MHA) provides maximum expressiveness but requires the most cache. Multi-Query Attention (MQA) uses a single KV head, which minimizes cache but often results in quality loss at scale. Grouped Query Attention (GQA) strikes the best balance by grouping queries to share KV pairs, making it the default choice for modern production models. MQA is largely considered obsolete for large models, while MHA is reserved for legacy or small-scale architectures.