179 questions
No questions match those filters.
← All questions
Inference & ServingMedium· KV Cache· Memory Optimization· Model Architecture
What architectural lever reduces KV cache footprint wit...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansCross-Layer Attention (CLA) is an architectural lever that reduces the KV cache footprint by sharing KV projections across layers. By halving the number of independent KV slices, you reduce cache memory usage, which compounds with existing Grouped Query Attention (GQA) implementations. A common rule of thumb is to use a grouping factor of G=2 to achieve significant memory savings while maintaining model performance.