Explain the KV cache and why generation has different GPU utilization characteristics than prefill.

During prefill, the GPU processes all prompt tokens in parallel using large matrix multiplications, making it compute-rooflined. During generation, the model produces one token at a time while attending to cached KV pairs, resulting in matrix-vector multiplications that are memory-bandwidth-rooflined. The KV cache stores projections for every prior token, with size scaling based on sequence length, head count, and head dimension.