What is the RoPE compatibility problem with MLA and how...

Standard Rotary Positional Embeddings (RoPE) require applying position-dependent rotations to keys before they are cached. Because MLA caches position-agnostic latents, RoPE cannot be pre-applied. The solution is to cache a small, separate set of decoupled RoPE keys alongside the main latent cache, which preserves necessary positional information without significantly increasing the memory footprint.