Why does RoPE make the attention score depend only on r...

The rotation matrix Rm applied to the query Q and Rn applied to the key K results in the inner product <Rmq, Rnk>. Because rotation matrices compose such that the transpose of Rm multiplied by Rn equals Rn-m, the resulting inner product simplifies to <Rm-nq, k>. This mathematical property ensures that the attention score is a function of the relative distance between tokens rather than their absolute positions.