You’ve shown me the U-curve. Where does this bias actua...

The U-shaped curve is not an unavoidable mathematical property of the transformer architecture. Instead, it is a result of the specific training distributions used to create the models.

Primacy is largely driven by the system message being fixed at position 0 during instruction tuning and alignment. Recency is driven by the pre-training objective, which naturally prioritizes nearby context for the next-token prediction task. Both are artifacts of how models are trained, rather than fundamental limitations of self-attention.