What are the accuracy risks of CLA and how do you catch them in evaluation?

Cross-Layer Attention introduces an expressiveness trade-off regarding the value projections. To monitor for degradation, one should utilize retrieval-heavy benchmarks such as needle-in-haystack or multi-hop QA. While G=2 is generally considered safe, G=4 represents a threshold where rigorous testing is required to ensure that the model’s language modeling capabilities remain intact.