179 questions
No questions match those filters.
← All questions
RAGHard· Attention Analysis· Model Debugging
The answer position puts 0.9 of its attention mass on t...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansSince the QK circuit successfully placed attention mass on the correct chunk, the retrieval mechanism performed its job. To diagnose the issue, one should permute the context to confirm the error is invariant, then patch the residual stream at the answer position from a clean run. To distinguish between an OV write error and a later MLP override, one must decode the residual stream layer by layer to identify where the correct entity stops leading.