← All questions
EvaluationHard· Generation· Positional Bias

Recall@k held steady - actually improved - after the de...

When recall@k holds steady but accuracy falls, the issue is likely in the generation stage. You should isolate the questions that are now being answered incorrectly and examine the position of the gold chunk within the assembled context. A common pattern is the U-shaped positional effect, where the model performs poorly on information placed in the middle of the context window. Confirm this pattern across the full eval set before attempting a fix.

Code playground (Python)