← All questions
EvaluationMedium· Retrieval· Metrics

Recall@20 on your eval set is meaningfully higher than...

Recall@k is a measure of retrieval coverage, but it does not account for the generator’s performance limitations. As you increase k, you increase the likelihood of the gold document being in the context, but you also increase the likelihood that it is placed in the ‘trough’—the middle of the context where model attention is weakest. If the increase in recall does not compensate for the loss in positional accuracy, the overall system performance will decline. Therefore, one must balance recall with the generator’s positional sensitivity.

Code playground (Python)