Your REALM-style run’s retrieval recall plateaus at ste...

When recall plateaus while LM loss improves, the retriever is no longer learning. This happens if the masking policy drifts toward tokens predictable from local context, causing the retrieval signal to collapse, or if the index is stale, meaning the retrieved documents (Zk) are not what a fresh search would return. Staleness biases the negatives toward blocks that used to score highly, which is a training-distribution problem rather than a latency issue.