Does a larger context window make the reranker obsolete?

While a larger context window allows for more chunks, the marginal value of additional documents decreases as the count grows. Additionally, ‘free’ window space is not free in terms of compute; processing ninety-five extra chunks adds significant latency per query. Finally, model accuracy is not monotonic with context length; placing the answer in the middle of a large context can actually result in worse performance than a smaller, well-ranked set of documents.