How do you balance retrieval recall and generation cove...

Do not treat retrieval and generation as a single monolithic request. Fan-out for recall is relatively cheap, while passing a large k (e.g., k=20) to the generator is expensive due to prefill costs. By retrieving a larger set (e.g., 50), reranking them, and presenting a smaller subset (e.g., 5) to the generator, you maintain quality while staying within the latency SLO.