Retrieval metrics improved after you shipped document-side query expansion, but a stakeholder insists users still can’t find internal documents they know exist. Two engineers disagree on the fix: one wants to regenerate predicted queries with a larger model; the other wants to change the generation prompt. How do you adjudicate?

Before committing engineering resources to model or prompt changes, you must perform a diagnostic check to localize the failure. By pulling the specific document by its ID, you can determine if it is present in the candidate set retrieved by the system. If the document never appears in the top-k results, the bottleneck is in the indexing process. In this scenario, improving query generation—whether through a larger model or prompt engineering—will not solve the problem because the document is unreachable by the current retrieval configuration.