Your offline eval score dropped 8 points after a routine deploy. What is the first thing you check?

Before investigating the embedding model or the LLM, you should recompute retrieval recall@k for both the old and new pipelines. Since the regression could originate in any stage of the RAG pipeline, this measurement allows you to quickly rule out the retrieval stage, which is often the most significant contributor to performance drops.