179 questions
No questions match those filters.
← All questions
LLMOps & ProductionHard· Evaluation· Debugging· Model Upgrades
Accuracy dropped six points after a generator upgrade w...
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansStart by ruling out the retriever; since recall@k is computed against the retriever and it remains unchanged, it is likely not the source of the drop. Next, examine parse and refusal rates, as format-induced invalid outputs are frequently scored as wrong answers. Finally, perform a sweep of 20 formats on both the old and new models. The diagnostic goal is to determine if the incumbent format performed well on the old model but poorly on the new one, rather than simply comparing aggregate points.