179 questions
No questions match those filters.
← All questions
EvaluationMedium· LLM-as-judge· Bias· Evaluation
What is the main failure mode of LLM-as-judge evaluations?
This is one of the questions in the full AI/ML interview bank. Pro unlocks all 1789 questions; Premium includes the same bank plus the highest daily Practice limit.
See plansLLM-as-judge evaluations are prone to biases inherited from the judge’s training data. For example, GPT-4 judges often favor longer responses (verbosity bias) or outputs that stylistically resemble their own training data (style bias). While verbosity can be corrected via length-controlled win rates, style bias is harder to mitigate. Position bias, where the judge prefers the first response, is typically addressed by randomizing the order of responses.