← All questions
EvaluationHard· Benchmarking· Metrics· Evaluation

Your AlpacaEval LC win rate improved 5 points but Chatb...

A discrepancy between AlpacaEval and Chatbot Arena often arises because the two metrics sample different prompt distributions. You should inspect which prompt categories drove the AlpacaEval gain; if they are concentrated in constrained-format tasks that are rare in the Arena, the metrics are measuring different things. Additionally, check for response length; if the model is significantly longer, the LC correction may not fully account for a ‘fluency halo’ effect that inflates AlpacaEval scores without improving real-world ELO.

Code playground (Python)