A competitor claims their model beats yours by 3 points...

To verify such a claim, first inspect the evaluation protocol, including prompt format, the use of chain-of-thought, and whether majority voting was applied, as these can easily manufacture a 3-point gap. Second, assess the statistical significance of the subset; 3 points on the full GPQA is meaningful, but the same on a small subset is likely noise. Finally, run your own evaluation on their model using a locked harness like LM-Evaluation-Harness to ensure the prompt format is identical.