Someone swapped Document 1: for [1] in your context template and grounded accuracy moved four points on a 500-question set. Real, or noise?

To determine if the change is real or noise, one must first calculate the noise floor. Given a sample size of 500 questions and an accuracy of approximately 64%, the standard error is calculated to be 2.1 percentage points. Because the observed four-point move is less than two standard errors, it cannot be considered statistically significant. Instead of immediately rolling back the change, the honest first step is to perform a paired test on the discordant questions to see if the difference holds up under closer scrutiny.