← All questions
EvaluationHard· Long-form Generation· Calibration· Metric Drift

We moved from one-line answers to briefs. The confidenc...

The confidence score tracks E[S] = p, which remains constant regardless of document length. This is why coverage did not change. However, the harm perceived by users is related to Pr(S=1) = p^n, where n is the number of sentences. As n increases, the probability of the entire brief being correct drops significantly, even if the average sentence accuracy remains high, leading to an increase in complaints.

Code playground (Python)