Your generator is behind an API that returns no token p...

Asking the model to state its confidence is a common baseline, but it is limited because the output is a token from the same distribution as the answer, often resulting in a narrow range of values (e.g., values above 0.9). This limits the number of possible cutoffs. A more robust alternative is to generate m samples and calculate an agreement score. This provides m+1 levels of resolution but requires m times the generation cost and potentially additional latency if parallelized.