An audit reports mean token overlap with copyrighted wo...

Evaluating copyright risk using mean token overlap is fundamentally misleading. Model memorization follows a bimodal distribution: the vast majority of generated sequences exhibit zero to low baseline overlap, while a tiny tail of inputs triggers high-confidence sequence lock-on, yielding massive verbatim emissions. Averaging these outcomes dilutes extreme violations and masks serious legal exposures.

To accurately evaluate exposure, move away from mean overlap metrics. Test the model using prefix prompts extracted directly from protected catalog sources and measure the maximum length of contiguous matching token runs. Use high-percentile run lengths (such as p99) as a metric for continuous model evaluation, and set a hard maximum contiguous sequence cap as a strict release gate.