Nemotron-CC uses LLM rephrasing to recover low-quality...

Using an LLM to rephrase low-quality documents risks introducing hallucinations or systematic biases into the training corpus. To detect this, one should monitor for perplexity divergence, where rephrased documents with unusually low perplexity suggest fabrication. Additionally, n-gram overlap checks can identify if the rephrased text deviates too far from the original source. Finally, treat factual benchmarks like TriviaQA as a canary; if performance regresses, the rephrasing model is likely corrupting the data. In production, it is advisable to cap rephrased content at 10-15% of the total corpus.