Would you serve a 1.3B model instead of a 7B model to r...

While smaller models might theoretically reduce some exposure, the trade-off is poor. Deduplication delivers significantly better results in terms of emissions and perplexity. Furthermore, shrinking the model does not fix issues with heavily duplicated text, which remains extractable regardless of the model scale. If you did not train the model yourself, the focus should shift to managing the boundary between the model weights and the datastore.