← All questions
Training & Fine-tuningMedium· Data Diversity· Long Context

Your book corpus is 90% genre fiction. What does that m...

A corpus dominated by genre fiction teaches the model long-range narrative patterns such as character tracking, dialogue attribution, and scene transitions. However, it fails to expose the model to the specific dependency structures found in technical writing, legal documents, and formal mathematics. Consequently, the model may be able to retrieve story facts from 80k tokens but will fail to track variable definitions or logical arguments across long, complex documents.

Code playground (Python)