← All questions
RAGMedium· RAG & Retrieval

You're chunking policy documents. What's your strategy?

Fixed-size chunking ignores the document’s own structure and routinely splits a clause from the heading that makes it findable — you end up with a chunk that contains the answer but not the context a retriever needs to surface it.

My approach:

  • Chunk on natural boundaries where the source has them — headings, sections, numbered clauses — rather than an arbitrary character count.
  • Keep enough overlap between chunks to avoid severing a sentence or a cross-reference mid-thought.
  • Attach metadata to every chunk — section, document type, effective date — so a query can filter before it searches, not just rank after the fact.

The failure mode to actively watch for in testing: a chunk that reads correctly in isolation but has lost the heading or clause number that made it retrievable in the first place.

Code playground (Python)