Why does naively chunking a table for RAG corrupt it, a...

A table’s meaning lives entirely in the relationship between a cell and its row and column headers — “47” means nothing on its own, it means something specific because it sits at the intersection of a particular row and column. Naive text chunking splits by character count or paragraph boundary with no awareness of that structure, and it’s entirely possible for a chunk to contain rows of numbers with the header row that gave them meaning left in a different chunk entirely. The retriever can then return a technically-relevant chunk that’s functionally useless, because the numbers in it can no longer be correctly attributed to anything.

The fix is treating tables as a distinct data type rather than prose: either keep the header row explicitly attached to every chunked subset of a table’s rows (so each chunk is self-contained and correctly attributable), or — for genuinely numeric, lookup-heavy tables — extract the table into a structured store (even just typed rows in a database) and query it directly rather than through text chunk retrieval at all. Similarity search over prose chunks is the wrong tool for precise tabular lookups; it’s built to find relevant text, not to guarantee a correct row/column attribution.