How do you choose a chunk size?

Choosing a chunk size is a downstream decision based on the specific requirements of your corpus. You must first identify the atomic unit that needs to survive intact, such as a table row, a function, or a clause. After identifying this unit, you should apply severance arithmetic, defined as (s - 1)/L, to determine the split. Additionally, you must ensure an overlap guarantee where s is less than or equal to o, acknowledging that this increases the total length of the data by a factor of L/(L - o).