← All questions
RAGHard· System Design· Latency· Compliance

Legal requires clause citations in chunks, but Platform...

The conflict is based on a misunderstanding of ANN latency. ANN search cost is independent of the text length that produced the vector; it depends on the number of vectors (N), dimensions (d), and graph parameters. Variable-length chunks only impact the generator’s prefill budget. You can satisfy Legal by using clause-aligned chunks and satisfy Platform by capping the returned span to bound the tail of the prefill latency.

Code playground (Python)