← All questions
System DesignHard· Latency· Memory Optimization· Recall

How do you adjudicate a request to delete a sentence in...

Platform’s proposal to save 9.8 GB and 2 ms of latency is flawed if it forces the system to prefill sections, which can increase latency by over 150 ms. The correct approach is to keep the sentence index for ranking, return parent documents only when requested, and use int8 quantization on sentence vectors to reclaim the memory without sacrificing the recall required for factoid queries.

Code playground (Python)