How does SGLang’s radix attention extend PagedAttention...

Radix attention improves upon basic PagedAttention by managing a prefix trie that tracks KV-cached sequences across all current and recent requests. By identifying common subtrees—such as shared system prompts, multi-turn history, or recurring RAG context—it deduplicates memory usage. While it provides significant gains in scenarios with high prefix overlap, it introduces management overhead and carries a risk of cache thrashing if the prefix diversity is high relative to the available block pool size.