Your autoscaler is scaling in correctly but P99 TTFT is...

Spiking P99 TTFT despite correct autoscaling is often caused by KV cache eviction storms. As sequences complete, blocks are freed, but if the scheduler allows too many long-context requests to accumulate, memory pressure forces the eviction of partially-warm cache blocks mid-batch. This triggers re-prefill for the evicted sequences, causing latency spikes. The solution is to implement admission control at the router, capping the number of admitted sequences based on remaining KV capacity.