← All questions
LLMOps & ProductionHard· Latency· Security· Caching

Security requires a live revocation check for every doc...

A live check at k=20 would triple the latency budget, making it infeasible. Rather than breaking the SLA or dropping the check entirely, a periodically refreshed cache provides a balanced approach. This allows you to explicitly define the exposure window for revoked keys while keeping the system within performance constraints.

Code playground (Python)