← All questions
RAGMedium· Caching· Latency· System Design

A teammate wants every answer to fetch its sources live...

Always-live fetching is inefficient because it forces the system to pay the cost of a crawl on every single query. This is a significant waste of resources compared to traditional search engines that amortize this cost over many users. You should challenge the requirement by identifying the specific staleness window the product can tolerate, allowing you to cache results for sources that do not require real-time updates.

Code playground (Python)