← All questions
RAGHard· Query Routing· Latency Optimization

How do you decide, per query, whether to run iterative...

Since there is no standard dataset for ‘needs N hops,’ you should train a hop-count classifier using silver labels. The decision-making process must account for the asymmetric cost of errors: routing a multi-hop question to a one-shot path results in a plausible but incorrect answer, whereas routing a single-hop question to an iterative path only incurs additional latency. Therefore, it is safer to tune the classifier to favor iteration and monitor the one-shot path for common failure patterns.

Code playground (Python)