How do you implement query routing in an Agentic RAG sy...

Routing in an Agentic RAG system is really three separate decisions, and conflating them into “pick an index” undersells what’s actually happening:

• Which source — a pricing question and an HR question shouldn’t hit the same knowledge base even if both are technically “documents.” A lightweight classifier (an LLM prompt or a small dedicated model) looks at the query and picks the destination index.
• Which tool — not every query is a document-retrieval problem at all. “What’s our Q3 revenue by region” is a text-to-SQL problem against a warehouse, not a vector search; “what’s the latest AI news” is a live web-search problem. The router has to be willing to bypass retrieval entirely.
• Which strategy — a simple factual lookup can go straight to dense search, while a multi-hop question benefits from being decomposed into sub-queries first, and an ambiguous one might need hybrid search plus reranking to separate close matches.

In practice this is usually one router prompt that outputs a category, and a lookup table mapping categories to a (source, tool, strategy) tuple — cheap to run since it’s a small classification call ahead of the expensive generation step, and it’s what separates a demo that only ever hits one index from something that behaves sensibly across a real, heterogeneous set of knowledge sources.