What is multi-hop RAG, and how do you implement it?

Some questions can’t be answered from a single retrieval no matter how good the retriever is, because the answer to one part of the question determines what to search for next. “What’s the salary range of the CEO of the company that acquired WhatsApp” needs three sequential facts — who acquired WhatsApp, who’s their CEO, what’s that person’s salary — where each hop’s query only becomes knowable after the previous hop resolves.

Two implementation patterns handle this. Iterative retrieval lets an agent retrieve, read the result, generate the next sub-question from what it learned, and retrieve again — essentially a ReAct-style loop. Query decomposition instead has the model break the full question into its sub-questions upfront, answer each one via separate retrieval, and synthesize the sub-answers at the end.

Production versions of either pattern need explicit guardrails: a hard cap on hops (three to five is typical, since cost and latency compound with each additional round), carrying accumulated context forward so later hops benefit from earlier findings, and loop detection that breaks out if two consecutive sub-queries come back identical.