What is HyDE (Hypothetical Document Embedding), and whe...

A quiet failure mode in RAG is that queries and documents don’t actually live in comparable regions of embedding space — a three-word question and a three-paragraph technical answer can be topically identical yet embed quite differently just because of length and register. HyDE works around this indirection rather than fixing the embedding model: ask the LLM to write a hypothetical answer to the query first, factual accuracy not required, then embed and search using that hypothetical document instead of the original question.

Because the hypothetical answer is stylistically much closer to how real documents in the knowledge base are written, the resulting search tends to land on genuinely relevant material more often than searching with the bare query would.

It’s not free, though — it adds a full LLM call before retrieval even starts, and it can misfire on time-sensitive questions, where the hallucinated hypothetical might embed confidently wrong recency signals that pull retrieval toward stale or fabricated matches. Worth A/B testing before adopting, not assuming.