Stan

Traditional RAG vs. HyDE, visually explained! One critical problem with the traditional RAG system is that questions are not semantically similar to their answers. Consider the example where you want to find a sentence similar to "What is ML?" It is likely that "What is AI?" will appear more similar than "Machine learning is fun." This semantic dissimilarity leads to several irrelevant contexts being retrieved during the retrieval step. HyDE solves this issue. The visual below illustrates how this approach differs from traditional RAG. Here's how it works: - Use an LLM to generate a hypothetical answer (H) for the query (Q). This answer doesn't have to be entirely correct. - Embed the answer using a contriever model to obtain E. Bi-encoders trained with contrastive learning are generally used for this purpose. - Use the embedding E to query the vector database and retrieve relevant context (C). - Pass the hypothetical answer H, the retrieved context C, and the query Q to the LLM to produce a final answer. Done! Now, of course, the hypothetical generated will likely contain hallucinated details. But this does not severely affect the performance due to the contriever model—one which embeds. More specifically, this model is trained using contrastive learning and it also functions as a near-lossless compressor whose task is to filter out the hallucinated details of the fake document. This produces a vector embedding that is expected to be more similar to the embeddings of actual documents than the question is to the real documents. Several studies have shown that HyDE improves the retrieval performance compared to the traditional embedding model. But this comes at the cost of increased latency and more LLM usage. Enjoyed this? You should also my RAG series! From building and optimizing RAG apps to evaluating performance and crafting agentic & multi-modal systems—it's all here. Link in the next tweet! _____ Find me → @akshay_pachaar ✔️ For more insights and tutorials on AI and Machine Learning!