More data makes RAG applications worse, not better. Relying on vector similarity search doesn't scale, and most people aren't talking about this. This is counterintuitive, but experiments don't lie: As you add more documents to a vector-based RAG system, its retrieval performance degrades. In theory, a RAG system should work the same regardless of the knowledge base's size. In practice, new information can trick a RAG system into retrieving the wrong context for a given query. The more documents you have, the more likely this is to happen. New research from @Eyelevelai shows something concerning: RAG applications using Pinecone with both LangChain and LlamaIndex degrade by up to 12% as soon as we scale the size of the knowledge base. You can read the full report here. eyelevel.ai/post/do-vector-d… The attached picture shows the accuracy degradation as we add up to 100,000 irrelevant documents to the data. This is a huge problem. By the way, there's nothing wrong with LangChain or LlamaIndex here. The real issue is how we put all of our eggs on vector similarity search. Modern encoders aren't good enough to organize a lot of information properly in vector space without cluttering it. @Eyelevelai's GroundX is an enterprise-grade RAG system that only shows a 2% accuracy degradation at scale because they don't rely on vectors. Instead, they use a combination of Open Search, traditional SQL, and several fine-tuned models to retrieve the context for a given query. This gives them SOTA accuracy at any scale, beating every other implementation by more than 50% on complex documents. If you are interested in an out-of-the-box RAG solution, you can get 5M free ingest tokens for your documents by going to this link: eyelevel.ai/ Thanks to the @eyelevelai team for partnering on this post.

David Cameron has been appointed foreign secretary.

Multiple fully Tesla-made Bots now walking around & learning about the real world 🤖 Join the Tesla AI team → tesla.com/AI