RAG Benchmarks with Nandan Thakur - Weaviate Podcast #124!

Nandan Thakur is a Ph.D. student at the University of Waterloo! Nandan has worked on many of the most impactful works in Retrieval-Augmented Generation (RAG) and Information Retrieval. His work ranges from benchmarks such as BEIR, MIRACLE, TREC, and FreshStack, to improving the training of embedding models and re-rankings, and more! Links: Nandan Thakur: https://thakur-nandan.github.io/ FreshStack Benchmarks: https://fresh-stack.github.io/ BEIR Benchmarks: https://arxiv.org/abs/2104.08663 BrowseComp: https://arxiv.org/abs/2504.12516 Sufficient Context: https://arxiv.org/abs/2411.06037 Check out the Weaviate Query Agent! https://weaviate.io/developers/agents/query Chapters 0:00 Welcome Nandan! 1:15 The BEIR Benchmarks 8:25 Evolution of RAG Benchmarks 12:25 Diversity in Search Results 18:20 Reasoning and Query Writing 23:03 Search Result Summarization 26:10 Looping Searches 29:45 High Recall Search 33:10 Paginating Search Results 38:05 Searching with Filters 44:35 Mixture of Retrievers 51:08 Creating FreshStack Weaviate 1:00:35 Exciting Directions for AI