REFRAG with Xiaoqiang Lin - Weaviate Podcast #130!

Hey everyone! Thank so much for watching the 130th episode of the Weaviate Podcast featuring Xiaoqiang Lin! Xiaoqiang Lin is a Ph.D. student at the National University of Singapore. During his time at Meta, Xiaoqiang lead the research behind REFRAG: Rethinking RAG-based Decoding. Traditional RAG systems use vectors to retrieve relevant context with semantic search, but then throw away the vectors when passing the context to the LLM. REFRAG instead feeds the LLM these pre-compute vectors, achieving massive gains in long context processing and LLM inference speed! REFRAG makes Time-To-First-Toke…