Lecture 5: RAG Conceptual Model

mentioning generation. Uh here is what we have talked a little bit about but very quickly so you might not really got this but today we are going to get into this the details of this. So here is kind of a conceptual formwork framework of augmented retrieval augmented generation right uh so the user is sending a query like what we did in prompt engineering but before that query hits the large language model we are going to have a retrieval model in between the user and the large language model and what happens here with the retrieval model is the query will be sometimes expanded sometimes used as a valid term to search for relevant documents from the knowledge base. And these documents are going to be returned, sometimes rewritten, reranked, summarized, do different things on top of this and then putting into the query as the context and send these two together to large language model and then uh the large language model provides the response. Okay. So compared to the prompt engineering paradigm there we have the query directly to the large language model and get the response. So basically it's like hey large language model you have studied all these materials. Now I give you a question answer me. Give you a question answer me. Right? So uh like the prompt will just customize the answer and trying to search all the knowledge the light language model has stored in those parameters but uh with so it's like a closed book test and for rack since since this retrieval model that we are adding here we are getting a few documents to the query it's like hey here are the documents that's relevant so take this and answer from this. So this that that's why rag will reduce the extent of humanation the probability that large language models make up things because now they have some cheat sheet that they can use to answer the questions and you can make the qu the query that's the prompt saying that only answer from my cheat sheet. Oh, it will only answer from the retrieved document and the chance of humination is further decreased. So that's advantage of rack and if you look at here the example right with the contract reviewing uh in the last time in prompt engineering what we were talking about is put few shorts learning few examples within uh the prompt. how you can separate that like because you might have hundreds or thousands of cases or documents and uh like putting all of them into the prompt is not going to be effective. We all know that prompts has lost in the middle problem. So now you can retrieve the most relevant document and then use them to enhance the the interaction the prompt with the n language model. Okay. So that's the conceptual framework. Uh, so here I'll have