Building LLM Chatbots Step-by-Step Tutorial | Large Language Models |Community Webinar

Key Takeaways

ations uh that are basically SAS applications and in today's webinar I will be talking about how we can build simple and efficient chatbot step by step so without any further delay let's get started first let's go through the agenda to get an overall view that what we will be covering today so firstly we will look at the concepts of chatbot and after that we have a brief overview of llms and after that we will discuss some common problems that are associated with the chatbots and then we uh we look into that how crafting efficient prompts and rag approach will uh deal with all these problems that are associated with the chat BS and after that we have the architecture of rag approach and then we have a demo that is basically en Hands-On implementation in which we will be creating our own chatbot and we will be bringing all the theory into practice right so let's just start now I want to start with a short story of Innovation when the chatbots enter the world of customer support service so uh suppose that you own an e-commerce company and um in order to respond to your customers query efficiently you got an idea that you should start a customer support service right so basically uh everything was manual first and uh that was basically the pre- chatbot era right in which everything will be manual so in the pre- chatbot era as I am discussing in this Slide the customers were facing some pain points the main pain points of the customers were limited availability as there were U uh there were no service provided uh beyond the working hours or on holiday the second thing was the long waiting times as the customer have to uh wait on hold on call for long times and this was causing frustration in customers the third thing is the repetitive interactions in which uh the customer have to basically repeat their is again and again and uh from one operator to another and this was also causing frustration in customer right and the fourth thing was the lack of personalized service so uh the lack of personalized service also uh give a sort of disconnected feeling with the company and it was also making the overall image of the company uh negative in the mind of customer right so after describing all of these pain points now the question is how can we deal with all of these pain points so as the owner we got an idea that uh what about introducing chat boards to this field so introducing chatbots chatbots uh what chatbots are technically chatbots are artificially intelligent software and uh whenever the user ask a question from this chatbot they basically comprehend that and based on its predent information it will be responding to the user in a very human way so this is a very cool technology and this is helpful in various aspects of life like you can use chat boards for your day-to-day work and many more things right and now when we start using these chat boards in our story like integrating the chatbots in our customer support service what happens is that this magical tool spark Revolution so as I have uh discussed here as you can see that the chatboard started to reply in real time and there were instantaneous responses to 24/7 the second thing was availability as they were highly available and regardless of the time zones or holidays they were available 24/7 right and the third thing was offering personalized solution so basically uh the chatboard were using the customer past history and they were offering personalized solution to each customer uh which create a sort of connection between the customer and the company and the customer do not have to repeat their question as well because uh the conversation between the customer and the bot was preserved and that was used to respond right so by doing all of these advancements the company start to make progress and there was a high increase in customer satisfaction and overall brand image also improved so uh by telling you all of this story what I want you to is to realize that how impactful chat boards can be when they are integrated in an organization or when they can be integrated in our day-to-day Life as we can see in this story it uh basically resolve many of the pain points that customer will facing right so uh this gives us an understanding about the importance that chatbot have in our life and now we are good to dive in the technical details of everything that work behind the chat boards right so moving forward the first thing I want to discuss here is the large language model this is the basic and the fundamental concept of chatbots and uh uh what the uh what large language model basically are so if you can see this picture what happens is that that uh when you give data like text images speech structure data and so on and you use all of that data to train your model the large language model can be basically used to adapt to different task like here you can see it can be adapted to uh the task of question and answering it can be adapted to sentiment analysis it can perform information extraction and so on right so it is an AI algorithm that utilizes deep learning and extensive data to understand generate and predict the textual context so one model can be used for many task as you can see here now as I'm mentioning some of these task here I want to mention them in a little more detail so that we can get an overall understanding about what the large language models are capable of so let me pull in my next slide and yes here as you can see that in the first uh thing I want to discuss here is the sentiment analysis so uh sentiment analysis basically means that you are instructing your model that uh to classify the text based on the tone right so if you can see here that this is a positive tone sentence right and the product okay I guess is a neutral tone sentence and the third sentence your support team is used as is a negative tone sentence so it is classifying the text based on its tone and the second thing I want to discuss here is the summarization so suppose that you have uh some research documents and you want to get only the key points from those documents so what you can do is that you can give all of that long text document to your large language model and it will extract the key points from it and then you can use these key points to understand whatever uh was written in that long text document right so this will help you a lot and by saying this uh if I can go to the third task that I want to discuss that is the task of translation so in Translation we mean that converting uh from one language to another and if you can see uh here I am converting an English language sentence into French so by using a train llm what you can do is you can convert an English language sentence into uh French language sentence right so this was about translation and the fourth thing here is the question and answering so generic question and answering like when you ask some question to the large language model it will basically uh based on its pre-trained information generate a very helpful result so it can be used for generic question and answering as well so uh as for now so far we have discussed the large language models and we know what uh they are capable of Performing right so by doing so let's move on to our next slide that is the working of uh chat boards so uh I want all of your attention to this flow diagram here that I have created in this you can see that whenever the user is asking a question what is happening is that uh this question is basically transferred to this large language model and this large language model based on its pre-trained information will respond to the user so this looks very uh simple right but there can be many cases in which uh this large language model of fire can generate undesirable responses so uh for dis for uh describing all of uh those undesirable responses let me pull in my next slide in which I am describing one of the case and that can be that uh whenever the user asks a very vake question for for example or a question that is very unrealistic so in that case what happen is that the large language model of ours will not be able to understand what the user is asking and it will make up things on its own and it will uh respond in an undesirable way that is basically it will hallucinate uh the things and it will generate an undesirable response right so this can be uh one case and the second case can be that when the required data is not available in the pre-trained data set so uh if you can see here as I have discussed uh already that the large language model was trained on the large Corpus of data set right so if uh what if that this data has not the information about what the user is asking in this case uh you can uh understand it like that that this large Corpus of data set must have a cut of date right there will be some point where this dat data set will be ending so what if the user ask a question about an event that uh was not present in this large Corpus of dat data set in that case also the large language model will produce an undesirable result and similarly if I can say that I ask a very personalized question for example so what happens is that it will not have uh the domain specific knowledge and as it will not have the access to my data that's why will uh result in an undesirable response in that scenario as well so these are the two problems that I want to discuss and all of these that I have just stated comes Under The Heading of limitations of large language model the first one is hallucination and the second one is the problem of data limitation in the model so I'm assuming that everything is clear till now if you have any question please write them in the chat B uh in the chat as well right so let's just move on to our next slides to discuss these uh in more detail so I am using some of exam some of the examples here to describe the problem of hallucination the first one is as you can see in this picture that when I ask a question when was Egypt transported to the uh uh for the second time across the Golden Gate brid so you can see that this question is very unrealistic right so in that case the bot is making up things on its own as we know that none of this thing happened in October 2016 right so it is making things on its own and it is hallucinating the responses and in the second example if you can see I have asked a question that how many M's are present in the word weather and the B reply that the word weather contain one M whereas we know that the word weather has no alphabet m in it right so in this case as well the bot is making up things on its own and it is generating an undesirable response to me so till now we have developed a good understanding about the problem of hallucination and now we can move forward to use examples for discussing the problem of data limitation as well so if you can look at this first example uh what I'm asking is that who is the president of US in 2024 and the B reply that as per my last update in January 2022 I do not have access to realtime information so this is the case that I was talking about earlier as well The Bard do not have access to this realtime data or you can say the data after its cut off date like the uh the pre-trained data that is present in this bot is uh ending on year uh 2022 it means that all of the events that are within 2022 are present in the pre-rain data set but the events that that happen after that are not included here that's why we got this reply and this is also an undesirable response right we do not want it here now the second example can be that where what are the core values of my company shop e so when I ask this question as you can see that I'm asking a very specific question that is related to my company like it's a very personalized question so in that case as well the B reply I do not have access to specific information about your company unless you provide it to me it means that it do not have access to my data and hence it is unable to produce a good result or a desirable result and we can see here that it has just uh uh give me an update that it do not have access to my data so uh these are the two examples that describes the problem of data limitation to us and now as we have a good understanding about the two common problems of large language model first was the hallucination and second was the data limitation now the question arises is that how can we basically overcome these limitations of large language model right so so far we have covered the two problems and now moving on to the solution parts to resolve these problem the first uh solution that I want to discuss is efficient prompting so uh here I'm using a very simple example to describe that and efficient prompting is basically uh related to the whole domain of prompt engineering and I'm not going into much detail about it but I have some points here and that must be included in the prom to make it effective so if I can discuss that the first one is to give a specific role to the chatboard so if you can see in my example here in the first line I have written that you are an intelligent assistant that count the number of times a specific alphabet appear in the given word as by a user so this is how you define a role to uh a chatboard that will basically help the chatboard to understand what task it has to perform or what role it has in the scenario and after that the second thing you can do is writing clear and Specific Instructions so here I'm writing these two instruction uh and I've written it in bullet points and you can see the line that follow each of the instruction given below this line is included to create an emphasis on these two points right so writing clear and specific instruction and different finding role to the chatboard the third thing can be uh to give examples and by this I mean giving more context to the uh language model by describing that what the user can ask and what the board should reply so I have written down two examples here that should be used uh in this scenario of mine and after that what I have done is I have asked a very clear and specific question so this is also one of the thing that you should be clear and precise in asking the question to a large language model so that it do not make up things on its own right so as you can see in this question I am asking count the number of times o appear in the word weather and it reply count is equals to zero but this question is pretty much similar to what we have seen in the problems right and there it was not responding correctly so uh it means that by giving all of this context that I have written here the board start to reply in a good way so this basic basically tells us the importance of prompts and we should include these points that I have written here to make effective prompts so this was the first technique and now moving on to my second technique that is the retrieval augmented generation so what we do in this technique is we enhance the output of large language model by giving them uh an additional information using this external knowledge base this external knowledge base will have all of our data that we want our model to know right so all of the information of our choice will be loaded in this external knowledge base and then we will utilize that to produce good results from the large language model uh if you can see this flow diagram you can remember that this is pretty much similar to what we have uh seen in the working of chat boards right so uh this is similar but the thing I've have headed here is this external knowledge base and the flow uh now is like that whenever the user is asking a query that query will not be directly transferred to this large language model that query will be used to First search in this external knowledge base and after that what we do is when we search uh for some relevant documents in this external knowledge base of ours uh we will get some relevant and will append them to our query and we will send that to our large language model right so what is happening here that we are giving more context to our large language model by adding some relevant information as well we are not passing this query directly to this large language model of ours so uh this will uh if you can understand this this will help us in uh removing the problem of data limitation to a great extent right as we were seeing previously that the model do not have knowledge about the data related to some recent events or about our personal information right like the legal documents or anything like that so now in this case as we are uploading all the information that we want the model to know that uh in that case the model will have the knowledge and it will uh reply to uh the user in a great manner right so by saying this now there can be two questions first one can be that uh how uh are we making uh this external knowledge base right and the second question can be that even if we make this external knowledge base how can can we integrate that in this whole flow of ours so for that we have the ra rag architecture so uh moving on to the ra R architecture the first step I want to discuss here is to create this external knowledge base in which we will be uploading our information for that we have to follow some steps here uh as you can see that there are four steps so the first one here is loading of data so by uh loing I mean that uh whatever data you want your model to know you that that can be in form of PDF or that can be in URLs or anything else right textual data can also be there so what you want to do is you want to extract all of the textual information from that and by doing so you make a big text block and by making this text block what you do is in the second step you basically break that text block into smaller chunk so what we have done we have extracted Ed some of the textual information from the text block and after that we have made chunks out of it right so uh by uh by splitting if uh someone can ask that why we are doing this splitting thing right so if I can move backward and show you this again that the reason for splitting is whenever the user is asking a question what we want to do is we want to uh add some of the uh relevant documents to the question right we do not want this whole external knowledge we want some of the documents that are related to the user query to be appended uh with the query and we are passing that as prom to our large language model so uh in this case the splitting will help us as it will chunk out our documents and the second reason for chunking is that or for breaking the text into small documents is that uh our prompt will have a token size limit and we do not want to exceed it that's why we have to put only the relevant chunks with our query and pass that to our large language model right so this was the reason that we do splitting and after that when we were when we are done with our splitting part we embed the documents so by embedding I means converting the text information into vectors why I'm doing so it is because of the reason that the machines cannot understand uh alphabets right so it understand numbers and we are basically generating the meaning of uh the alphabets in form of numbers so that our machine can understand that in a great way uh by seeing this if I can move backward again and explain that using my diagram so as I have discussed the user is asking a question and we want to extract some relevant documents from it right but uh uh what we want here to uh what we basically want here is that uh we do not want exact keyword matching of this query with our external knowledge based information we want to infer uh the meaning of this query and based on the inferred meaning of this query we want to retrieve the relevant document so this means that we want to semantically uh look for the in relevant piece of information and we want to extract that for that we have to perform Vector search and for performing the vector search we have to convert the strings into vectors and by doing so we are good to move forward to our next step that is storing the documents and in that we are basically storing our chunks that is in the form of vectors into a vector store so by performing all of these four step uh we have created an external knowledge base so far and now we are going to move towards our next step that can be the retrieval and generation part so in the retrieval and generation part what happen is that uh uh we whenever the user is asking a query this is pretty much similar to what we have discussed so far right so whenever the user is asking a query what I'm doing is I am uh joining this query with these relevant textual documents that I'm retrieving and these are basically uh the relevant documents that are based on this users's query inferred meaning right so I am retrieving those documents I am appending all of these things in the prompt and I'm passing that to our large language model so when we do that what happens is that the large language model is able to produce more efficient results so by saying all of this till now we have covered the two uh techniques uh to uh to make the large language models respond in a better way the first technique was creating efficient prompting and the second technique is the rag approach right so now we as we uh are done with our Technologies of making uh good responses we can move forward uh to this part that is the uh Essential Elements for our implementation part right so I will be quickly going through this and this will be helpful in our implementation part that we will be covering soon so the first thing here is the large language models so uh as I have discussed what we want for uh making a chatbot are the large language model then we apply some prompt engineering techniques to it and there will be a vector store to perform rag right so so to in order to connect all of these things and to make everything join we need a framework this framework will join everything and it will make an AI application for us so by saying so the framework that we are using here is the Lang chain framework Lang chain is an open source framework designed to simplify the creation of application that use large language model uh the cool thing about Lang chain is that it is an open- Source framework with a lot of uh buil methods like you can see in this picture also all of the technique that we have covered so far can be easily utilized using this uh these pre-build uh methods like it has method for creating models it has method for creating prompts for retrieving the documents loading the documents using different type of vector store and so on so this is a very cool framework that will help us to design our AI application in a very simple manner so as we have discussed length chain as well and we are all good with uh the technologies that we have discussed to make efficient boards now we can move forward and put everything that we we just see in the theory part into practice right for that we have some prerequisites of the code as well so if you can see here we need access to openai Azure service so Azure open Service uh is something that I'm utilizing here but if you do not have access to that right now you do not have to worry and I will be sharing my code with you and you will have the video with you gu with you so you can always look into that and the second thing here I want to discuss is you can also use the trial version that is your provide that will be enough for experimentation as well right uh and the second thing here is that I'm using Jupiter notebook for execution of my code so let's jump on to the code and let me share my screen again for that all right is my screen visible to everyone perfect so let me start that so uh here in this case what I am doing is that yes so here in this case what I'm doing is I building a customer support service chatbot for a company name shop e so uh this is an hypothetical company that I'm using here and if I can show you my text file as well let's just zoom it a bit so yeah this is my text file and all of this data that you are seeing right now is the data that I have created this is a dummy data basically and as you can see I'm the founder of the company and we have about us page as well core values of the shop uh Vision products available at the shop e and we you can see that there are electronic Goods fashion goods and so on right so this whole data is basically the data that I have created this is a dummy data and that is uh for an hypothetical company naming shop e so if I can go back to my notebook what uh you can see here is that firstly uh we will be looking at all of the cells that uh that are present in the notebook and we will see that how each of the line is working so for that if you can see the first thing is the installation part so installation of necessary libraries will be done firstly and for that we use pip install command of python and I can run this and let's see yeah yes so the next thing here is creating an external knowledge base so by meaning of external knowledge base as discussed in our theory part as well what we want to do is that that we want to upload the information that we have the the personal information uh to so that it can be accessible to the large language model right so for that case we have to make an external knowledge base and for that case as we have discussed in the slides as well we have to follow few of the steps for making an external knowledge base right so for that if I move a little bit down you can see that the first step is to load the document I have shown you my document that is this text file name demo data.txt as it a text file so what I'm doing here is that I am uh first uh loading its path in a variable name file path and after that I'm using this text loader component of Lang chain document loaders and in the in this text loader document I am passing this file path that I have given and when you do that what happens is that uh you have created an instance of text loader and you have to basically use this load method of of the text loader right so when you use this uh do load method all of the textual information that I was just showing you and that was present in my text file right that will be extracted and it will be assigned to the variable a able document here so let me quickly run these cells and if I can show you the document this is how the document look it look a pretty clustered here but if you can see that this is a document type object and in that we have the page content attribute and in that we have all of our textual information right so uh I can show you the uh only the text as well for that what we have to do is as you can see that this is a list so we will write this and in basically using the first element of the list and using the page content attribute let me print that for you yes so this is pretty much similar to what we were seeing in the text file right it has about us page it has core value vales it has vision and the data goes on so all of the textual information from my text file has been loaded to the document variable till now so as now we have a big text block with us the Second Step was to split that big block of text into chunks right for that I'm using a recursive corre text splitter now the cool thing about recursive character text spitter is that it will be uh splitting the big text block again and again until it it reaches the uh fixed chunk size that we defined like here you can see that I am defining the chunk size as 2,000 it means that the uh block of text will be splitted again and again until the chunk is within the range of 2,000 now the second thing here is the overlap as you can see here that I have written overlap is equals to 800 so overlap means that uh it is the count of correctors that you want to be overridden or you can say overlap in your your adjacent chunks like if you can say you have two chunks right the first chunk the the basically the last line of the first chunk would be the first line of the second junk so this is how you will be overlapping everything together and this is basically to maintain continuity and to maintain the information and relation between the uh chunks right so for that we are doing overlapping and the third thing here is the length function this is to Define how uh the length of the chunk is calculated and here we are using this length function of python let me run this cell for you and now if you can see in this line here what we are doing is after initializing this recursive corre text splitter we are using split document method of the Lang chain of course and we are passing document variable that have all the text block that we have just seen earlier in our code to this split document method and it will make chunks for us so if I can run this line and let me quickly show you the chunks as well yes so these are the chunks if you can this is a bit clustered but if you can bear with me uh this is the first chunk and if I can scroll a bit this is basically the second chunk and the list goes on so this looks a bit clustered but let's just print the length of the chunks as well so we can see that how many chunks have been created from our large text document that was in this uh text file right so this was my text file and now I have created chunks from it and if I show you guys the length of chunks that is 20 so it means that the large text document is converted into 20 small pieces right so now as we have our chunks ready what we have to do is we have to embed those chunks and store them in a vector store right so for that purpose for embedding I'm using the text embedding model and as I have said in my prerequisite as well I'm using a openi service right so I'm using a text model of openi and if you can see in this line I am importing the Azure open a embedding component from Lang chin open Ai and if I scroll a bit you can see that I am initializing this Azure open embedding in this way so if you can see here that I am passing the open AI API key embedding model base URL and so on so these things are should be passed in a secure way and the best practice is to use a EnV file for that so what we do is whenever we create an open AI service on a portal we will be given all of these things so we put all of that in in a EnV file and we use this load. EnV method to load everything load every environment variable up and then we use the those variables in in our instantiation so that everything can be retrieved securely as some cost is associated with it so that's why we are using this best practice here and let me run these cells as well so so far we have our chunks with us we have created our embedding model as well that will be converting uh the textile inform into vectors right so if I can show you how this will happen suppose that this is a string I am making a string and I can assign a variable text to it now what I can do is I can use this embeddings model that I was showing you that has the capability of converting string into vectors right so I can use this model and I can use its function name in bed query and I can pass this text variable of mine to it and if I print that you can see that this text is basically converted into vector format and it means that the information present in this text is uh demonstrated to my language model in the format of vectors so this is how your language model will be working and now as we have our large language model with us uh sorry the embedding model with us what uh we want to do is we have to uh store everything in the vector store right for that I'm using uh the chroma DB you can use other Vector stores as well so I am importing chroma here and in chroma there is uh basically in langin there is a method name from document and what you have to do is you have to pass the chunks that you have and you have to pass your embeddings and by passing the these two parameters it will make the vector store for you so that's why Lang chain is a pretty simple and straight forward framework it has many pre methods in it so now we have our Vector store with us right so the second step or we can say the last step in it is to make this Vector store as a retriever object so the purpose of making a vector store as retriever is that we want to retrieve some relevant documents on the go right whenever the user will ask a question we want to retrieve some relevant documents out of this Vector store and we want to use them in our uh result generation right so for that we have to make the vector store as Retriever and in that what you have to do is you uh what I am doing basically is that I'm passing this search argument as well so by passing this argument what happens is it will retrieve only four relevant splits as I want only four relevant chunks because my uh text document that I was just showing you is uh very small so that's why I'm retrieving only four documents from here and now we have our knowledge base ready and we can move forward to our next step as we have discussed in the theory as well right our next step was to retrieve and generate so in the generation part basically uh when the llm generate responses we are using uh Azure open AI model again and here I'm using openai 3.5 turbo and if you can see here if I scroll a bit down I am again importing uh an Azure chat open model from uh Lang chain. chat model and it is also pretty much similar it's instantiation is basically pretty much similar to what we have seen in the embedding model right so we have to pass all of these value and these values will come from the EnV file and the new thing here will be this temperature so uh what I mean by uh this temperature is that uh temperature defines that how creative you you want your model to be right like uh if you want to build uh a chatbot you can say for uh blog writing or for essay writing you want your model to be very creative but in our case as we want to make a chatboard for a customer support service and we want a model to be very specific and very strict uh that should stick to our instructions right so in that case we have to make the model less random or we can say less creative and that's why we place a smaller value of temperature here I'm putting 0.2 here so that my model can stick to my instructions and we can ignore this warning as this is for only the updated for using the updated version actually and now as we have our large language model with us we have our external knowledge base with us so the next step can be to create a nice and efficient prompt right as we have discussed in the theory part as well that what we will do in that is we will firstly specify the role in The Prompt and then we will specify some instructions so here if you can see in the string that I have created name prom uh I am writing that you are a dedicated customer support representative for a company named shopes a leading online retailer and I've also defined the goal here and after that what I'm doing is I am mentioning some of the instructions that I want my model to follow in this scenario so for that I've listed that answer should be always generated from the sources listed below your response should be complete and concise and do not list the sources in the answer so these are the instruction that I want my model to follow and this is the role now uh prompting is a very fundamental thing and uh this basically make a very great impact on the large language models uh response so you have to be very careful when crafting your prompt right so if I can go a little bit down you can see here that under the sources heading in the question heading we have two variables as well so the value of these variable will be uh uh replaced by the basically the value will be uh pasted here and in the query uh on the go like when the user will be asking the question the value will be pasted here and the context that we will be retrieving will be pasted here and in our case that will be four relevant documents right so all of that thing will be placed uh on the go and as we have made our string for our prompt now what we have to do is we have to convert this prompt into uh we have to convert the string basically into a prompt so for that I'm using promt template and I will pass my prompt to it this is the variable that has my instruction right so I'm passing that to it and after that we will uh be uh declaring the input variables that were context in query in our case and it will make the prompt for us and after that what we have to do is we have to make the chain like uh by this is uh a simple chain and I'm I'm using only the Lang and expression language for it and what I'm doing here is that I am uh the the prom that I have created the customized prom I will put that here and I will use this partical line and I will put my large language model that I've just instantiated above in my code so after putting all of that uh when I run this cell our chain will be created and we are good to go and let me just uh use uh this example here that for understanding for making you all to understand the flow that will actually will be followed when the user will be asking a question right so for that if you can see here uh firstly I'm asking the question that what is the vision of the company as again I am repeating that that I have made uh a chatbot Customer Support Service uh chatbot for the customer support service and that is for a company named shop e so what I'm doing here is I'm asking a question about the company that what is the vision of the company and if you can see here that this question is pretty wague but now as our uh model will have context about our company as I am making sure that in my prompt I'm declaring that in a very clear way so now the model will not misunderstand things and it will generate good responses that's why we have done all of that prompting thing right so uh when we have this query the uh step uh that will that we will follow is that we will pass this query to get relevant documents first that is by passing uh that is by using this get relevant document method on the retriever object and when we have our four relevant documents we will pass all those relevant documents along with the users's query to our prompt and we will invoke our chain so this will be the flow and if I can run these cells and this uh line of code is basically to show you all the the relevant documents that have been retrieved in when the query is asked about the vision of the company so if you can see here that in my first junk we have some information related to the vision of the company right you can see this heading vision and it it has information related to the vision and this is basically the Chunk from the source that I have given and what I have shown you previously right and the first uh the first count can be this is uh I am starting the count from zero so you can say that this is the second source and in that we have the content that is also related to Vision like it is talking about Innovation teamwork sustainability somewhat related to Vision right and in the third chunk of ours we have uh information related to Vision as you can see in pursuit of our vision we aim to and all of these points are given so this was our third chunk and our fourth chunk is this in which it is talking again about Vision as we work towards achieving our vision we need to join and the information goes on right so these four chunks basically related to Vision that has been reved from our document and we are not pasting our whole document into our prompt but rather we are using these four chunks so far to paste them in the uh prompt on the go right this is what we have discussed so far as well and we will pass all of that and here if I can run this cell our result have been generated and we can actually see the result as well so this is the result that have been generated right it has uh this content field and in the content field we have all of our result uh and if I can read that out the the board said that the vision of shies is to become the preferred destination for online shopping Renown for exceptional customer service and the information goes on so this lines basically are retrieve as I've have created the dumy data so I know about the document and if you can see here that uh this line was present in my document like uh if you can see here it says at shop e our vision is clear we aspire to become the preferred destination for online shopping Renown for exceptional customer service and the information goes on so this means that the bot has replied to me from the document that I have given and uh and in the prompt as well I have written that answer only from the sources so it is also following those instructions so this is how you are making a chatboard that is purely related to your document and and it is uh behaving in a very efficient way following or your uh instructions right so this was uh all about the technical aspects of what we have seen in the theory part as well and we have covered all of the technical aspects like creating efficient prompts like using dag approach and so on so every technical aspects have been covered and now I want to quickly discuss the user interface as well so for user interface I'm using the gradio application gradio is basically a open source python package that allow you to quickly build a d so this is a very simple or and and a vast of course vast documentation and it has many boiler codes for its uh components as well right so in gradio I am using its chatboard component and what I have done here is this is basically the Boiler code that gradio gives and I am just tweaking that a bit by adding markdowns here by adding some avatars and uh some of uh changing some of the scales of the text box and so on so I've have made some changes and after that the main thing here will be uh this respond function so if you can see here message do submit means whenever the user will be clicking on the submit button or uh or whenever the user is hitting the enter button so in both both of the cases what happens is that this respond function will be called and if I can go above and show you the respond function this respond function is also given in the Boiler code and what happens here is that when the user question and the chat history is uh passed as parameters in this respond function and then in the bot message I am using my own functions pipeline to respond to make the response right so why I'm doing this this is because of the fact that we want to use all of our technicalities like we want to use our rack pipeline we want to use our customized prompt right so for that I have made my own function and I'm passing users query to it and all of the reply that the bot will give to me will be passed to this bot message in that way every time this respond message will be uh called like by clicking on the submit button every time my uh pipeline will be used to generate result right so after getting this bot message we will append that in the chat history and we will return that and if you can see here this is the function that I have created and if you look closely to it it is pretty much similar to what we have discussed so far that whenever the user is asking a question this question is basically used to retrieve some relevant document using retriever do get relevant documents method and after retrieving the relevant documents in our case that will be four chunks right so after retrieving that what we are doing is we are passing those relevant documents along with the user's query in our prompt and then uh when I use this chain. invoke method what happens is that my prompt will be modified and instead of this query variable and the context variable in my prompt the values from this question and this relevant document will be pasted there and the that modified prompt will be sent to my uh large language model right so the result will be like this as I have shown you here as well right manually I have shown you that as well that it will it will look like this that in the content field we will have the result right so what we can do is we only want to use this string so for that we can use result. content and we can use that as our response so I am doing that here and I'm returning the response so let me quickly run these all and if I can show you yes so let's go to our interface this is the interface that I have created from using the gradio Boiler code and if I can and show you if I write that hi need some help here and the board replied of course I'm here to help and what I can assist with you and you can let's just ask a very specific question that is related to my document so if I can ask uh let's say about the products of the company so if I ask what products do you offer again if you can see this is a very vake question right I'm I the model will not know what you means but in the prompt I have specified that the bot is related to the customer support service for the company shop e so it has some context and it must reply in a good manner let's wait for a minute yes so you can see that the the bot has given me a pretty good result describing all of the products that they offer and it has not misunderstood things because I have written everything in the prompt and I'm giving some relevant documents to it right and let's ask another question that what is the return exchange policy here and the board replied that I return exchange policy at shop e allows customer to written products within 2 days of recept and the information goes on so this is also a pretty good response purely from the documents that I have given and it is not hallucinating not giving me errors like uh for the data limitation because I've given everything to it right so this is how you make uh a user interface for the chatboard and if you can see closely that only the US Us's question and the board's reply is shown on the screen right every technicality that we have used like rag approach or crafting efficient prom everything is hidden beh behind the scenes right so this is why you create a UI so that the user can only see the easy part and all all the technicalities can be abstracted behind uh this interface and you can use that to make your chatbots respond in an efficient way right so this brings us to the end of the coding part of building simple and efficient chatbots and now I am open for my question answer session so let me pull that screen up yes so let's just go through some of the questions someone is asking about the GitHub link please give me the GitHub link yeah for sure I can give you that just give me a moment this is the geta blank and let me just maximize my screen okay let's go through the questions chat board very which Gemini with um should not be use uh fine tune the pre trained llm on data set yeah you can also fine tune uh so someone has asked that if we can fine-tune the pre-trained model on the data set that is uh related to us right the specific data so you can fine tune that as well but it uh require you can say more knowledge and it requires more cost as well for that as in in my case what I'm doing is I'm making the model uh learning the things on the go right I am not changing the model I am basically uh using utilizing the prompt to for context learning that uh in technical way you can say that so we can also do fine tuning but it is a very uh comprehensive thing and a hard thing also and you are also not sure about the result but you can do that as well um Can Can you askm to do predictive analysis uh yes it can do predictive analysis but again as I am saying that you have to craft the prompt like that like you have to tell your model that what you want it to perform and uh for uh for that purpose what you can do is you can in the prompt you can specify the role like you are specifically for this purpose like for predictive analysis and so on what products or platforms are most commonly used for rag creation okay so products um you can basically uh I'm utilizing Frameworks for performing this de concept like uh right so the rag is a concept and you have to use uh some Frameworks like you can use Lang chain or you can use l index to perform rag so that you can uh basically perform it in a very easy way you can say in a very efficient way and if there is a drag API for Gemini um yeah you can uh you can follow my steps and you can see how Gemini can work for for R uh I was basically telling you the the concept that how can you use Rag and you can use any any of the large language model and see what happens if you use re to it like it is a concept and you can apply it to any model and you can use any of the vector stores as well so if that answers your question how to know the chunk size so we can make sure the whole answer is is in one chunk let me give a moment to understand this question how to know the Chun size okay so someone has asked how to know the CH size so we can make sure the whole answer is is within one chunk or is it or is it able to use more than one chunk uh and get one answer uh yes so you can use more than one chunk as in my uh tutorial that I was showing to you I was retrieving four relevant chunks right so you can use more than one chunks to get one answer basically that I was doing in my case and uh you can uh make the there are many methods to optimize the chunk size as well like there are many methods to U like in my case I'm using the recursive character text splitter right you can use semantic uh Splitters as well so there are multiple things to optimize the chunk size as well like based on your information it will uh chunk your document so that is also the thing usual size of the split is uh the usual side of the split is uh it depends upon your document like I have shown you my document it was a pretty small document So based on that I was specifying the characters that were 2,000 in my case right so I want only 2,000 characters in one Chung that is why I have used that for my small documents so are we using the LMS itself in the QA system to get the embedding of the query um no uh so we are using the llm itself in the Qs system to get the no the embeddings are created from the text model that I was showing to you I'm I was using an openi text model and the llm that I was using was open a 3.5 turbo that was used for theq and it right so these were the two these were two separate models that I was using that both were of aor open as I have discussed so far as well is there any risk to load your personal data to the large language model can that information be uh no uh uh someone has asked Ami Singh has asked that is there any risk to load our personal data to large language models can that information be compromised by the outside source source so uh this is the reason that we are performing rag that we are not making our data public we are making our own external knowledge base and that uh we can secure that in many ways uh if you use Azure services that can be secured in multiple office and you are not making your data public so uh you can load your personal data as well do the chunk boundaries form on complete sentence of the text input so someone has asked that uh do the Chun boundaries form on complete sentence of the text in put um uh I'm not understanding this question perfectly but I think that you mean about the characters it is using for splitting the document so in that case uh I was using recursive character text spitter and I have if I can go back to my slide um in the recursive characters Tex splitter what you do is you use where is the director text spit yes so these were the characters that it was using previously to uh to basically make the chunks like the line changing characters and as I was discussing that uh instead of line changing characters you can also Define your characters on your own and you can also use other uh Splitters as well Lang chain have a very vast library on text Splitters so you can go through that as well um what is the length of the document list so by document list I think that you mean the document I was uploading I was uploading only one document and and uh that was uh a pretty small document and I have basically converted that into smaller chunks that were of 2,000 characters right what is the scalability of chroma Vector stores so uh yes so for the scalability thing you can always go to this official documentation as well there are many things to describe that like uh we have uh many parameters to see that it can be scalable or not is it in memory or not so there are multiple of things and you can basically go to uh the vector store documentation every time to see that get presentation I think that I have if someone has if I have uh not answered someone question so you are always uh you can always reach out to me on my LinkedIn profile as well if I am skipping someone's question so you do not have to worry in that case as well I will reply there as well where is the openi text model uh so uh someone named deep has asked that so the open AI text model is basically present on my uh Azure portal and when you log into Azure portal you can see that you can uh create many of the services and in that I'm using the uh open service there and in that I'm I'm creating the two models that one was the text model and the other was for the Q&A that was GPT 3.5 T is [Music] there is there uh someone has asked that is there a free service available in place of a your open service so yeah uh there are many uh basically there are many open uh open source models that you can use for for using this rag approach uh and and everything that I have covered there are many openi models for that as well and uh uh I've also made a Blog on llama indexing and there are multiple videos as well uh at data science dojo's page you can go through them as well for using open source models so if that answer your question yes free services are available in place of aour openi when you say chunk you meant a corrector no by chunk I mean the pieces of information that I have so that was the chunk like if you can see here where I have printed the chunk I can quickly show show you that I think in the last yes so this is basically one chunk this is not one corrected right this is one chunk and this is the second chunk and so on what is there are many uh someone has asked about my blog so I can share the link here as well and there are many blogs uh blogs available at the data science Dojo page as well so you can go through them uh every time if I can quickly share that with you and okay I think we are pretty much up the time I am just going through few more questions s and then we will wrap up so um what uh Ami Singh has asked that for a developer who is getting into gen AI llm development or prompt engineering what learning paths do you recommend so so for that uh you can always go through the boot camps that data science Doo provide they are very comprehensive uh actually and you get a very good grasp on on the fundamental concept from that and also on the advanced topic so you can go to that as well and you can also this was basically the my webinar was basically uh telling the the most fundamental things so you are good to go and you can explore prompt in in more and dag more in more detail like