LLM ChatBot with Gradio UI Tutorial - Part 2

Key Takeaways

hey what's up codash welcome to one little coder in this tutorial we're going to learn how to build a chatbot UI using Radio application this is the second part of my first tutorial where we used Google's flan a large language model and Lang chain and GPT index or Lama index to use an open source language model to build a q a bot but we left that code at Google collab and we did not create a UI and a lot of people had been asking me that we should actually build a UI so this is basically the second part of the first video so I would strongly encourage you to watch the first video if you need to understand how this chatbot actually works but here we are going to build the front end and then we are going to build the front end in such a way that it looks like a chatbot and we are going to use one of our favorite tools radio for that so first I would like to show you a quick demo of how it is going to look like then I'm going to take you into the code which is also in Google collab and explain the chatbot part I'm not going to explain the large language part large language model part where we use flan to build the Q a board but this is only going to be the front end and the grade your pattern itself so let me show you how the chatbot looks like this is how it looks like you can go here and ask any question so I've already pasted a document about Ashton Kutcher so I'm going to ask a question so um what are the companies companies where Ashton has invested what are the companies where Ashton is invested okay let's see if it mentions one or two companies okay it says version Galactic okay what are the three companies let's say if it gives me more number of companies it again gives me Vision Galaxy so what are the startups let me ask a question different way so what are the startups where Astron has invested let's see if it says something it is research gate Okay cool so we have got an answer okay who who is Ashton Kutcher who's who is he by the way so he is an actor cool when was Ashton born and let's see if it gives the right answer empty response we didn't get any response okay let me ask it in a different way what is the birth date of Ashton Kutcher and then see what happens February 21st 1971 let me go to the Wikipedia and then see February 7 1978 that's wrong so it is probably picked up a different date but overall our objective is to say that this chatbot is actually working fine so now what we are going to do is we're going to see the code part on how to build this chat button itself first let me take you to the Google collab so I'm going to stop this so this is not going to work for now but okay let me stop it again so this is the part like if you have seen the first video you know we ended the code somewhere here so we ended with index.query and then we asked a question and it responded so now the rest of the part begins where we are going to build a UI user interface a chatbot user interface using radio now first thing that you need to do is PIP install radio once you have successfully installed radio then next you need to import graduate as gr now typically if you have been watching our Channel you know that for grade you we need three important things one as a function second is an input and third is an output so typically most of the radio applications that we have built on this channel had only one function but today we are going to build two functions and with two different tabs as you have seen in the interface you can see that we have two tabs one is the tab where you can go paste the knowledge the text in and itself and then ask it to build an index and then you can ask questions with the knowledge bot so for that we need to we need to have two core functions that grader will use to do whatever we are asking it to do the first function I'm calling it build the bot and the second function I'm calling it chart so what is the first function supposed to do so the first function is going to take the input text and then it is going to build a lamba index document and it is going to build a GPT simple Vector index which is the lamba index so this index we can use now to ask question about the chatbot now again the input text string goes here and from that we are going to build a llama document and from that we are going to build the GPT simple Vector index now the documents embed model llm predictor all these things are predefined so the document comes from this the llm predictor is something that we have already defined using the large language model in this particular case the flan model and also the embedding model in this particular case is a hugging phase embedding model so these things are already predefined so we are simply using gpt's simple Vector index to build an index you're not returning the index it's being it's just stored there as an object but you can also store save the index as an index.json or something and then you can load it it's up to you the code is already available here if you want to do that but for now we have built the index object and with that we are going to return a message to the user saying indexing is successful so you can see that we have returned a message to the user saying indexing is successful after we have done that now we are going to build the second you second function what is the second function the second function is a chat function where somebody can ask a question and then it is going to respond something but we want it to be a chat interface and also you know wouldn't it be fun if we have like the streaming response like how chat GPT does and this is exactly what we're going to do so the first object in this function is chat history so that is going to be always there so you're going to receive chat history and you are going to send chat history and then the next object is the user input so this is the actual question that the user is asking now um when user asks the question you are going to do index.query with the user input so index has been already built index.query will let you ask or query this index so you're going to take the question from the user and do index.query and then get this output and that is going to be the bot response now this bot response is good it's a llama object llama response object but what we want to do is we want to kind of simulate what charge GPS has been doing the streaming response and thankfully gradier has a very easy hack to do the streaming response and that is exactly what I've done so we are going to create a temporary string here called response and that response is basically what we are going to use to create the streaming response so the entire response the bot response that we got it is a response object from that we are going to extract response so at this point bot underscore response dot response is a string now we are going to split this ring by every character let's say I've got how are you h o w everything is going to be separate iterative element so that's what we are doing here now we are going to iterate through every single letter in that string which is the response that the bot has given us and we are going to add it to response and while we are doing that we are going to return we are going to yield like in this case not return we are going to yield it so that it is it can be displayed to the user along with the chart history now that is done now we are going to talk about the UI design the user interface design so as you can see the gradu application we've got a very simple title you have got two tabs inside the first tab we have a text box where the input is collected and then we have a text box where there's an output message is displayed and then we have a button in the knowledge board tab we have a simple chart interface at the top but there is no button all you have to do is type the question here send it type the questionnaire send it so that's exactly what we are going to design now we're going to use radio blocks radio blocks as demo we have a simple markdown here so if you want title you can do title whatever you want q a bot with hugging face models and we can say we want the first tab which is gr dot tab input text document here they have a you have a text input you have a text output and then you have a text button now when you click the button then the function this is called with this input and this output so basically you're telling this that when the first tab in the first tab you're going to have three elements on the screen an input text box an output text box which you cannot edit and then a button and when the button is clicked the function called build the bot is going to be called with the input text and whatever the output text is being returned by the function will be displayed in the text underscore output now the second tab is called knowledge tab now what do we do in the knowledge tab we're going to build a chatbot that's why gr dot chatbot and we are going to have a text box where you can send a message and instead of having a button to create a chat interface we are going to use the text box itself as a submit option so you can submit like directly from action from the text box in itself and when you do that we are going to call the function call chart and instead of sending only one object we are sending a list here which has the chat bot which is the chat history and also the message which is the user what they've typed and we are going to get the response as chatbot again so instead of sending a string getting a string we are going to send the chat bot and again get the chatbot as an output that's what this yielding is going to the chart is three plus user input response is going to help us do so now at this point all these are good and well the UI has been designed the way we wanted and you can see that we have got the demo we are queuing it and then launching it I'm doing it debug is equal to true just because I wanted to see if there is any error but if you are very sure that there is no error you can remove that I am going to run this and then I'm going to show you how the interface looks like and then I'm going to give you a quick recap of the code and also this time instead of Ashton Kutcher we can try something so let's see the code is run I'm going to click this link this is going to open in a new tab for me I wanted it to be new to q and a bot with hugging face models cool well and good so this is the place where we can input a text box so I'm going to go to Wikipedia and I'm going to look for somebody new so I'm going to say Elon Musk okay probably I don't know how much the bot is going to screw up but let's see Elon Musk I'm going to copy all these things it's quite a big page honestly it's a big page copy this go to the radio application in the text box paste this and then click build the bot once you click build the bot it's going to take a bit of time it took me about like 200 two point something seconds then it said indexing is successful so indexing is done I'm going to go to the knowledge what I'm going to ask a simple question click here send the document what is this document about it's it says Ashton Kutcher is this still about Ashton Kutcher let me go see if there is any indexing issue so the reason why the result showed Ashton Kutcher is because I made two rookie mistake one when I was reading this user input you would have seen that I was using text one which is wrong I should have been using input text and the second mistake is even though that we are building an index we are not actually making this index available globally so we had to make this index available globally and that's exactly why you can see that now I've made the index available globally the right way to do is to return it or store it as a Json object and then retrieve it but just because this is a demo I'm making this index available globally the disadvantage in this case is if you have got multiple objects with index then it might mess it up so Global is usually not advisable to do it but because I know for sure that I'm using only one index here this is going to work fine so I'm going to rerun this code like I said I've fixed two things one is the text list where I'm using input text second is making it index globally available so I'm going to rerun this again this this and this and let's see what is going to happen this time when I'm going to ask questions about Elon Musk so I'm going to go click the link now once again paste the article from Elon Musk and then click build the bot and then ideally it should say index F successfully go to the knowledge bot I'm going to ask what is this about it is going to tell me Elon Musk maybe no it's his Tesla that's fine and then let's ask another question saying when was Elon Musk born and let's see what does it say it's going to tell us that June 28 1971 so let's go see if it is June 28 1971 okay it's June 28 1971. so the next question that we are going to ask is we are going to ask what all the companies started by Elon Musk and then let's see if it answers one of the questions right like at least like one of the companies okay it says Tesla that's fine is there any other company Elon Musk started other than Tesla most likely it will say yes I think let's see okay it's a Tesla it just said Tesla anyways so we have successfully learned how to build a knowledge bot if you still dot the code I would like to quickly show you one more demo so instead of Elon Musk let's pick somebody else in this case I'm going to get Bill Gates so get Bill Gates get everything about Bill Gates probably this is fine yeah copy this go back to the pace paste it click build the bot it is going to now create the index save it go to the knowledge bot and then ask what is this document about and then let's see what it says it says it's about Bill Gates okay who is Bill Gates and it's going to say most likely Bill Gates it's just Microsoft okay fine so we have successfully learned to build a boat you can now optimize this board you can you can use different large language model and play with it but at this point we have successfully managed to build a chat bot on top of radio I did not edit out my mistake because I wanted you to know what kind of mistake can cause what kind of problems I have I've kept that if you are annoyed by that I'm really sorry that I kept my mistake in the video but I've heard from a lot of people who appreciate when I show my mistakes and the like the bugs and the errors that how do I resolve it and people have told me that helps that's why I've kept it so so far in this video we learned how to build a UI a radio friend and for a knowledge bot that we learned to build using flan open source model GPT index a llama index and also Lang chain if you have any question let me know in the comment section otherwise see in another video peace