Develop Unlimited Chatbots Free Custom Data Set RAG & Open Source Models, OpenAI's Approach Included

Key Takeaways

hi mes how are you uh so we are here to see how we can uh work with open AI apis how we can consume them uh plus if we don't want to use open AI then how we can uh elaborate some custom models how we can integrate it uh within our python code and get use to some amazing stuff that uh over the Internet we have our different platforms so uh let me start introducing myself you can see my details from here and I have various repositories I generally push C reled to a lot of text Tex which I have worked with or I'm working in future so let's continue our main focus is this rep at the moment so I will teach you and guide you how it can work step by step so get started with me uh it is not mandatory that you have some specific OS you can execute in any OS all of these commands uh but make sure you have talker installed and you also have get install uh it is it is not a very beginner uh beginner friendly execution so you should be some type of level developer at least to get understand all of this stuff so let's get started first I have copy this first command and I have enter it now it will clone my Roo I can see that uh it is cloned in this current path that I have let's see into it now uh I have executed this and this both of these commands now I will Docker build it will take some time I may I might pause the video so you don't have to wait for this long if I go to if I go to my that [Music] folder I okay so and if I do code it will open this folder in the vs code so let's understand uh a few things first of all you see that uh uh there is flask written here don't I confuse with uh this name uh actually I have initially decided to do a lot of work uh over this ra but I didn't get enough time to make it done so please ignore this FL there is nothing in this repo done yet which is based on specifically flask so what you can see there we have one service our main code is written in the service plus we have this few services for uh M which is Vector database which we can see later how we can uh work with it so our Docker is just uh has bu now we can do composer so it will take some time as well uh what we can do uh we can see the code behind the same so uh these are the services uh which will be up in in few minutes so uh what you can see that we have two folders here like custom and openi so there is nothing happen is anything relevant code in the folder of app so just focus on these two folders custom and open so if you have uh access to open API key then you can create aemv file here and you can follow this step here you can put this key in this format okay if you are choosing this uh to execute it from open a and I will also show how you can use it without open a okay so first of all uh we have to understand uh embeddings first of all this file will create the embedding uh if I can see that I have this file cities uh I have just uh developed a test case for myself that I have few list of cities and uh you can see they have following attributes and data of fous places foods and tourism info so what I can do I can uh if my user is ask any thing relevant to that cities like therefore like what kind of T place is best for them so my my chatbot or using open or custom models uh we will have we can answer all of these queries so what what is the process behind this scene because computer do not understand the text and it is not like just like your SQL queries where you do where and likes it is it is some some little bit advanced stuff so if you really want to understand what embedding is and how it works and how computer understand this I will give you these links uh of a few videos you can watch with them and hopefully you will have a good idea how how it works so first of all what I do I I have this file this C is embeddings you can see that I have the same description uh that I have generated from from using uh this data and there is an embeddings a huge list of embeddings here uh else nothing else okay so what I did I have uh write some description from the information from the any information that I have stored in that file so it is generating embeddings uh you can get a very good idea of embeddings about this I have already told you so please watch this uh before uh before moving forward so it will be more easily understandable to you so what I do I just delete the content of this file please do not delete this file just follow along with me and empty the content of this file and we will write few commands uh this embeddings will be autogenerated from from the custom model because we are in custom folder so so that that that's a little bit of understanding now let's move forward and see if if our has been installed or not yeah it is up so let's move forward with next step we are here now our composer is up let's go to this container which I have shown you here in this uh doer compos ml file we have to go to inside this container to execute our commands so let me open a new terminal I have copy paste that command now I am inside this terminal now I am not showing uh this CD open AI stuff at the moment just navigate to this CD into custom directory now you can see that I have explained you what this will do this will actually create our embeddings from by reading uh this file and embeddings will be generated here so let's run this command here now it is downloading uh the model that we are using uh because it is not downloaded in my system please uh wait for some time now the model is downloaded you can see it is generating this custom embeding and with the the dimension of 768 so what dimension really means you can see that this file that we have blanked it is already populated as well so dimensions are basically uh these few Flo digits with the count of for example if I count them 1 2 3 4 5 so they will end up to 768 uh it depends on which type of model you're using different type of models will uh reflect different dimensions here so uh you can see the code here I am using this B model uh you can see it is reading this file from here and the generation of embeddings is actually happening is in this code this uh will be have description here and I am passing that description in return it will provide me embedding and I can I you can see that we can check the shape of the vector shape shape of the embeding written here which is 768 which is actually dimension for us so I have appended them uh in this simple array and finally we are going to save this in that new file this file is execution has is started from at this point so so our first file is done uh you can see how it is able to generate embedding so next uh we will see how uh we can uh dump these embeddings into any Vector database here I'm using you can use f chrom DB and there are a lot of others as well so you can do it with any database so I will execute this command in my project folder um keep this in mind that I am inside uh the docker container so do not execute this command uh in your host directly make sure you are inside your Docker container just execute this file you can see it's a data insertion completed we will see the code of this file uh if I go to milus we have ATU here it is a GUI of Milas you can see this uh we have it uh we have this data here and if I go to this part this is schema you can see we have defined this Dimension 768 so if you are planning to use this quote uh then make sure that whatever shape you get from here which is 768 in our case then in second file you also have to Define this number here otherwise it will not work uh because if you use open a I will tell you later that that Dimension is 1536 here so if we go to this file we have just made a connection to Milas then we have defined the collection name plus the schema kind of things embeddings float Vector plus this Dimension this max length and we check that if this collection is not created so is not in the list of the collection we will create it otherwise we will uh not do anything things so what I do I do here I have just read that file that CZ B file which is generated in our previous script and I Loop over uh the the object that we have here like these these objects individually one by one and I inserted it in M okay so we have logs here that the document inserted successfully but if we go to here and we can see this all collection is one but loaded collection is one so but I I you can see that my data is inserted but I cannot see it here because you have to load that uh data uh first to see it for that part we have to run our this this script which is uh doing indexing and uh loading uh the data into Milas uh you can see this file here we let's execute it you can see it is creating an index and it finally says Milas toal loaded so the same content we have here as well that we have created uh our connection with the mil which is in Vector database uh plus we have just uh load this collection now if I go to this my GUI here you can see it is saying loaded collections one which was pre which was Zero previously so we can see and now we can perform searching over this over this part plus if I go to next part it will say perform search over with eming so let's execute this file what it will do it will actually search uh in the Milas uh with the provided query so we can go to the code of this file we can see that uh we have some standards import uh we have defined our models plus what we have done here you can see if I go to this part uh I go to this part where is my query this this one is my query actually which uh which the which I I want to search because this is actually the question of the user you can take it like this so I have written here Bollywood glamorous City okay this is my user query now I have I I of systems or computer do not understand what this say because uh we are not doing an exact M kind of scenario so we have to uh generate embeddings of this as well so then later we will do a comparison from an embedding to embedding means for example uh we have embedding of this description in in in this form but when my user ask this question I also create embeddings of it and that embeddings will be compared with all the embeddings that my Vector database have and my Vector database in return will uh return a score with the matching content as well so so let's see how how it it work so here I have generate generated embeddings of the of the you can see this function if I go to this definition uh it is just generating its embedding and nothing else using the same model make sure the models that you use for uh generating your embeddings is same okay so otherwise it will not work so uh we have this embeddings of this user question here in this variable now we will pass it to this is the connection object of mil database which is Vector database we will pass we have passed our Vector here now in return return it will it will return some results with some matching scores you can apply some conditions uh by seeing score so if we go there it will say final six time content the score is matching is 50.1 now the score is depend on your model if you use openi it will give you score close to one uh 1 point something and sometime lower than one as well but this model is uh giving S A score in this format so we can see that it it returns Mumbai is a city in India but if you can see here that uh I have not given anywhere India here but it's still able to find uh the relevant content this is the power of embedding uh that it can search uh within our relevant content so now my relevant content is here some uh in nowadays we can also uh called is a rack system uh to keep my models updated because model have outdated data so let's go to the next part fifth part now we will do uh a chat completion uh with it let's see how it works now I have I haven't used exact this example here because uh this chat completion is the most complex part of uh of this uh this whole part which we are doing uh because it is very hard to generate an exact and a good quality of text with with some model you have to find a very very good model uh to make it work according to your needs so my motive is just here to explain how it can be made work with so I am using this gpt2 model it is not a very good model I have explained you already so what I am doing here I am giving him the question just like user have asked I have I provide some context that you should uh explain uh this within this uh from this context please reply from this context the this is the user question and this is the context that we provided now if I hit this python command now it will download yeah it will take some time so please uh wait now you can see I have just downloaded uh that model uh it is a very tiny model gbt2 uh just 550 megabytes so you can see uh this is the question and this is the context that we have given so it generate answer that France is a country located in Western Europe its capital city is Paris the country's economy is growing at a rate of 2.5% a year according to the and and it stops so uh basically I know the result is is not very good but you can get the idea that my question is this I have provided this context but it is uh it has replied with some uh some text which is not part of my context so means uh it will able to generate uh some responses just like we do uh we can see how chgb is doing it with its 3.5 turbo model so that is the idea behind the scene how all these things work so let me uh repeat in in shortest possible steps how what we have done okay so let's go uh first of all remember that we uh the search that we are doing here is not an exact match it is actually a semantic search okay so what we have done we have a data just like for example you have your cases as well for example just assume that we have this data uh in in our database we have created edting of that data by by converting it into some meaningful information you can see these Tings here okay so this is the first step now in second step I have inserted this data in my Vector database because only the vector database will be able to search through your embeddings not your normal database will not be able to do it because it is it is a stices okay so keep this in mind so we have in next step we have inserted this data into the vector database in this way now in third step we have loaded this data into M so we can cury with it in fourth step we we have done searching from embedding to embedding actually when user ask the question we have converted it it into its embedding and then that embedding will be compared with the embedding that we have generated which we stored in M uh VI semantic okay so please focus on keywords that I'm using here you can use these keywords to get more clarity if there is some confusion or you can ask me or contact approach me if you have any difficulties understanding this now in chat completion part uh the I want to understand the process that how how it works when user uh raises any question okay when user this is the most important part when user raises any question we will create eding just like we did here okay and we will uh pass that eming to the vector and it in return it will uh return some uh some sort of content which is from our database which is match from our database so what we do here we will put this content here in this context okay uh because uh uh we we why we did that we are doing this because we might have some updated information that our model does not have okay or we want our model to reply from uh this data only so we have some prompts as well we can Define some prompt there are there are multiple uh uh multiple ways of defining your prompt it is based on model what kind of uh request or or pil it is expecting or it is in one word format it is okay so it it's very easy if you understand till here up until now so it's it's very easy if you go through the documentation there is some uh some system and some role some user some context type of things you can you can Define it here it's very easy so you have you pass this uh data from your database to this value to this variable you can Define any variable it is not important that you have to name it context but uh in in this field normally we we name this as context so we pass the both of these information to the model through the chat completion model and the chat completion model will understand uh this and reply just like this now we can uh pass different types of information uh but it depends on our model whether it is an instruct model or not so if you are going uh to Hing face and searching for the model try to use any instruct model okay so I will post these links with the video as well if there is any confusion please let me know I know uh I have tried to explain it uh in a short manner now plus uh let's have a quick walk through over this open AI part as well if you have an open a how you can do it so for that I have to CD into open a first of all I have to move One Directory back and uh the second important step that I mention here you have make sure that you this en file is created with this value and you must have your open a key in it in cour as well so I am here now the files names are same so if I go here I just close this customer and I close I open this open a you can see file names are same so if I quickly go here and I make this file empty I have the same file in open a under the open a folder as well so if I quickly execute these files one by one you can see it will use open a to generate emings if you can count it it to it the dimension of this are is 1536 we can see when we go to this script now the documents that we have generated edings has been inserted into Milas now in third script it is loaded into the Milas okay in fourth step we are doing searching you can see because the question has been changed here if you can quickly I can show you uh here you can see uh can you mention a popular uh past often associated with French cuisin so uh definitely it will bring up France uh the code will remain mostly same I will have a quick walk through now if I go to chart completion now you can see now you can see let's go through this this file is almost the same uh only that we are just using open AI here you can uh see that we can uh Define it like this okay so all the codes are exactly the same except that part okay so this one uh we be dumping our data to milus this one is same as well and now in third part this one is same as well now in fourth part where we have um we are doing searching from Milas it is same as well uh the only difference is in this two file this CB2 this chat part two and cv5 chat part 5 so you can see that I have just given this Q here uh this is the system content this is the prompt okay you can see that's what I am explaining you that we can Define any model here uh because we are using open a now you can see that we have this response from a you can see it's quite uh impressive uh it is the power of model that you are using because open I has number one models so their response is the best you can see that we have not provided any specific information uh in in the system content but is it still able to generate a pretty good text so so so so that's all thing you can you also have a very good idea how we can deal with open this repo is here uh at my profile you can see it and please execute it step by step and make sure that you execute every command from inside the docker container if you execute it outside it will will produce uh errors there are some helpful command as well if you have some issues uh you can see uh this you can do this Docker psfi quickly show you these commands as well for example if I want to see but this command should be written uh should be executed for two your hosts so I can see I want to see that these containers are Runnings okay so if there is any error popup uh please uh make sure that it must be from your host t Okay so try to resolve with this do not uh think that there this repo has some issues in code and here because Docker is always work on any system plus you can see the logs of the container if it is fine or not you can see it's fine or you can delete all your Docker images or volumes from this command if you have only uh if you have only set up this first time in Docker so please execute this so it will free up your memory which have been consumed by executing these commands uh thank you so much