Create Chatbot: Turn ANY Open-Source LLM into a Conversation Pro! 🤖

Key Takeaways

hi everyone I'm really excited to show you how to run any large language model as a chat bot let's get [Music] started I have already covered this program in the previous video in regards to create a chat bot using chat GPT API I will link that in the description below so it's just few lines of code input open AI then create a chatboard function the chatboard function will get the prompt and return the response in the main function you're asking the user to to add an input if the input is quit exit or buy then it should break that means exit the chatbot or else print the response from the chatbot now from here how we can change this to a large language model we are going to use text generation web UI I have already covered how to set text generation web UI in my previous video I will link that also in the description below now we going to go to the folder where we created the text generation web Bui and Bash start mac. sh and click enter this should automatically start the text generation web Bui and you can see the IP address here now I am in the text generation web UI interface here I'm loading the mral 7 billion parameter model load here you can see successfully loaded Mr 7 billion next you click the save settings this should automatically save the settings then go to the session tab at the top click that then go to the available extension section and click AP I open AI again API and listen that's it this will automatically allow open AI endpoint to be used in our application apply and save this will reload now it got reloaded now let's go to the terminal in the terminal you can see the open AI API base as this port and the URL I'm going to copy just the URL right click and copy now in the code Type open ai. API base and then equals enter that URL that's it now we are going to run this you keep the other terminal running now navigate to the chatbot folder type Python app.py and click enter now the chatbot got initiated what is your base AI model you can look on the right hand side it'll get automatically generated my base AI model is Transformer based a you saw in the right hand side the web generation text UI got updated so whenever you type something here on the left hand side automatically the right hand side will get updated let's ask what is the capital of UK now you can see on the right hand side it's getting loaded it is London that's it just by adding open AI API base you are able to talk to any large language model which you download in text generation web Bui it doesn't matter what ever name you gave here it'll automatically refer back to the API base I hope you like this video please do like share and subscribe thanks for watching