Create a LOCAL Python AI Chatbot In Minutes Using Ollama

Key Takeaways

today I'll be showing you how to make a python AI chatbot in just a few minutes and the best part is this chatbot will run locally meaning we don't need to pay for a subscription or connect to something like open AI now the first step here is to download oama so go to ama.com Simply press on download install this application and this is what we'll use to run llms locally on our machine once you've installed o llama the next step is to make sure the installation is working properly to do that you're going to open up a terminal or a command prompt and you're going to type in the following command which is simply olama now this should be working if AMA is running on your computer and assuming this command Works you're ready to move on to the next step now llama is software that allows us to download and run open-source llms I have a list of them on the right hand side of my screen and I'll link this in the description you can see we can access llama 3 with 8 billion parameters 7 billion parameters we have a lot of other models available to us as well now notice that these models vary in size and the larger the model is the more difficult it's going to be to run in terms of the type of Hardware you need on your machine now if you scroll down here you'll see there's some different specifications you should have at least 8 GB of RAM to run the 7 billion parameter models and then 16 to run 13 billion and 32 to run the 33 billion models so make sure that you meet that criteria before pulling these specific models now for our purposes we're just going to use the Llama 3 model with 8 billion parameters and in order to get that on our computer we're going to type the following command this is a llama pull and then the name of the model that we want and this is simply llama 3 when we do that it's going to download this model for us this will take a bit of time depending on how fast your internet connection is and then you'll be ready to use it in my case it's already downloaded so it happened instantly but for you it will take a few minutes to download and then I'll show you how we can test it now that the model is downloaded we can test it out and in order to do that we can simply type amaama run and the name of the model that we want it's worth noting that you can have multiple models here and you can use them by simply specifying their name so I'm going to run llama 3 I'll just go with something like hello world and then you can see that it actually gives us a response if you want to quit this you can type slash bu and then you will exit that prompt so now that we have access to this llm it's time to start using it from python Now using llms like this in Python is actually surprisingly easy and you don't need to be an AI expert in order to work with them however I always find it interesting to learn about how they work at a deeper level and that's where our sponsor brilliant can help brilliant is where you learn by doing with thousands of interactive lessons in math data analysis programming and AI they use a first principles approach meaning you'll get the why behind everything each lesson is interactive and filled with Hands-On problem solving which is six times more effective than just watching lectures the content is created by award-winning teachers researchers and pros from place like MIT Caltech and Google brilliant focuses on improving your critical thinking skills through problem solving not memorizing while you're learning specific topics you're also training your brain to think better learning a bit every day is super important and Brilliant makes that easy their fun bite-sized lessons fit into any schedule helping you gain real knowledge in just minutes a day it's a great alternative to Mindless scrolling Brilliance even has an entire AI Workshop that deep dives into how llms work and teaches you about the importance of training data how to tune in llm and more to try everything brilliant has to offer for free for a full 30 days visit brilliant.org techwithtim or click the link in the description you'll also get 20% off an annual premium subscription now the next step here is to create a virtual environment that will install a few different dependencies in that we need for this project what I've done is open to folder in Visual Studio code and the command I'm going to use now is python 3-m ven V and then I'm going to give this virtual environment a name which is chatbot if you're on Mac or Linux this will be the command if you're on Windows you can change this to Python and this will create an environment for us that we can have some isolated dependencies inside them you can see the chap off folder has now been created and the next step is to activate the virtual environment and then install our packages now if you're on Mac or Linux the command to activate this environment is the following it is simply Source the name of your environment which in this case is chatbot and then bin SL activate if you did this successfully you'll see that the name of your virtual environment will prefix your terminal now if you are on Windows the command can vary depending on the shell that you're using one of the commands that you can attempt is the following back SL venv or in this case it's going to be chatbot the name of your virtual environment SL scripts with a capital slash activate Dot and then this is bat executing this should initialize the virtual environment if you are running in command prompt if you are running in Powershell then you can change this to say PS1 and put a capital on activate so try one of these commands to activate the virtual environment on Windows and again make sure you have that prefix before we move forward now the next step is to install the packages that we need I'm just going to make this a bit smaller so we can read all of them we're going to install the Lang chain module the Lang chain dasama module and the olama module so go ahead and hit enter this will install it inside of our virtual environment and then we are good to go and start creating this application that will use our local llm The Next Step here is to Simply create a python file we can call this something like main.py and now to start writing our code now in order to interact with a llama what we're going to do is say from Lang chain uncore oama we're going to import the ol llama llm now here we can connect to AMA so we can say that our model is equal to AMA llm and and then all we need to do is specify the model that we want to work with now in this case it is llama 3 but if you have a different model you can put that here now we can actually use this model in order to use it we can say model do invoke and then we can pass to this function here a prompt that it will act upon so you can see that I can pass some input say input is equal to hello world we can store this in some kind of result and then simply print this out to make sure that it's working so let's print out the result and execute our code so from my virtual environment I'm going to type Python 3 and then main.py and notice here that we're going to get some warning you can ignore that for now and you can see that we get the response which is hello there it's nice to meet you and we've invoked the model with this input so that's the basics of interacting with the model but I'm going to show you a slightly more complicated script here that explains how we can pass some context to the model and how we can make this a little bit more user friendly and have a full chat window interface to interact with it so in order to do that we're going to start by bringing in something known as a prompt template so I'm going to say from Lang chain core. prompts and we are going to import the chat prompt template now Lang chain is something that allows us to more easily interact with llms and one of the things we can do is create a template that we will pass to the llm that contains our specific query or prompt and this way we can give it some more description and instruction on what it should do so I'm going to say template is equal to and then I'm going to do a multi-line string which is simply three quotation marks and I'm going to tell the model to answer the question below but you can give this any kind of detail that you want now I'm also going to provide to this some context I'm going to say here is the conversation history and I'm going to pass to this the context now whenever I want to embed a variable inside of a prompt that I'm passing to the model I can surround that in curly braces which I'm doing here and then I'm going to say question and I'm going to pass the question variable as well and then I'm going to have the answer which I'm expecting the model to generate so you can see I'm giving this a template for how it should behave or respond to my queries now we have our model the next thing we're going to do is create our prompt so we're going to say the prompt is equal to the chat prompt template. from and this is going to be template and then we're going to pass that template that we just wrote which is called template now we have a prompt and we have a model and what we need to do is chain these together using L chain so what I can do is say chain is equal to prompt and then I can use this pipe operator and I can pass my model what this will do is create a chain of these two operations so the first thing we'll have is our prompt which we'll have the question and context embedded inside of then we will pass it to the model where it will be invoked automatically so in order to use the chain now we can change this slightly so rather than model. invoke we're going to say chain. invoke o but this time what we need to do is pass to it the various variables that are inside of our prompt so we're going to say the context is equal to in this case we don't have any context so we'll leave it blank and then we'll say the question is and then whatever our question is and we can just say hey how are you so hopefully you get the idea here let me make this a bit bigger so we can read it we're simply embedding these two variables inside of the prompt and then passing that prompt to the model where it will be invoked using Lang chain so now we can test this Python 3 main.py and it says I'm doing well thanks for asking now that's great but we want to be able to continually talk with the model and store a conversation history so let's write the code to handle that so what I'm going to do now is create a function called handle and then conversation and this is where we'll kind of put all of our main code inside of here I'm going to start by storing some context which will just be an empty string and I'm going to print some kind of Welcome message so I'll say welcome to the AI chatbot and then we'll just tell them that they can type something like exit to quit so they have a way to get out of this on the next line we're going to make a while loop and we're going to say while true and inside of here we're going to collect some input from the user so we're going to say user input is equal to input and we'll just put U col in and then a space that the user has the ability to type we're going to say if the user input. lower so converting this to lowercase is equal to that exit keyword then we are going to break out of the loop so that we don't have an infinite Loop now next we're just going to generate a response so we can take this right here and bring this up so let's paste that inside of here and indent this properly okay let me just get rid of this print now we're going to say the result is equal to chain. invoke but this time we're going to pass the conversation context and we're going to pass the question the user asked which is simply the user input now what we can do is we can print the response so we can say the bot said and then whatever the the result was here and then what I'm going to do is just store all of this in the context so that this bot has kind of the conversation history and it can respond to previous things that have been said so I'm going to say context plus equals to and then we're going to use an F string available in Python 3.6 and above I'm going to put a new line character with the back sln I'm going to say user is equal to and I'm going to embed the user input and then I'm going to do back sln and I'm going to say the AI and then this is equal to the result so I'm just storing what the user asked and what the result was so now every single time I use this prompt I'm passing all of this context back into the bot so it knows what was said previously and can use that to respond so that is our function now all we need to do is call that function so I'm going to say if uncore name equals equals uncore maincore uncore this just means that we're directly executing this python file and then I'm going to say handle conversation and now we are going to run this function function which will simply ask the user to keep typing in things until they hit or enter exit and it will store the conversation history and pass that back to our model so that it can respond based on what we said previously so let's test this out and make sure it works Python 3 main.py welcome to the AI chat bot we're going to say hey how are you doing it's going to give us some response I'm going to say what is your name great no personal name that's fine can you help me understand and history let's see what it says and it gives us this long response here which I imagine is saying yes it can help us if we give it a good question let's try to exit with exit and now we are gone and there you go we have now created a program that we can actually interact with a local llm we don't need an open AI key we don't need to pay any third party service and we can run this open source model on our own computer and utilize it from our python script obviously this is a very simple example you can do some really cool things that are a lot more complicated but that's what I wanted to show you in this video hopefully you found this helpful if you did make sure you leave a like subscribe to the channel and I will see you in the next one [Music]