AutoGen + Open-Source LLMs: UNBELIEVABLE! Step-by-Step Tutorial You Can't Miss! 🤯

Key Takeaways

hi everyone I'm really excited to show you autogen with other open source large language models let's get started first let's create a virtual environment cond create minus n autogen python 3. 11 now the virtual environment got created to activate the virtual environment goonda activate autogen this will automatically activate the virtual envirment next pip install P autogen enter this will install the python package for autogen next let's create a file called app.py touch app.py previously I've shown you in a separate video as a beginner guide on how to set up autogen so this will be an extended version of that so if you haven't watched that video I will link that in the description please do watch the video first import autogen next assistant equals autogen do assistant agent this is the assistant agent and we name it as assistant now we'll have a user proxy autogen user proxy agent you are giving a name as user proxy after that you're going to enter code execution config equals and you're going to provide the working directory that's where the code everything will get saved I'm going to mention that as coding that is my working directory that's it the basic setup is done to initiate a chat we are going to type user unor proxy do initiate uncore chat then we are going to give the name of the assistant and the message this will be what we want to create generate five even numbers that's it now we're going to run that python app.py this will automatically generate five even numbers and save that in the folder I'm going to click enter this will continue execution of the code now you can see a new folder called coding got created you can see an even numbers 2 4681 got printed so this is using open AI because initially I exported my open AI API key equals the API key which can be generated from opena platform website but in our case now we are going to change that to a local large language model which is the open source to do that we are going to use LM Studio this also we have seen before I'll link that in the description below so to download the model go into the models tab you should be able to see the list of all models available I'm going to download myal 7 billion instruct model whatever model you like you can go and click the download button which is listed there once that is downloaded you should be able to chat directly here and you have chosen that model at the top here we're going to make this as a server so there is the option if you click the icon on the left hand side like a local server click that icon then you should be able to see the settings we are going to click Start server by clicking this button you'll be able to get code completions and even query using this API so the API documentation is provided here so I'm going to copy this U URL and take this back to our autogen project in autogen we are going to add another config list which is like a list consist of API base API base and that's where we are going to paste our URL which is the API and then we are going to mention our API key that will be null and new LL then this config list will be going to the llm config there we are going to mention request timeout as 800 and config list as config list request time out is the amount of time it takes to get a response from the large language model just because we are running it locally sometime it might take a long time so I've increased the timeout but in regards to chat GPT it should work even without mentioning any timeout now we got the llm config I'm going to pass that llm config in to the assistant so llm config equals llm config that's it we are adding just two section of code which is config list which contains the API base and API key and llm config request timeout and config list this is the only difference we have currently compared to the previous demo which I showed you now we are going to run this python app.py now it's querying the Mr 7 bill parameter model and we got a response here with the code as well we type exit now here I'm going to query what is your name and what model are you can write this and then Python app.py and now it says my name is mrl and my model is mrl 30b V1 so you can clearly see that it's saring from the mrl model you can change whatever model you want in the LM studio in LM Studio at the top you can see what model you want to choose so in this case I've I've chosen mral instruct 7 billion parameter model if you see there's one more thing you might need to note to increase the context length here I have increased that to 1,500 you might need to change based on the amount of text you are sending to the model that's it start running open source model locally on your computer and integrate that with autogen I hope you like this video do like share and subscribe thanks for watching