Building Custom LLM Researcher Agents with Telegram, LlamaIndex & Python

Automata Learning Lab · Beginner ·🧠 Large Language Models ·1y ago

Key Takeaways

Building a custom LLM researcher agent with Telegram, LlamaIndex, and Python, integrating OpenAI's capabilities for task management and leveraging llama-index for creating useful agents.

Full Transcript

what's up folks welcome back to the channel in this video we're going to build a very simple Automation and I'm going to demo it right now so that you see what we're going to do so I'm going to start it off and I can send a message like uh research about prompt engineering and I can give my agent a few tools and then uh research about how to use llms for code generation and also I can give this agent a few tools so essentially I'm going to be Distributing tasks for agents with a customized set of tools that I input during runtime and then I can just send this off and then what that does is it's going to trigger an automation that it's going to yield a response that not only the model is going to be uh performing the task that I asked for in telegram but the model is writing down files and whatever other artifacts needed locally on my computer and we're going to learn how to set this up and it's pretty fun so let's get started okay so I'm going to minimize this for now and what I'm going to do is I'm going to go over here and I'm going to stop running this agent for now and we're going to build it from scratch so that you see the entire process how it's done uh how it's organized I kind of organized it here so then uh you can follow along let me just organize here cuz uh the code is quite extend so I'm going to use some check codes here on the bottom folks and let's see what happens okay so one of the first things we're going to do is uh we're going to start with the AI code first and then we're going to write the code to integrant to integrate with the telegram bot all right and then I'm going to walk you through uh the API tokens that you have to set up and all that stuff so it's pretty simple so first things that we're going to do is we're going to import uh ptic for the structured output uh stuff that we're going to use and I'm going to save this as uh agent so I'm going to save this as distributed agent Tasker something there you go and then we're going to use uh open AI so I'm going to say from open AI import open AI so that we can import G for ag4 Etc and then we're also going to need uh lists from typing so that we can do some type hinting uh we're going to import OS we're going to import sis to handle inputs from uh the command line so for testing and then for the agent stuff folks this time we're going to use LMA index so I'm going to go LMA index. lm. openai import open AI uh import open AI as uh L open AI do I kind of don't like this this import but okay L open I like this and then from LMA index. core. agent import react agent which is going to be responsible for doing the the search and all the good all the other stuff and then from Lama index from Lama index uh core. tools import function tool so that we can create different tools for different purposes during this uh automation all right and then finally we're going to use requests for uh handling web scraping tasks when when needed and uh beautiful Soup for the same purpose all right folks and then after that what we're going to do is we're going to use Sur API which is from that Google search results python package which is going to handle the the part of the code that's going to be the searching uh the web search stuff so it's the web search tool that we're going to use okay first thing that we're going to do is we're going to set up an open API client so I'm going to say client is equal to open a now if you don't have an API key set in your environment do that all right so set something like API key you call and then you put your open AI API key all right beautiful stuff so if you don't have that do that I already have it as an uh environment variable so I'm not going to be setting and then we're going to set up the models we're going to use so I want to I like doing that in the beginning of the script so I'm going to say model task Handler to be uh GPT 4 mini and this model is going to be responsible for um uh taking a big string containing multiple tasks and then um extracting exactly all the tasks inside okay pretty simple task and then we're going to also use model agent and this is the model responsible for actually doing performing and executing each task and for that we're going to use a more powerful model so we're going to use GPT F right and finally we're going to use a folder base that's going to contain all the artifacts that going to be generated during this execution and that's going to be called Agent artifacts all right so everything that it's created during this uh uh during the execution of any agent is going to be saved under this folder called Agent artifacts all right so now that we've done that what we're going to do what we're going to do is we're going to set up a few extra parameters and we're going to start writing the actual code so the first thing that we're going to do is set up the um parameters for the Sur API which does the web search so we're going to do Sur API params is equal to and we're going to set the engine to Google and besides that we could set a bunch of stuff but all we need to set is this and the API key which if you don't have it you should go set it up and I'm going to leave a link to description in for all the websites that you might need to go to set up your API Keys essentially we're going to have to set up Ser API key open your IPI key if you don't already have that and the telegram bot token which is pretty easy to do when you have the telegram app installed in your machine all right so now that've done that folks we can actually start writing some actual code so the first thing is that we're going to use pantic to do structured output extraction from the string that the user sends to this to transform that into a bunch of tasks so for that we're going to create a class called task Handler and then we're that's going to inherit from base model and what that's going to do is it's essentially going to have one attribute which is going to be a list of strings which is going to be a list of all the tasks contained in the message sent by the user all right and then we're going to give that description here so it's list of tasks um uh list of tasks extracted from the users message okay and uh once we've done that we can now write um a function that makes a call to the chbt API and leverages this class task Handler uh to uh as the response format so that when we send a message to the to the model saying look um these are the tasks the model is going to return a list containing all the discretized tasks inside so that's pretty easy we're going to write a function called generate task list and that's going to take in one input called task input and it's going to return a list of tasks all right a list of tasks perfect so now we do that and what we're going to do is we're going to first let we can document this so generates a list of tasks from the user's message and we're going to set response equal to client. beta. chat. completions dopar so this is because we're going to be doing structured output extraction here and we're going to set the model to the model task Handler that we said before and cursor is already helped me out here so the messages to the to the model are going to contain not only that so this is missing one thing and that's it the system message and the system message um is going to be the message that that describes the behavior that we expect from the model to extract these tasks okay and to not be writing all of it again I'm just going to copy the system message that I have here all right so let me just put so this should be here so this is row and then this is system and then the content of the system message is going to be what it's going to be uh something that I'm just going to copy from the reference code that I have here folks because because it's way too much text for me to write again but we're going to walk you through what that Co what that message is saying so don't worry about that and let's do that okay perfect so we're copying it and here we go folks there we go we go back boom so let's walk you through what's going on over here folks right so we start with you take in so you take in text there's a list of bullet points containing test descriptions with potentially two calling indicator specified with uh at TXS and you output a list of those tasks uh and you output a list of those tasks and you output a list of those tasks where each bullet point is an individual element in the list and for example if the input is basic techniques if the input is then I could put like a so I'm improving as I show it to you folks so that's uh that's some live updates just for youu so if the input is this and this is what you should show there we go so if the input is something that looks like this uh yeah so if the input is basic techniques of prompt engineering and then I give this and then llms for research what is going to be the output the output is going to be essentially a list of those tasks so as you can as you can see here I'm giving the model example of that list should should look what that looks look like to just guarantee that the model doesn't get it wrong and the the weird thing that we're doing here folks is that the desk not only contains the description of what should be done but I'm always and this is just the choice I made right now because I want to play around with this idea I want to create the agent when I when I send a request to the agent instead of having the agent already created beforehand because that allows me to customize which tools the agent can use right from my prompt to it so I'm using this at tag and then I'm saying look do web search do write file do read file and we're going to have um a set of other uh functionalities that we could integrate to this agent all right so we do that that's awesome that's great beautiful stuff so now that we have a system message now that we have the message that we're sending to the model what we're going to do is we're going to uh get that response paret back and we want the parse it sparing it back to have the the list of tasks so that's why response for is set to task Handler right and for some reason this is giving me some grief so let me see what's going on over here so this is there this is over there this is over there okay so I should have done this there you go so now that we've done that folks we can actually we're going to par and return those tasks that are retrieved now this is not a perfect solution because there uh we we could Implement some error handling here to avoid having issues where the um maybe the model doesn't identify any tasks so there's some problem with the extraction but I feel like for this example we're we're good we're good with just that okay so now that we've done this part what we're going to do is we're going to go to the next part which is to extract the tasks and the tools from each of these inputs what does that mean this is what it means um what we're going to have is we're going to have a function that extracts um from a string that looks like this the what is the task and what is the tools that I'm giving to the model which is the the stuff that has this at TAG associated with it now I don't know if this is the best approach but since I'm going to be writing some python code to extract this I feel like this could be pretty uh consistent reliable Etc okay so just for you to get a sense of what that this function does let's just write a a little test here so I'm going to write generate task list generate test list and then I'm saying let's create just um uh like a couple of examples yeah there we go and now that we've created this example let's run it and see what happens okay so I'm going to run python distributed something and now I run it and oh I got an error K an import react agent from LMA index sore. agent that's weird so let's fix that error right now uh cannot import L index C agent let me see here let me copy from my original index okay so llm open import open that's correct this one it's giving me an error LMA index core agent LMA index. core. agent for react agent ah there you go react agent like this I just I I was just getting it wrong the the letters there so now we can run it folks and there you go so the task that we sent now it's uh parsed into this list perfectly we have basic techniques of prompt engineering and then web search and then llms for research and then web search right so that would mean that we have a task that would be about researching about basic techniques of prompt engineering and we give this tool so that the agent only has the tool that it needs to perform what we want the agent to perform now obviously for like many many many tools it would be kind of like ridiculous to expect the user to describe I want want you to use these twos but for me I want to have the most control possible uh when sending the messages to the agent so that I can have some control over the execution of the task okay so that's pretty cool so now that we have that folks we can comment this out and we can write the next function which is going to be to extract the agent tasks and tools all right so let me just uh let's go back here agent tasking tools there you go and tools and what this is going to take is this is going to take in the task input and then it's going to return um essentially what this is going to return is just the um uh a tupple containing the tasks and the tools inside of each of those tasks okay and we're going to show an example to to make that super clear so um T so we're going to have to import as well so let's uh maybe I'm not going to I'm not going to add we don't need typ hinting here that's fine there you go and then we're just going to write a little doc string here that extracts the tasks and the tools from the from each uh task uh for each task in the list there you go so what this thing does is it will and it will Loop over all the it will take in um a string containing the tasks and then it's going to split that into parts and then it's going to extract the part that is uh task and the part that is um a tool so what does that mean that means this so words equal to task input. split okay and then what we're going to do is we're going to say what are the text so we're going to say word for word in words if word if word do starts with this because this guarantees us that we're going to take in the tag the tag is always going to be one full word so we're going to if we use split we can identify this immediately and then when we've done that what we're going to say is look the task is uh everything everything all the words that we have for uh accept the stuff that has the attack so it's essentially all the stuff from task input besides these TXS and we could have done this with index but you know I like doing like this so we're going to say word for word in words if not word. starts with uh tag okay now now that we've done that this is pretty great now I'm going to say task and now that we've done that we're going to say return test. strip and then text all right that's what we're doing that's the that's the whole thing and the reason why why am I doing test strip here uh it's a good question I might have generated this line with AI and I don't remember why I do test Off Script here but let's see what that looks like all right folks so let's print an example extract agent and then here here the agent is going to take in this this task right here right which is going to this is going to be an example of a string extracted let me just go back one there we go and let's see what it looks like so let me just delete this there you go okay perfect folks this is what we need so as you can see the extracted stuff is a string containing the task description and a list with all the tools that were extracted from the from the uh from the string there okay and now that this is working great we can move on to the next part and the next part is going to be to write all the tools that we need to give to the agent and we're going to write we're going to give this agent web search capabilities where it takes in a string and um and returns it returns a string containing the the results of the research we're going to uh give a to to write a file we're going to give a to to read a file and I feel like for now yeah the only one that's going to be extra is going to be the uh web scraping the website which is uh in situations where we you might have like a URL for example that you want the agent to go in take in the information from there and maybe write like a summary and save that to a file etc etc etc okay so um so there is another one that's going to be called scrape web side and then it's going to take in some URL and the output is going to be a string containing the results of the scrape okay so now that we have all of these folks we got to write them down right so uh I'm just going to just to be quick now let's write them from scratch because they can be sometimes a little bit tricky so as sech as web and Returns the results there you go so we're going to be using the Sur API like I was talking about and for that we just need to save Google search and then uh we give the parameters that we specified in beginning of the script right and then we give the query right we set query to this and then we set the number of searches that we want so for that we need to do this and then we say Q to query and then we set n to five we could have a different n with different number of search results this describes how many results we get but I feel like n to five is pretty good okay now that we've done that we're going to get some results so the results are going to be what search. getet dict organic results all right and so this is going to contain the organic results that we got before and then we can save all that information into uh a single big string summarizing everything this is a a trick I've learned by reading a lot of the Lang chain documentation so I'm going to going to say uh n and I'm just going to say n like this and and then like this yeah and do join and then while we're going to join we are going to join what we're going to join uh all the results from the search okay and this search is going to contain what it's going to contain the title of the URL the snippet containing summarized information from that specific resource and the original link okay so what does that look like it looks like something like this uh join X title and I'm not doing this like live folks I'm copying from my example but I find it's more educational if I just write it down so that we can see everything little by little but yeah I definitely would have had some issue remembering all this stuff from uh from scratch there we go so we have the title The snippet and the link and then once we have all of that we just say forx in results we close up that list and this list was opened where it was opened right here Excel and this should not be here it should be right there so that's pretty good and we should be good to go now X title boom bo boom do join yeah and yep I think this is should be everything should be correct here um I see okay I see so this like this there we go and now we return the contexts okay so the the context from the search uh yeah I'm just going to say search results okay search results that's a better way to say it and now folks so for you to get a sense of what this web search function does Let's test it out so we're going to write here print web search what's the best way to learn uh how how to use llms for personal productivity something like that right so we're going to write that we're going to come over here we're going to run this and we got a result issue with Organic results organic results is not get get dick search API API Google search search. get dict that should be correct why are we getting an annoyance here query number FES organic results let me just see if I did something wrong oh that should be correct okay that's annoying k organic results organic results search let me just see if I did something wrong here search get so let me just print search get first and see what happens ER invalid API key ah okay so I invalid API key that's fair uh yeah that's true because that's not the correct name of my API key yeah this is the correct name folks so I just misook the name that I had on my API key so this is the correct name so now this should run correctly let's see so I can run python that distributed something and oh that's weird there you go python. distributed there you go and now it should run and it's still giv me issues with d API key invalid API key that doesn't make a lot of sense serp API key oh it just has the the wrong name there but now I think it's correct there you go folks so this is very cool so this is the result of the search as you can see it contains the title it contains some a piece of text from the resource and it contains the original URL and we have a bunch of those results here all beautifully organized which is very very nice right okay so pretty cool so now that we have that we are very happy and we can delete this one and writing a reader file it's a little bit easier right where're I'm going to let cursor write that for me um and reading a file same thing right and there's one thing we're not going to yeah we're not going to assume that the model is going to update any files we're just going to assume mod is going to write a new file every time all right and we're not going to use scrape website as well we're going to simplify it for now for this demonstration so we have the ability to write a file we have the ability to read a file but there's one thing missing here which is the file path is going to be the joining with the folder base so that we always save it in the place that we want okay so that's what we're going to write every time here there you go and now that we have that that's beautiful we can move on to the next part write a file read a file and now that we have those parts we can uh we can write a little dictionary that essentially uh Maps the string names of the functions to their actual functions to the actual functions and we're going to use this tool mapping to uh create our agents with the right tools when we run the agent every time so web search write a file read a file there you go so this is what we have okay and now that we have that we can finally write the function that creates the agent okay so def we're going to write create an agent and this is going to take a llm model like chpt or CLA or whatever and some tools okay and what this thing is going to do is create an agent from given the task and the tools given all right and so it's going to create an agent with with a set of tools okay so it's going to say we're going to have tools list here this is going to be the list of the tools for the agent and we're going to look we're going to say for Tool in tools okay because these are the tools that were identified in that specific task and then here we're going to say two is equal to 2. strip. at that I added and this is a step that you know obviously I could have done this maybe when I parsed the the tools before uh and I don't know what I'm doing here and this is probably a useless step that I'm going to remove at some point but for now this is what we have and now we're going to we're going to create the tool in this structure that we need for LMA index so that requires us to do function tool from defaults okay and now here we're going to say FN is equal to two mapping tool okay because the tool is going to be the string after stripping this like web search read file Etc and that's what we're going to do this gives us a natural tool that we can add to our list so we can say tools list. aen tool and now we're good to go folks so now we have all the tools that we need and we have the LM that we're going to use so what does that mean it means that we're going to return the react agent uh given the tools list so we give we give it a tools you give this list of tools we set llm to llm and we verbos to true just so when we demonstrate it works great and Max iterations we're going to set to 10 which is a good number for uh any agent to perform a simple web search and wrer Report task okay so this is awesome uh we have an agent and now we can finally put everything together and see if this automation works so I'm going to write Main and then in the main function this is where we're going to run everything okay so the first thing that we're going to do is we're going to treat this as a CLI tool so the task uh the task string is going to be ss. arv1 Arc V1 so that means that we're um expecting the user to run the python script and then give one string with all the tasks okay at least for the beginning this is what we're doing and now we're going to set up our llm so Lum open AI uh we'll set the model to GPT 40 mini no we actually set the model to uh the model agent right so model agent and then weet the system prompt and the system prompt is kind of I did a big system prompt for this because this is where we control how the agent performs the tasks so I'm just going to copy the system prompt that I wrote It's not like a super optimized system prompt but it should be good enough so what is going on over here we're saying you're a helpful assistant that can search the web and we're saying you're a helpful assistant that can search the web uh read and write files I'm going to remove this stuff because we're not going to be doing this right now at least not right now if you're given the right file tool you should always use it at the end of the task to compile the outputs to compile the outputs of the research or other tasks when given the web search tool you should always use it to search the web for information relevant to the task uh one to three times at the max then write the results okay so it's essentially I'm trying to control a little bit how the model is going to uh perform the task so that we control for issues like you know a never-ending Loop or not writing to file Etc okay and I always want an organized document with the sources that were used etc etc a basic basic input for an agent that does research okay so now we we have to generate our task list so we go generate task list from that task string okay so task string task strn there you go we upad we updated that so that creates our task list and then what we do is we say um uh we're going to capture everything that happens in that Loop so we're going to say responses equal to a list and then we're going to say four task and task list um agent task and then agent tools is equal to extract agent tasks and tools and then agent is equal to create an agent with the llm and the tools the response is this is what we're calling the agent to perform the task and then we append that response and then we return that respon I don't know why we return that response but yeah sure we returned the response at least no that doesn't make sense we're just going to yeah we're just going um um response agent. chat and then maybe we print the response so maybe do we just print the response I feel like that's good enough uh yeah so we can write something like uh processing task X and then we can say output something like print output and then the output is like this and then this and this can go over there and we're not going to do a list anymore I don't know why I was doing that before all right folks so this pretty good so now we have this we can test so now let's test this out so we say python distributed agent Tasker and we're going to give it a test so uh research about best foods to eat to improve cardio and then I'm going to say web search and then the ability to write a file and I ALS o want uh write a mini report on how to study effectively using AI tools so I want the model to search the web and write a file again all right so we have two tasks let's see if the model properly identifies the tasks and does everything that we need running that now folks processing the right task so as you can see the model is now uh make doing the research on the stuff that we asked okay so it generated a very cool list of like a bunch of foods and then everything looks nice and I don't know if it wrote a file I I don't know if it wrote a file fol so let's see if it wrote a file somewhere and if it didn't that's uh we're going to have to do something about that okay beautiful if if it finished the task it it did the research on all the stuff that we asked um and we can test Ah that's one mistake that I made folks I did not create the folder agent artifacts so that was bad so let's just create agent artifacts there you go and now let's test again and let's test again with the simple task I'm going to test again with a simple task like um uh research about llms for uh for research and productivity and write a report as a markdown file then I'm going to say web search web search and then I'm going to say write file I'm going to run that and let's see what happens research perfect so the model is again doing the correct research great looking good looking good okay and then Roo file and now folks if I go list agent artifacts you can see that the file was created and if I open that file we have that little mini report on that topic based on the research that it did with the correct URLs for the the request so that's awesome so folks now that we have this thing and it kind of works well what we're going to do is we're going to set up a telegram bot so that we can send these types of requests from our phone or from telegram all right so I'm going to create a new file here it's going to be called simple telegram agent maybe I'll set I'll say Tasker in the end here nice and now folks uh we're going to import the functionalities that we wrote here into this telegram body agent okay and so let me I'm going to copy uh the the configuration and the setup for the for the telegram bot so that I don't have to write boilerplate again there you go so here folks what we have is we have uh the telegram stuff this is using the latest telegram uh package and this is for building uh in interacting with the telegram API okay uh this is actually we're not going to use l chain this is a node version of this example that I updated and there you go from the script that we created which I have to update now distributed agent Tasker we are importing the generate task list extract agent task and tools and create agent let me just make sure that yeah so let's give the name there you go oh extract agent testing tools let me put the Ry name here there you go oh there you go okay so now I think we're good extract agent testing tools beautiful stuff okay so we have that we have create agent we have Lum index input okay this is all looking good let's just test out make sure that the boiler plate is doesn't have any kind of weird issues so I'm going to just run this simple telegram agent taser it should run perfectly beautiful all right folks so now now what we're going to do is um I need to set up just like we did before the llm stuff so again border play code so I'm just going to copy that from my original file and now we're going to start writing the good stuff and this is the telegram stuff which I've learned how to do recently so definitely not an expert on telegram API but I've learned that you got to start the bot so you write a little star function and then for update it's going to take in this update object and then context is going to take this context types. default type and then none all right beautiful stuff and now here this is the function that sends uh message when the command start is issued so this is kind of like we you how you start the bot from the telegram site okay so then the user is going to be update. effective user and and then await context dobot send message and what is the message you may ask well we got to first set the chat ID to the user. ID actually you set the chat ID not to the user there's no user here is to update. effective chat ID there you go and then you set the text to hi I am the agent asker whatever whatever okay and we set parse mode to markdown V2 okay that's interesting user. mention markdown V2 I'm okay so I'm going to copy the one that I I to I think I took this from their documentation maybe with like a few differences I don't know why I'm doing mention markdown I don't think I need this I feel like I probably this part I probably took from some code or whatever so let's just say let's just say hi I'm a I'm an AI agent ask me any ask me to do anything so hi I'm an AI agent I'm an AI agent ask me uh ask me to do anything okay and there you go folks I think that this is pretty good so now we have our async start function and now we need a function to uh um to handle the message so for that folks the handle message function is going to be essentially the function that reproduces our script from before right the script that executes the AI logic and gives that to the user okay so for that I'm going to I'm going to recopy because this is just going to be redoing the main function folks from what we just did so I'm just going to copy that and then we're going to walk through what that code looks like there you go so folks what is going on over here let me just delete this so folks here this is what we have so the inputs are going to be the update it's going to be you know a reading for the messages from the user the context and what's going to happen is it's going to take in the message and then it's going to generate that task list just like we were doing in the main function of the previous script it's going to Loop over it print so that I can inspect and debug in the terminal and then it's going to extract the agent tasks and tools create an agent execute the agent and then append the response to a list and we do that so that we can then um return that as a response to the user so we're going to say uh we're going to say uh await context boss send message we're going to send the message to the user and I'm going to say look for this task this is the response so we call response um response and we we do response. response because it's the the response it's going to be an object that has this response attribute because that's how uh you when you run an agent with L index that's that's how it works okay beautiful stuff so now folks that we have that and everything is working gorgeously and marvelously we do a little exception where if there's a mistake we just say sorry could not process your request and then log the error now that we have that everything is going wonderfully well what we can do is uh add some error handling which is pretty boiler plate for the telegram API so that's essentially just saying look update caused error and we showed the error and once we've done that we can finally write our main function for this bot so uh for this one we're going to say um uh we're going to say start the bot start the bot and then we're going to say we're gonna now we're g we're going to need to get the token for the um for the telegram bot now if you don't know how to get this token it's pretty easy all you need to do is you need to go to uh your telegram app you go to bot Father which is a bot that you can interact with and create and edit different bots in Telegram and then there you can say new bot and then uh Creator bot which will give you a token so that you can use it to send HTTP API requests to that poot and I'm going to leave a link in description uh kind of indicating a tutorial where you can easily set that up it's just a few steps it's just literally you can take your phone and just figure everything else pretty easily now on the API side on the telegram side we're going to set up our application and for that we don't need application we need application Builder so cursor was doing a mistake there so yeah it's token. token. build not the way they was doing before there you go Application Builder do application Builder like this. token. build perfect and then command Handler start I did not add a help message because this is just a demo and filter taxt handle message everything working beautifully I think that everything here is correct I added the error handling and now I can run the app so now folks finally we are ready to play around with this and check it out so what am I going to do I'm going to run this I'm going to run simple telegram agent Tasker and hopefully oh that did not work because I have to you must pass the token you receive from B father there you go so uh what I'm going to do now is I'm going to pass the telegram token by exporting my token to the to this example so I'm going to just say export telegram bot token and I'm going to delete this token after this video folks so don't go around copying this token and now that I did that I can run that and this should run just fine there you go folks and now this application starting I can go to telegram so I'm going to open up telegram there you go and uh I'm going to clear this chat history here and now I'm going to execute a task so first I start so that as you can see it sends me the correct message so I'm going to say look first thing is um research about optimal studying practices for uh for researchers for optimal studying practices for people using AI I don't know I just some random task and I'm going to say web search and um write file but I want also want the model to uh to write a mini report named uh why pancakes are better than waffles MD uh arguing for pancakes versus waffles in uh and then I'm going to say web search and then I'm going to say write file all right so we have true tasks let's see if the agent can perform them and do something useful so I'm going to send that out so as you can see in the background here the agent is executing everything it's researching stuff it's going through resources Etc it sends me the report about studying that's very nice and now it's running the task about the pancake and it did wrote the report that's awesome and now it's finished and it's actually super fast so that's pretty cool and let's see if it generated what I asked it did look at that why pancakes are better than waffles it has all the research it has all the information and also it it generated this llm for research and productivity that's pretty nice so folks as you can see our agent is working beautifully and and uh yeah I guess that's it folks if you like this video don't forget to like And subscribe and see you next time cheers

Original Description

In this video, we're going to build a simple automation system that integrates OpenAI's capabilities with Telegram for managing tasks, leveraging llama-index for creating useful agents. We start by creating a custom distributed pipeline capable of assigning different tools like web searching, reading, and writing files to custom agents created during runtime. We guide you step-by-step on setting up this amazing iteration from scratch, demonstrating live coding sessions, debugging errors along the way, and offering useful tips. Perfect for tech enthusiasts looking to explore the realms of automation. Join us, explore these cool capabilities, and enhance your productivity toolkit!. 📚 Chapters 00:00 - Introduction to the Automation Project 01:09 - Overview of Automation Process 01:22 - Discussing Basic Tools and Setup 02:05 - Importing Required Libraries 03:23 - Setting Up the OpenAI Client 05:19 - Initializing Models and Folder for Artifacts 06:46 - Setting Up API Parameters 08:05 - Task Extraction Function Implementation 12:54 - Parsing Tasks and Tools from Input 15:22 - Testing the Extraction Code 18:24 - Implementing Additional Agent Tools 22:54 - Performing Web Search with SerpAPI 25:14 - Handling API Key Issues and Demonstrating Web Search 27:26 - Writing Function to Create Agents 29:29 - Main Function and Command-line Execution 36:07 - Integrating with Telegram Bot 38:19 - Writing Telegram Bot Start Function 41:06 - Handling Messages for Telegram Bot 44:25 - Setting Up and Running the Telegram Bot 45:28 - Demonstrating the Telegram Bot in Action 47:12 - Conclusion and Final Remarks 🔗 Links: - Source code: https://github.com/EnkrateiaLucca/distributing-tasks-across-agents.git - Subscribe!: https://www.youtube.com/channel/UCu8WF59Scx9f3H1N_FgZUwQ - Tiktok: https://www.tiktok.com/@enkrateialucca?lang=en - Twitter: https://twitter.com/LucasEnkrateia - LinkedIn: https://www.linkedin.com/in/lucas-soares-969044167/ - AI p
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Automata Learning Lab · Automata Learning Lab · 0 of 60

← Previous Next →
1 A Quick Tutorial on NLP Basics
A Quick Tutorial on NLP Basics
Automata Learning Lab
2 Automating your Digital Morning Routine with Python
Automating your Digital Morning Routine with Python
Automata Learning Lab
3 Exploring Problem Solving with Python and Jupyter Notebook #1
Exploring Problem Solving with Python and Jupyter Notebook #1
Automata Learning Lab
4 Summarize Papers with Python and GPT-3
Summarize Papers with Python and GPT-3
Automata Learning Lab
5 An Experiment Tracking Tutorial with Mlflow and Keras
An Experiment Tracking Tutorial with Mlflow and Keras
Automata Learning Lab
6 Automating Google Forms Submissions with Python
Automating Google Forms Submissions with Python
Automata Learning Lab
7 Productivity Tracking With Python and the Notion API
Productivity Tracking With Python and the Notion API
Automata Learning Lab
8 When your Machine Learning Model Fails Do This ;p
When your Machine Learning Model Fails Do This ;p
Automata Learning Lab
9 Machine Learning Tip#1 Practical Deep Learning Course
Machine Learning Tip#1 Practical Deep Learning Course
Automata Learning Lab
10 Machine Learning Tips: Deep Learning Monitor
Machine Learning Tips: Deep Learning Monitor
Automata Learning Lab
11 Machine Learning Tips#5 MLOPs specialization in Coursera #machinelearning
Machine Learning Tips#5 MLOPs specialization in Coursera #machinelearning
Automata Learning Lab
12 Automatically Changing Desktop Wallpaper with Python and the Nasa Image API
Automatically Changing Desktop Wallpaper with Python and the Nasa Image API
Automata Learning Lab
13 Building an Image Classifier to Filter Out Unused Images From Your Photo Album with Machine Learning
Building an Image Classifier to Filter Out Unused Images From Your Photo Album with Machine Learning
Automata Learning Lab
14 Automating VS Code Snippets with Python
Automating VS Code Snippets with Python
Automata Learning Lab
15 How to Set Up a Machine Learning Environment with Conda and Pip-Tools
How to Set Up a Machine Learning Environment with Conda and Pip-Tools
Automata Learning Lab
16 9 Google Search Tips for Machine Learning
9 Google Search Tips for Machine Learning
Automata Learning Lab
17 Thinking Tools
Thinking Tools
Automata Learning Lab
18 Automating Car Search with Python and Data Science
Automating Car Search with Python and Data Science
Automata Learning Lab
19 Generating Images from Text with Stable Diffusion and Hugging Face
Generating Images from Text with Stable Diffusion and Hugging Face
Automata Learning Lab
20 A Practical Introduction to Data Science using the Spaceship Titanic Dataset from Kaggle
A Practical Introduction to Data Science using the Spaceship Titanic Dataset from Kaggle
Automata Learning Lab
21 Jiu Jitsu App with Python and Streamlit
Jiu Jitsu App with Python and Streamlit
Automata Learning Lab
22 2 Apps for Coding In The Ipad Pro
2 Apps for Coding In The Ipad Pro
Automata Learning Lab
23 From Tensorflow to Pytorch?
From Tensorflow to Pytorch?
Automata Learning Lab
24 Building an Audio Transcription App with OpenAI Whisper and Streamlit
Building an Audio Transcription App with OpenAI Whisper and Streamlit
Automata Learning Lab
25 Productivity Tracking with Python Short Summary
Productivity Tracking with Python Short Summary
Automata Learning Lab
26 Automating Expense Reports with Python
Automating Expense Reports with Python
Automata Learning Lab
27 ChatGPT, Angry Pandas and AI Code
ChatGPT, Angry Pandas and AI Code
Automata Learning Lab
28 7 Strategies To Learn Anything Using ChatGPT
7 Strategies To Learn Anything Using ChatGPT
Automata Learning Lab
29 Building a Thought Summarization App with Whisper and GPT3
Building a Thought Summarization App with Whisper and GPT3
Automata Learning Lab
30 Visualize a Neural Net Learning Polynomial Functions
Visualize a Neural Net Learning Polynomial Functions
Automata Learning Lab
31 Automating Notion with Python
Automating Notion with Python
Automata Learning Lab
32 Pose Tracking for Jiu Jitsu - Update #jiujitsu #machinelearning
Pose Tracking for Jiu Jitsu - Update #jiujitsu #machinelearning
Automata Learning Lab
33 Update to my Pose Tracking for Jiu Jitsu Project #machinelearning #jiujitsu #ai #deeplearning
Update to my Pose Tracking for Jiu Jitsu Project #machinelearning #jiujitsu #ai #deeplearning
Automata Learning Lab
34 ChatGPT API Released by OpenAI
ChatGPT API Released by OpenAI
Automata Learning Lab
35 ChatGPT API Response Format #machinelearning #ai #datascience
ChatGPT API Response Format #machinelearning #ai #datascience
Automata Learning Lab
36 Beyond Stable Diffusion with Composer | Automata Learning Lab Paper Series #1
Beyond Stable Diffusion with Composer | Automata Learning Lab Paper Series #1
Automata Learning Lab
37 Beyond Diffusion Models with Composer #machinelearning #ai
Beyond Diffusion Models with Composer #machinelearning #ai
Automata Learning Lab
38 Machine Learning for Jiu Jitsu
Machine Learning for Jiu Jitsu
Automata Learning Lab
39 Prompt Engineering Basics #machinelearning #gpt4 #chatgpt
Prompt Engineering Basics #machinelearning #gpt4 #chatgpt
Automata Learning Lab
40 Visual ChatGPT: Integrating Images with ChatGPT Paper Series#2
Visual ChatGPT: Integrating Images with ChatGPT Paper Series#2
Automata Learning Lab
41 Visual ChatGPT #machinelearning #ai #artificialintelligence
Visual ChatGPT #machinelearning #ai #artificialintelligence
Automata Learning Lab
42 LERF - Language Embeddings + NERF for Querying 3D Spaces #machinelearning #ai
LERF - Language Embeddings + NERF for Querying 3D Spaces #machinelearning #ai
Automata Learning Lab
43 Summarize Papers with Python and ChatGPT
Summarize Papers with Python and ChatGPT
Automata Learning Lab
44 Large Language Models can use Tools Now! #artificialintelligence #machinelearning #ai
Large Language Models can use Tools Now! #artificialintelligence #machinelearning #ai
Automata Learning Lab
45 Sparks of AGI in GPT4? #machinelearning #ai #agi #artificialintelligence
Sparks of AGI in GPT4? #machinelearning #ai #agi #artificialintelligence
Automata Learning Lab
46 Toolformer: LLMs can use Tools! #chatgpt #llms #gpt4 #gpt3 #artificialintelligence
Toolformer: LLMs can use Tools! #chatgpt #llms #gpt4 #gpt3 #artificialintelligence
Automata Learning Lab
47 Talking to Your Notes with LangChain #artificialintelligence #llms #gpt4 #chatgpt
Talking to Your Notes with LangChain #artificialintelligence #llms #gpt4 #chatgpt
Automata Learning Lab
48 How to Talk to a PDF using LangChain and ChatGPT
How to Talk to a PDF using LangChain and ChatGPT
Automata Learning Lab
49 Query Your Own Notes With LangChain
Query Your Own Notes With LangChain
Automata Learning Lab
50 HuggingGPT #machinelearning #artificialintelligence #huggingface #gpt4 #chatgpt
HuggingGPT #machinelearning #artificialintelligence #huggingface #gpt4 #chatgpt
Automata Learning Lab
51 Do as I Can Not as I Say Paper #artificialintelligence #llms #reinforcementlearning
Do as I Can Not as I Say Paper #artificialintelligence #llms #reinforcementlearning
Automata Learning Lab
52 Automating Anki Flashcards with OpenAI and GPT-4
Automating Anki Flashcards with OpenAI and GPT-4
Automata Learning Lab
53 Building A PDF Summarization App with  Gradio and LangChain
Building A PDF Summarization App with Gradio and LangChain
Automata Learning Lab
54 Auto-GPT #artificialintelligence #gpt4 #llms #autogpt
Auto-GPT #artificialintelligence #gpt4 #llms #autogpt
Automata Learning Lab
55 DocGPT - Chat with Github #artificialintelligence #gpt4 #chatgpt
DocGPT - Chat with Github #artificialintelligence #gpt4 #chatgpt
Automata Learning Lab
56 LLMs for Research and Planning #artificialintelligence #gpt4 #llms
LLMs for Research and Planning #artificialintelligence #gpt4 #llms
Automata Learning Lab
57 How I Use ChatGPT for Interactive Language Learning
How I Use ChatGPT for Interactive Language Learning
Automata Learning Lab
58 Building an Audio Transcription App with Gradio and Whisper
Building an Audio Transcription App with Gradio and Whisper
Automata Learning Lab
59 Summarizing and Querying Multiple Papers with LangChain
Summarizing and Querying Multiple Papers with LangChain
Automata Learning Lab
60 Mojo - The New AI Programming Language?
Mojo - The New AI Programming Language?
Automata Learning Lab

This video teaches how to build a custom LLM researcher agent with Telegram, LlamaIndex, and Python, and demonstrates how to integrate OpenAI's capabilities for task management and leverage llama-index for creating useful agents. The agent can perform tasks such as web search, file writing, and file reading, and can be customized using prompts and system messages.

Key Takeaways
  1. Import necessary libraries
  2. Set up an OpenAI client with API key
  3. Create a model task handler with GPT-4 mini
  4. Use LlamaIndex for search and tool creation
  5. Handle web scraping tasks with requests and BeautifulSoup
  6. Create a dictionary mapping string names of functions to their actual functions
  7. Define a function to create an agent with a given LLM model and tools
  8. Use function tool from defaults to create a tool in the required structure for LlamaIndex
  9. Return a react agent given a tools list
  10. Generate task list from task string
💡 The key insight from this video is that custom LLM researcher agents can be built using Telegram, LlamaIndex, and Python, and can be integrated with OpenAI's capabilities for task management and leverage llama-index for creating useful agents.

Related Reads

📰
OpenAI scored an own goal with HuggingFace attack, showing how open Chinese models are winning
OpenAI's criticism of Hugging Face's Chinese model highlights the vulnerability of closed models and the rising competitiveness of open Chinese models
The Register
📰
citesure 0.5.17: soft-check ORCID iDs before you ship
Learn to soft-check ORCID iDs in bibliographies using citesure 0.5.17 to ensure data accuracy
Dev.to · SybilGambleyyu
📰
Capacitor Showcase – LocalLLM
Explore LocalLLM, a capacitor plugin for offline LLM inference, and learn how to integrate it into your Ionic projects
Dev.to AI
📰
The Stack Under the Model: oMLX, llama.cpp, Hermes, and Why There Are So Many
Learn about the various stacks under AI models like oMLX, llama.cpp, and Hermes, and why they exist, to improve your understanding of AI infrastructure
Dev.to AI

Chapters (21)

Introduction to the Automation Project
1:09 Overview of Automation Process
1:22 Discussing Basic Tools and Setup
2:05 Importing Required Libraries
3:23 Setting Up the OpenAI Client
5:19 Initializing Models and Folder for Artifacts
6:46 Setting Up API Parameters
8:05 Task Extraction Function Implementation
12:54 Parsing Tasks and Tools from Input
15:22 Testing the Extraction Code
18:24 Implementing Additional Agent Tools
22:54 Performing Web Search with SerpAPI
25:14 Handling API Key Issues and Demonstrating Web Search
27:26 Writing Function to Create Agents
29:29 Main Function and Command-line Execution
36:07 Integrating with Telegram Bot
38:19 Writing Telegram Bot Start Function
41:06 Handling Messages for Telegram Bot
44:25 Setting Up and Running the Telegram Bot
45:28 Demonstrating the Telegram Bot in Action
47:12 Conclusion and Final Remarks
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →