Coding is Changing Fast... AI Agents Explained

Key Takeaways

what exactly are AI agents and why is everybody talking about them well in this video I'm going to break down everything you need to know about AI agents including what they are how they work and why they're so important I'll also discuss how they differ from standard llms and give you an example at the end of the video of how you can run and use your own AI agents locally oh and by the way this video is made possible from our sponsor envidia whose goal is to help educate people about AI agents and how you can experiment with them using Nvidia RTX AIP PCS easily using anything llm on your own computer we'll talk more about that later but let's get into it now ai agents are really just systems that are powered by artificial intelligence that can autonomously complete tasks make decisions and even adapt to new information think of them like proactive tools that don't just answer questions but can actually take steps and actions to achieve a goal now let's look at a quick example of a travel planning agent rather than just suggest in flights and hotels or places to visit it can actually book that accommodation for us it can book the flights it can reschedule them if it gets delayed it can book a hotel it can make reservations for dinner it has access to various tools and can actually take actions to achieve the result rather than simply recommending things or just giving us a block of text now with that in mind there are three Key properties of AI agents the first is that they're autonomous this means they can act independently within a given scope number two is is that they're goal oriented they're focused on a specific outcome like solving problems or completing specific tasks often times you'll see agents specializing in one certain area or one certain task next they are environment aware that means that they respond to data or events from the world around them whether online or physical they can also store memory often times so they know what you asked them to do previously and any information or context you've provided so that's a quick brief on what AI agents are but now let's talk about how they work AI agents combine multiple Technologies in order to operate but here are the four main ones you need to be aware of the first is the core AI model or the llm now this is the brain of an agent and it's often a large language model like GPT or llama 3 it handles understanding and generating language reasoning and the problem solving involved in the agentic capabilities number two is tool usage these could be apis databases various software may be custom ridden to achieve some specific specific task and they go beyond simply generating tasks they can actually call out to different tools you provide to them for example your agent might have access to a calendar API where it can create new bookings or different appointments now the third component is memory many agents have access to short-term memory and some even have access to long-term memory which allow them to track what they've done in the progress of a specific task and also to adjust and adapt their behavior now the last and likely most important part of an agentic system is the feedback loop now this allows the agent to continuously Monitor and analyze Its Behavior and if something didn't work it could actually go ahead and try again A lot of times when you're using AI agents they'll try multiple steps they'll see that they didn't work and then they'll move on to something else now all of these features are used to create a Genentech AI but they do need capable hardware and machines like an Nvidia RTX AI PC in order to process at the fastest speeds now all of these massive advancements that we've seen in AI recently are really due to the innovation and technology and Hardware from companies like Nvidia and that's personally why I choose to use a 4090 in my machine so I can mess around and train AI models locally now these four components are coordinated by something known as the decision-making framework this could be something like reinforcement learning or rule-based logic but the point is this framework uses these different components and guides the agent on how to achieve a specific task so here's a quick overview of what the decision-making framework looks like because it's pretty important to understand and how the agent works the decision-making framework starts by defining a goal or accepting one provided by the user it then breaks this goal into smaller actionable tasks just like we would as a programmer trying to solve a problem now the llm that's used with the agent typically excels at reasoning and structuring information this means they can interpret vague instructions like plan my day and they can decompose that into subtasks such as check the calendar review emails and schedule a workout now the next step is tool and action selection the framework will decide what tool to use and what actions to take for each subtask for example an AI agent may choose between searching the web quering a database or executing a predefined function now the llms as a part of this agent act as the central problem solvers they generate outputs that guide the decision-making process for example the agent might ask the llm what tool should I use to check the weather the llm then responds use the weather API the agent would then use this tool to fetch real world data analyze the output of that tool and see if it succeeded in that part of the task now the third step is the feedback loop after executing an action the agent evaluates whether the action brought it closer to the goal if not it adjusts its approach dynamically now the llm can analyze the results of an action for example if the agent sends an email but gets an unexpected response then the llm will interpret that reply and suggest the next best course of action this creates a loop of act evaluate and adjust enabling the agent to refine its actions and try different approaches if the first one didn't work now step four is memory and context management to make informed decisions the agent needs to track its progress and context what it has done what remains and any external changes now the llms can retrieve or update memory like keeping track of conversations the tasks that have been completed or intermediate results for example if an agent is writing a report the llm will remember the sections already written and ensure consistency in the tone and facts across the document now step five is prioritization and conflict resolution the framework will prioritize tasks and resolve conflicts when multiple paths or goals compete for example if an agent must book a flight but also ensure it fits a budget it weighs the options to balance both constraints now the llms can evaluate in rank options using reasoning for instance the agent might queer the llm should I prioritize the cheapest flight or the earliest departure the llm will then suggest a balanced Choice based on the user preferences or the preset criteria and step six here is integrating the reinforcement learning many decision-making Frameworks will incorporate reinforcement learning to improve over time the agent can get some kind of reward signal when it successfully completes tasks or achieves goals efficiently now while reinforcement learning focuses on optimizing decisions the llms can add some flexibility by handling tasks where predefined rules don't apply for example an llm can help the agent explore Creative Solutions when a standard response will fail you can make all kinds of advanced AI agents you can do something as simple as an email bot or you type in something and it sends an email or something as advanced as a customer service agent that can actually delegate to other AI agents and have a whole team of AI agents working on a complex problem this is a really interesting field there's a lot to learn and what I want to do now is actually show you an example of some real world AI agents and how you can use them yourself for free locally on your computer now fortunately for us Nvidia is sharing about a fantastic application called anything llm now anything llm allows you to run the best llm models locally on your own computer using an Nvidia GeForce RTX GPU everything from the model the documents the chat history is all stored locally on your own computer with no account needed so you have complete privacy and you can run this all locally and recently anything llm actually released an AI agent Community Hub where users can build deploy test and play around with VAR I different AI agentic skills and because anything llm is built on ll. CPP that means it's fully optimized to run faster and better on GeForce RTX and Nvidia RTX gpus now personally I've got a 4090 in my computer but even if you don't have the highest end GPU this is totally fine and you can still take advantage of this application even with lower-end Hardware so I encourage you to check it out and with that said let's hop over to the computer and look at a quick demo of this app so I'm here on the computer and I'm going to give you a quick demo of how anything llm works and how you can use it to invoke different agents and mess around with them privately now the important thing here is that all of this is running locally yes you can connect to other providers like open aai you can use API keys if you don't have a GPU or you don't have high performance on your machine but if you have a good enough computer you can run all of this yourself completely privately completely locally which I think is quite cool anyways you can download this for Mac windows or Linux I've already got it installed and as you see local by default no account needed and compatible with pretty much all of the operating systems so what I'm going to do is go into my any llm app I've already downloaded it it does take a second just because it's pretty large it needs to install a lot of files and the first thing we're going to do here is make a new workspace so I'm just going to call this demo and then save and then inside of here you can make different threads kind of like you would in something like chat GPT anyways what we need to do in order for this to work is we need to enable our llm or configure our l so if you go to settings sorry I should show that it's right here this little kind of gear wrench icon then you can go to all providers llm and then you can choose your llm provider now we've got a bunch of different options here right so you can use things like open AI or an Tropic if you want or you can just use anything llm which is what I'm going to use where it's very easy to get set up with models like llama 3 the Microsoft models uh Google models IBM research Etc so I'm just going to use llama 3.2 the way you do this is simply click on it and click activate and then once this is active you can go to save changes okay so press save changes and then this is automatically going to download the model for you if you don't already have it and set it up and configure it for you now I already have it installed so it just happened instantly but for you it will download it and you'll see it in the top right hand corner kind of the progress of that download now you can configure things like your vector database the embed or a text splitter and chunking and that's because you can actually upload files to this and get it to reason based on those files this also has some agent functionality built in so if we go here to agent skills for example you can see that we can have retrieval augmented generation we can view and summarized documents scrape websites there's a bunch of other stuff and you can configure some of these as well so for now let's just mess around with the basic functionality then I'm going to show you how we can use something called the community Hub to bring in some more advanced agents and use them locally on our computer so I'm going to go back here I'm going to go into demo and let's just start by doing something like hello world and make sure that this is able to reply to us okay so it seems like it's working now we can ask it something like can you generate a new coding challenge for me and let's see what we get and there you go we've got a quick coding challenge now where this gets cool is when you start interacting with agents and files so for example I can attach a file here so let me just find one that's interesting and that it can reason based off okay so I've got this PDF called how to make money from coding by the way way I'll give this to you completely for free if you sign up for my newsletter free newsletter if you sign up I give you this as a lead man that you can do that from the link below we send coding challenges and all kinds of stuff there it's pretty cool pretty valuable don't worry no spam just pure value anyways point is I have this PDF and now I've embedded this file so I just hit enter and then I kind of added this file to my system and what I can do is now use this at symbol so I can say at agent and I can say rag search my how to make money from coding file and summarize it for me so if you want to use the agent capabilities you just tag the at agent and now it's going to swap over to the agent chat and it's going to start using whatever tool is necessary so you can see hey it says it's attempting to call the rag memory tool found four additional pieces of content to help answer this question and it does this by looking through my PDF and then pulling out all of the reasonable information or all of the relevant information sorry so you can see it kind of gives me a summary of the lead magnet here you know mastering coding can lead to financial Independence coding is a high demand skill freelance tutoring all of the different things I've got in there a strong Foundation is essential freelancing tutoring offering tutorials join a community a bunch of stuff that I've gotten there right there's a ton of information but I was able to actually use that PDF running again completely locally so I didn't have to share this on the cloud and answer the question for me okay sweet so that is interesting and I want to show you that if you press on this little button here in the workspace you can view all of the different files that you've uploaded so you can see I uploaded some files previously that are in here you can just drag and drop them or you can even fetch them from a link and then you can kind of Select if you want to use them in the specific workspace or not so in this case we have the how to make money from coding one in this workspace so now we can reason based on that use rag all of that kind of stuff if we go to data connectors you can see that we can connect to GitHub repos YouTube transcripts we've got all kinds of settings here you guys can mess around it's not going to be an in-depth tutorial but what I want to show you now is a new thing that anything llm has which is their Community Hub so I'm just going to open up the web browser here and I'm going to go to hub. lm.com now you can actually access this page directly from this uh application if you go to open settings you go to community Hub and then go to explore trending you're going to see pretty much the exact same page now these are a bunch of open-source and community-driven uh kind of agent skills as well as system prompts and slash commands that you can bring into anything llm and run locally so for example you can generate calendar events you can open the slack app either just getting started here so there's not too many but you can save a file to a location you can get the current date and time which typically llms are not able to do because they don't have access to realtime information you can generate or get news from the BBC news feed all kinds of stuff right so if you want to bring in one of these skills let's say we want to maybe get the date and time we can simply press on this here and go import to anything llm we can copy the import ID go back to anything llm and then go to import item from here we can just paste the ID continue with the import wait a second we're just going to have to confirm that we do actually want to bring this in and then if we want to use this tool we just need to invoke the agent and ask it for you know what is the current date and time so now that we've got that we're just going to go to agent skills you just got to turn these on by default they're going to be off so date time I'm just going to toggle that on because we've just added that and now if we go back to anything llm we go back to our thread we can say what is the current date and time we've got to make sure we ask the agent so we're going to go at agent what is the current date and time and let's see if it can give us that response and utilize that tool okay and there we go it's given me the current date and time by using this tool again I know that doesn't sound crazy impressive they're really just getting started with the community Hub but I wanted to make you aware of it because by the time you watch this video there might be a ton of other agent skills that you can pull into this tool obviously a lot of interesting stuff you can do I've person just been using this kind of like a local chat GPT because it's way faster it's completely local and it means that I don't need to upload everything to the internet I don't need to share sensitive documents if I want to for example analyze some of my finances or find quick information from receipts I can drag those into here and just analyze those right and I can do that safely from my own computer using my own Hardware so agents are super cool definitely kind of the future of AI and a new trend that's popping up quite quickly you guys have seen on my channel I've made a bunch of videos I've built AI agents and in this one I'm just talking to you about exactly how they work and how you can mess around with them on your own using something like anything llm and by the way if you do want to learn more about how Nvidia GeForce RTX PCS are powering Ai and the massive advancements we've seen recently then make sure you check out this page from the link in the description [Music]