Autonomous AI Agents - why YOU should be building them... and HOW.

Key Takeaways

imagine this if you can you wake up in the morning you have your coffee or whatever other stimulants gets you going you turn on your computer and pops up your very own autonomous AI agent you named him goal getter or Gigi for short that was very clever of you Gigi reports to you what it's been doing while you were sleeping and it's been very busy you see you created Gigi to run a business for you the business idea is simple millions of people in the world are looking for simple inexpensive custom automation Solutions some one little script that helps them automate their home some want a morning routine automation where the alarm rings their curtains open the coffee maker turns on Etc some want their emails answered and sorted in very specific ways some want a thing that feeds their cat automatically other people want solutions for their business that are a little bit more advanced requires cameras and tracking Etc but the point is that people are lazy people are busy and you have developed quite a reputation for delivering smart and inexpensive custom automation Solutions one-of-a-kind automation Solutions people go on your website they type in what they want done they don't even have to know exactly what they want they just type in something like my stupid cat keeps running away now you respond within a minute or two with an exact plan of how to fix that problem and multiple choices with different price options depending on how fancy they want to get starting from a simple tracker for your cat to an army of drones following it around everywhere it goes they choose whatever option they want and they provide a credit card for you to Bill once the automation is done you type a script doing the thing that they want maybe you have to order some parts like trackers or video cameras and have it shipped to their house and then you record a quick voice over with visuals on how to set it all up you're kind of like the Ikea of automations the instructions are great they're brain dead simple you made sure the documentation can be understood by anyone regardless of their Tech background if they can use a smartphone they can set up the thing that you've made for them if they run into any issues you provide unlimited support through text and emails or whatever they want once you send everything over to them all the instructions all the details Etc you charge their credit card and you're done but here's the key you didn't do any of that you were sleeping the whole time remember everything was done by GG I talked to the customers and figure out what they wanted it brainstormed some solutions told them where to put their credit card information Etc then I went to work building needed software ordering the parts that were needed and writing out very clear instructions for how to use the thing that it just built it shipped the product either digitally or the physical components that I needed it handled the customer service and made sure that everybody was happy Etc recently you've been teaching it how to do marketing campaigns and you're seeing some great results of that it learned to hook people in with little free animations then it goes online to find whatever information about them that it can and then starts pitching them ideas that it thinks they will respond to it produces content showing how their lives could be improved by these various Solutions custom-made tailor-made to them now your customers are ecstatic and you are basically printing money you don't do any work to run the business zero the only work you do is on improving the agent the autonomous AI agent that is running this whole thing you work on optimizing it on teaching it new skills now you are aware that eventually people will catch on and these agents will be more commonplace and more available to everyone but until then you basically found an unlimited money glitch Gigi so my question for you is when will this scenario likely play out never is it just science fiction is it possible 10 years in the future 50 what do you think what number of years will pass before a handful of people have something like this running now on this channel I've covered what various people in the industry think about this some of the people like the co-founder of deepmind top people at open AI Etc it seems like the consensus that we're somewhere between 2 to 10 years away from something like this me personally I have no clue but like I've covered in multiple videos on this channel the research out of Google deepmind Nvidia open AI Stanford Etc it really seems to show that we're kind of close now I always assume that in order to be a part of something like this you would need to have some sort of background in machine learning currently have to be living in San Francisco and be employed by one of the big Tech firms but that's not necessarily the case in fact there's a clip of Andre carpathy basically saying that he expects of the people who create something like this won't be the people in big Tech but rather it's going to come from a small group of entrepreneurs hackers Etc I don't know about you but I am completely obsessed with this idea because it's becoming very obvious that yeah there will be this window of time when individuals are at the Forefront of this massive AI wave and they have the potential to have massive impact and the example I gave was of a business making money but it really doesn't have to be that education is the other really big thing that I'm seeing as a very interesting potential for this so my goal in this channel is to one learn how to do all this and two teach how to do all this this will be one of many videos on this subject this subject is kind of the whole point of this channel I have no idea where to start talking about this so I'll start with this let's talk about the skill step skill stack the skills you need to start building what collections of skills do you need to be able to go after something like this because often it's not just one skill that you're gonna need in other words it's not that you have Mastery and just one skill really it's being decently good in a number of skills that all of them kind of combined make you really dangerous at something here is as far as I can tell what you need to know take everything with a grain of salt we are very early things are fluid but with that said this is what I would be learning right now if you're already great at the skill good for you if not consider spending an hour a day learning slash applying this skill learning doesn't always feel good sometimes it's uncomfortable sometimes you feel stupid when you're trying to learn something new that's okay sometimes I have to tell myself that I should be working or learning instead of playing battle bit what I do to kind of remind myself to get back into it is this the fact that we are live at the exact time when artificial intelligence is entering the world and that we're in a position to be able to do something about it to interact with it you can say that we're here Against All Odds either that or we're living in a simulation and things are just kind of unfolding according to plan but either way this is kind of a big deal coding so do you need to learn how to code yes what language should You Learn Python more and more people are saying that python is the language of AGI python is powerful people love python which is not something you hear said about other programming languages python has a huge community and is growing and expanding its abilities all the time and that compounds the more people use it for applications the more people will jump on board the bigger it will snowball Etc now if you've tried to learn to code in the past and fail field or you simply decided it was too big of a task I would urge you to try it again now and specifically by using Chad GPT gpt4 to do a lot of the heavy lifting for you Chad GPT can produce code it can explain code it can tell you what to do with that code how to execute it how to use it Etc it can tell you what third-party apps might give you the functionality that you're looking for for example if you want to create an application that texts something to your phone Chad GPT will give you some ideas for how to make that happen it'll give you some third-party services they can use to make of that app if you have a big project in mind you can have Jazzy BT breakdown step by step kind of how to think about approaching it do this first and do this Etc you know how sometimes if you're playing a video game that's getting a little bit too difficult sometimes there's an option to like just click it over into easy mode and then the exact same thing just gets a lot easier a lot a lot more chill that's kind of what just happened to coding some new software like cursor for example are actually making code loading AI first so instead of typing in code you start by prompting change BT and it writes the code for you it's going to basically go through this step-by-step process of figuring out what the project actually needs from a structural standpoint and write all the starter code for you over time you start developing a list of prompts like a library of prompts that you use to prompt Chad gbt to code for you so instead of writing code letter by letter by letter you have a collection of prompts written in the English language that you kind of rotate through to build your applications and we're just getting started where this thing is going to be a year from now two years from now I can only imagine now this doesn't mean that you shouldn't learn how to do all this stuff yourself but what it does mean is this back in the day before you could build useful applications you had to spend a long time doing boring stuff reading memorizing Etc now you can start by building useful stuff on day one you can learn as you go for a lot of people myself included this makes all the difference in the world learning becomes an addiction if you haven't ever coded a single thing in your life please try this now open up Google collab on one side of your screen and then open up chat GPT on the other gpt4 really is the better coder so ideally use that so ask it to produce some python code paste that code into Google collab and hit play you just created your first program now there's still a lot you have to learn but this allows you to quickly build and deploy stuff and as you start trying to do more advanced stuff you have to learn more and more about the language and all the various things that it can do so with python you can do a lot of stuff you can do things like automation to spreadsheets web automation social media interaction home automation image processing expense tracking and even automated stock trading not suggesting you do but you could so start by automating the boring parts of your life keep doing that in one day you're gonna wake up and realize that you're wielding python like a samurai wheelchair next we have prompt engineering when challenging PT just came out I thought that prompt engineering so the ability to write the right prompt to get the llm the AI to do the thing that you want to do I thought that would be kind of like the ability to cast magic spells as time went on I kind of changed my mind I thought that over time people will kind of find the best prompts and we would all have access to them and the AI would be better at sort of understanding what we're trying to do and I thought that maybe prompt engineering wouldn't be all that important now that I kept learning about llms kept using it now I think that prominent engineering is kind of like being able to cast magic spells like an Arcane wizard that reads an incantation from his grimoire prompting the LM in just the right way can produce amazing results there were scientific papers published where the authors test the gpt4 and found the results to be underwhelming the reality was that they made silly mistakes when prompting the llm they didn't understand certain best practices and their results were terrible I replicated those tests in the video that I did it and gpt4 passed with flying colors you need to know things like Chain of Thought reasoning where basically we ask gpt4 to think through something step by step before answering we need to know how to break up complex prompts into smaller chunks and we need to understand certain limitations that these models have for example right now there's no vision and there's no memory by the way when both those things get perfected I feel like those will be to the two last pieces do we need to start building autonomous agents like this that would be like the starters Pistol saying okay go another thing to understand is that in Chad GPT sort of that back and forth conversation that you're having with it that's kind of an illusion Underneath It All you're still just giving it one prompt at a time and what it does is it kind of tries to summarize the previous conversation that you've had with it but that doesn't always work well so if you go back and forth a couple times and forgets what it's set up here or it says something different to what it's set up here that's not it being foolish that's user error and knowing little things like that will make a big difference and how well you're able to use these models people who've spent hundreds of hours using chatgpt often think that it's very intelligent and very powerful as you use it you start developing an intuitive sense of what it can do and what it can't but prompt engineering is a big deal somebody that spent a thousand hours messing around with it is going to have a much better ability to use it than someone who hasn't spent that time since the whole thing is a little bit of a black box developing an intuitive understanding of it is often just a function of how much time you've spent using it for example I spent about probably like 10 hours trying to get gpt4 to solve a particular problem for me it was a reasoning test where you had to arrange certain pieces on a three by three grid following certain rules it could not solve this problem correctly out of the box but I found that if I explained certain terminology and I explained some of the assumptions that I was making well that allowed it to solve it there was a very specific thing that I needed to do that I needed to add that added the capability for it to solve that problem now if I encounter a similar problem in the future I'll have a set of tools that I can approach it with to fix it and make it work now this isn't going to be common knowledge for a long time you know how when sometimes you call customer support to help you if something the first thing they'll do is like well try restarting it and you're like well duh of course I've tried restarting it that's like step number one but guess what that's not step number one for like 90 of the population next building and training models so next on our list of skills that we have to decide whether or not it's going to be useful to us building autonomous AI agents or just AI applications in general that's going to be building and training models our very own models now I'm going to go ahead and say no you don't need this now to be clear I do encourage everyone to learn about this to understand about this and and if you have the time then certainly do it this is a fascinating area and it's like the Bedrock of where all these AI abilities come from I kind of think of the the field of AI kind of like this at the deepest layer we have the mathematical foundation and Theory this is the computer science the math the statistics all the optimization techniques that's like all the stuff that allowed us to even figure out how to build this stuff next level up you have the algorithm development and research so this is designing new algorithms for AI testing and refining those approaches and understanding like the underlying mechanics of machine learning great episode to listen to is Lex Friedman interviewing Ilya satscover they go deep they can be kind of hard to follow in places but that's a lot of the foundational stuff behind this Ilia might be like the number one brand in AI that we have live right now if not the number one certainly in the top five hearing him talk about some of the stuff is mind-blowing but also difficult to understand if you're not versed in machine learning next you have the thing that we're talking about is building and training AI models nowadays this is something that you can do at home either with an Nvidia card or by using cloud services Android no has a great class about this it's called machine learning specialization I think something like 250 000 people wouldn't throw it on Coursera it's a great class I personally enjoyed it very much I also send text on YouTube talks about some of the stuff he trained neural networks from scratch to play games like Starcraft GTA Etc interesting fun he does a good job explaining this stuff his background isn't in machine learning or coding either I think he's a lawyer that transitioned to doing all the AI stuff which gives me hope for the world and then on top of that layer and again this isn't anything official I'm just kind of this is how I'm kind of seeing the field but the next layer is the application layer this is building specific application using these trained models this is integrating AI into existing systems and deploying models into production and of course building autonomous AI agents so basically you have the theory the research the model building and training and then the actual application so what we're talking about is the actual application that's what we're doing that's the thing that we need but you should learn all the other stuff as time and interest and dopamine allow but training models might end up similar to hosting a website now you could set up your own server install Linux setup Apache and all that other stuff that you need to do so that you can host your very own WordPress blog the reality is most people just pay hosting company to do all that stuff for them same thing will likely happen with these AI models Amazon is basically already doing that Amazon Bedrock is a fully managed service that makes Foundation models FMS from Amazon and leading AI startups available through an API so you can choose from various FMS foundational models to find the model that's best suited for your use case the big caveat here that I want to make is that people that are building the open source models that are available to everyone and the people that are using it and improving it Etc that's a huge counterbalance to potentially all these big Tech firms just kind of accumulating all the AI power and all the cloud resources and all of the models Etc and all the money that's flowing into it and then being in control of who can use it and how they can use it Etc that would be potentially a scary world to live in so we do need open source we do need those people to to build and develop and to move open source forward Ilya seskaver gave a talk somewhere I'm blanking on where this was it was when ham and Sam Altman did that whole world tour where they talked to everybody but he expects that open source will always keep Pace with the most advanced models but it's always going to be you know a few years behind several years behind so it's never going to fall too far behind but it's never going to be at the Forefront now to use open source you will likely have to know how to build and train these models you have to know a lot more about that so I would say yes learn as much about this stuff as possible but if you can't you're not locked out of the game you still have everything that you need openai API so open AI API can be used with multiple languages python is one of them so if you know python you're good you just need to learn all the things that openai API can do and use them if you're learning python already this won't be a huge addition for those of you who don't know what API is that's kind of like eating at a restaurant if you come into a restaurant you don't go in the kitchen and start cooking your own food you're not allowed back there and you probably don't know what you're doing instead you point to the picture of the Burger on the menu and you say I want that and then the waiter brings you the food the ready-made product that's what an API is and our example of openai API opening eye is the kitchen they have tons of useful stuff that you might want to use like for example all their models as well as whisper their audio to text transcription service so for example if you have an audio file you tell the waiter slash API I want this audiophile transcribed by Whisper the way it goes and he brings you the transcribed file from you know the kitchen open Ai and that's it and tons of places have apis Gmail has one if you need to answer emails for example Minecraft has an API apparently called Mind flare stripe has an API for credit card processing so let's say you have access to three apis open AI Gmail and stripe now you can automate customer service emails and providing them with refunds canceling subscriptions Etc right now I'm building a little AI app and I'll do a full video on how I did that and what you need to know to do that yourself but here's the code for it so we're going to use Python to check a podcast feed we download the latest podcast episode and then we use the openingi API to transcribe the audio file then we use GPT 3.5 to summarize the podcast and we can also pull out other details for example if there's a podcast guest we can find his name what organization he works for we can access Wikipedia to try to get some more information on them and furthermore if we wanted to create a blog post or a newsletter that summarizes this content we can have it do that and then automatically send that newsletter or post it to our WordPress blog so as you can see this is a pretty powerful app you can use it for personal use like if you wanted to catch up on the latest podcast episodes but just don't have the time or you can use something like this to build an entire publishing business so how difficult would building something like this be for a novice without changing PT I would say this would be moderately difficult you would have to think about how to plan the whole process out you have to write the code for each step of the process you would have to find ways to check to make sure the code is working that nothing is screwed up Etc then you would have to put it all together and run it with Chad GPT you can have it do a lot of the heavy lifting and get this done rather quickly I don't think we fully process this as a society back in the medieval times Only The Chosen Few could read and write now most people are expected to read and write just to participate in the economy what percent of the world's population do you think in code take a guess I tried to find the answer I think it's like 0.5 percent half of a percent here's Greg Brockman president and co-founder of openai here's what he said programming is so fun because you get to go from lack of understanding to Mastery time and time again at a rate determined by your speed of iteratively writing and running code and with real world impact when you succeed iteratively is a hardware word to say but Chad GPT allows you to rapidly write code iterate and then try again it's faster it's easier there's less barriers to entry and it also is much more powerful because it allows us to tap into the other abilities that AI has so the real world impact is Multiplied as well so I think having this skill stack that we just discussed will be crucial in the next few years now this is just a core skill set I think there's a number of other skill sets that are going to be more and more important and we'll talk about it at a later time one of them I don't even think it has a name yet I began thinking of it as Conjuring but I think that the impact of this technology will have it's going to be world changing in both good ways and bad I'm very optimistic about AI in the long run I think it's going to help us solve a lot of problems and improve lives for everyone after the world adapts and the transition will happen after that life will be good but short term I don't know there are ways things can go wrong in the short term very simply the value of human labor will keep falling why would you pay 50 000 a year to a human when AI can do a better job for 300 bucks law of supply and demand states that the price that we are willing to pay for human labor is going to continue falling it will Trend towards zero how are we gonna handle that now I'm sure we'll be fine our government leaders are smart tech savvy forward-looking people who are selected for their abilities to get stuff done but just in case if they are not able to rapidly solve the problems raised by these never before seen exponentially growing Technologies just in case learning these skills might be a good idea my name is Wes Roth thank you for watching