Andrej Karpathy GPT - Advice for building AI agents

Key Takeaways

this is the first time I've seen you in person hi everyone the microphone is working so I was recruited uh to say some words of inspiration on the topic of AI agents um so I actually wanted to begin with a story I think like AI agents are near and dear to my heart to some extent because if you have a story from very early open AI That's when it was like maybe a dozen people and this was 2016 or so and design guys of the day actually where are all agents and so everyone's really interested in building agents but very much at the time it was in the context of the beings primarily so it was the height of excitement around Atari and all the game playing and so on and so my project at open AI was trying to focus the roh but it's not on games and Montezuma's Revenge and so on but on you know using computers using keyboard and mouse and so I wanted to make it useful I wanted to make them do lots of tasks and this project was called a world of bits and I I worked on it together with tinsi and Jim fan there's someone here I think I saw them and so the three of us ended up publishing a paper it's not a super amazing paper because basically at the time all you have as a hammer was convertible and so we have these very simple web pages that we're trying to like order order or apply for order some something to eat in very simple book pages and we were mashing buttons and mouse clicks and so on and we were trying to like stumble our way into the higher Awards and obviously didn't work so the technology was just not ready and that was not the right thing to work on at the time and so it turns out that the right thing to do at the time was actually to forget about AI agents altogether and start building language models and then language models now we're back here five years later I got distracted with self-driving for a bit there again and so the way you would approach these problems today is completely different in fact all of you are working on AI agents but you're not using any reinforcement learning probably and so that's so crazy and I don't think we would have anticipated at the time it's just the way this played out it's very it's very interesting so I want to spend a bit of time on like okay what is causing all this High I think obviously all the reason that all of you are interested in this topic is that I think that very obvious to a lot of people that AGI will take a form factor of some kind of an AI agent and it's not just going to be a single agent thing there's going to be many agents and they're going to be in organizations or civilizations of digital entities and I think it's just extremely inspiring to sort of like thank you it's kind of crazy I also want to pour some water elements too though I think there is a large class of problems that I think are very easy to imagine very easy to build demos for are actually extremely hard to make a product out of and I think many things fall into this category it's all self-driving is an example of it actually uh self-driving is very easy to imagine very easy to build a demo of a car driving around the block but making it into a product takes a decade and the same is true I think for example VR we're used to imagine little demos but getting a tour again takes a decade and I think agents are kind of like that as well very easy to imagine excited to get excited about build demos off but I think if you're in it you should be in it for a decade and actually making work one other thing I want to say is that I think it's kind of interesting to go back to Neuroscience now and take inspiration from it again in some ways I think this was true in the early days of deep learning but it's now true also again for AI agents so it's been kind of interesting for me to think about sort of all the equivalents so in particular I think a lot of us when we're it's clear that language model is a part of a solution but you know how do you build an entire digital entity and that has all of the cognitive tools that humans have so obviously we all think we need some kind of a potential system to to actually like plan ahead and think through and reflect on what we're doing and there's a lot more that you can take inspiration from so for example the hippocampus is important what is the equivalent of the hippocampus for AI agents well well it's uh somehow like recording uh memory traces and maybe indexing them using embeddings and retrieving from them uh something like that so maybe that's kind of like the retrieval kind of thing in the brain we I think understand how to build visual auditory cortex roughly but there's many subtle things that are like what are the equivalents of them in AI agents what is the hazel game we are doing what does it look like in an AI gym like what does the thousands doing sort of integrates all this information together it's kind of like potentially the seat of Consciousness and it's got this very interesting neural circuit where if you think of your brain is sort of like multiple entities fighting for the microphone as to like what you're going to do the thalamus is probably where that happens it's kind of interesting so actually a book that I like the girl says it's called bringing Behavior by David Eagleman and I found this very interesting and inspirational so I leave it here if you'd like to go through it and pace through it and I just think there's some interesting inspiration to again draw from neural science just like we did early on in with respect to how we design an individual neuron we can do that again today me finally I wanted to end with some words of inspiration but what's interesting and not obvious is that you guys building AI agents are actually at the Forefront of capability of the aiation today and all the big laughs like lln Labs opening Ai and so on I suspect are not at the edge of indicator but you are at the program so opening up for example is very good at training massive Transformer language models so as an example one way to put it is if the paper comes out that proposes some different way of training a Transformer the internal socket open AI is something along the rise of oh yeah someone tried that two and a half years ago and here's what happened and here's why it didn't work and it's very well understood and very very well mapped out but when a new agent paper comes out we're all interested in mooncals and we're like oh that's really cool that's novel and that's because you know the team didn't have like five years to spend on it and it's competing now with all of you and the entrepreneurs and actors the song it's really hard to do so yeah I think it's really inspiring that you are at the edge of capability and on something that is obviously very important in transformation and so with those words I think I'm eager to see what you guys build [Applause]