Your Brain Doesn't Command Your Body. It Predicts It. [Max Bennett]

Key Takeaways

What's really interesting about about this book, Max, is um you know, obviously I' I've read loads and loads of books in in the space and there's, you know, people like Hinton and Hawkins and Damasio and Friston and uh I mean, God, you know, even like Sutton and what's interesting is it's a bit like the blind men and the elephant. So, they've all got a completely different story to tell. And I think the magic that you have pulled off with this book is somehow you've woven it together into a coherent story. like what what do you think about that? >> Well, first uh I'm very appreciative of of the kind words. Yeah, I think I came from a very unique perspective just because I was a complete outsider. Um and I think uh you know and I didn't come to it with the objective of writing an academic book at all. I came to it from the objective I was just learning on my own. Um, and I just started building this corpus of notes, um, cuz I was so independently curious. And I kind of stumbled on this idea really for myself of how do I make sense of all of these disparate opinions um, and really this complete lack of information about how the brain actually works. And I had my own set of I think biases coming from sort of the technology entrepreneurial world where uh, we tend to think about things as ordered modifications. uh when you think about product strategies or how to roll things out, we like to think about things as what's step one then what's step two, what's step three. Um so I think I did have sort of a cognitive bias to when presented with an incredible amount of complexity to try and make sense of it in a similar type of way. Um um but yeah, I think as an outsider I felt very free to sort of explore and cross the boundaries between fields. I mean I I look at the book as a merging of three fields. One is comparative psychology. So trying to understand what are the different intellectual capacities of uh different species. Um evolutionary neuroscience. So what do we know about the past brains of humans and the ordered set of modifications of how brains came to be? And then AI um which is um how do we ground sort of the high flutin conceptual discussions about how the brain works in what what works in practice I think is a really important grounding principle because it helps really hold us accountable towards the principles that we think work if we can't implement them in AI systems it makes it should make us question if we actually have the ideas right. Um so so yeah I think being an outsider in some ways you know comes with disadvantages uh um but there are some advantages also which is you're kind of free to borrow from a variety of different fields and think freshly about things. >> Yes. I mean if you can point to any particular um ideas that you found really really difficult to reconcile what what would those be? >> One thing that's really challenging is um if we were to actually lay out uh what's the data richness of comparative psychology studies across species. If you put that on a whiteboard and looked at it, you would realize we have so little data um on what intellectual capacities different animals in fact have. Um for example, uh the lamprey fish, which is sort of the canonical uh animal that's used as a model organism for the first vertebrates. Um because it's the of of all vertebrates alive today, it's one of our most distant vertebrae cousins. To my knowledge, there are absolutely no studies examining the map-based navigation of the lamp prey fish. So, so we have no idea if it's in fact capable of recognizing things in 3D space. Now, when we look at other vertebrates like tio fish, it seems like they're eminently capable of doing that. We look at lizards, they're eminently capable of doing that. So, we we sort of infer um it seems likely that the first ver vertebrates were able to do this. We know the brain structures from which it emerges and and reptiles um and tilios fish are present in the lamprey. So, we we sort of back into an inference that okay, probably the lamprey fish can do that. Um but this is all sort of uh in some sense guessing and trying to put the pieces together from very little information. So I think that's one challenging uh aspect to reconcile. Um the other one uh that's really hard uh is in in neuroscience there's a lot of really interesting ideas um about how the brain might work that have not really been tested in the wild um from the an AI perspective and then there's a lot of uh AI systems that work really well that have diverged substantially from at least the evidence what the evidence suggests is how brains work. So how do you bridge the gap between these two things? I think is a really fascinating uh space to operate in which is like what can we learn about the brain if anything from the success of transformers as an example. Um what can we learn if anything from the success of generative models in general? Um what can we learn from the success of and failures of modern reinforcement learning? I mean in some ways reinforceable learning has been a success in other ways it's really uh fallen short of what a lot of people hoped it would be. Um so I think the the gap between neuroscience and AI is still a challenging one. uh to bridge in a lot of ways. Um for example, Carl Fson has all these incredible ideas in active inference. Um in a hundred years, will we look back on this and be like Carl Fen was on to something. Um if you look at the AI systems today, there's very little, you know, usage of active inference principles working in practice. Um so that could mean that the ideas don't have legs or it could mean that there's a breakthrough behind the corner where we're actually missing some of the key principles that he's devising. And these are, you know, questions we don't have the answers to. I think there might possibly be some breakthroughs around the corner. I don't know if you know, but I'm I'm Carl Friston's personal publicist. Um I I do all of his stuff. I I I probably interviewed him more than anyone else, but I love my friend. He's an amazing guy. Yeah, he's an amazing man. >> Honestly, he he is the man. He is the man. And um >> so kind. >> I know. For me as well, he has so much time to explain things. But yeah, um you could cynically argue that, you know, the the the effective active inference agent is just a reinforcement learning agent of a particular variety. I think it's equivalent to a um like an inverse reinforcement learning maximum entropy agent or something. But there's so much more than that. There's so much richness and explanatory power of of of kind of modeling this thing as as a generative model that can generate um policies and plans of actions and and so on. And also we want to have agents that you know we understand what they're doing and and like with steerability and being able to do the simulations that you talk to so eloquently in in your in your paper but just I love for instance we'll do we'll do him properly later but um to come back to what you were saying though so you were saying oh you know there might be a parallel between transformers and AI models in in your book actually on this page you you sort of like g you you kind of analogize um modelbased reinforcement learning and the neoortex And of course I interviewed Hawkins back in the day and the main criticism of his book is that uh you know the triune brain type argument. So he's kind of giving giving the explanation that uh the brain developed a bit like geological strata with one layer and then another layer rather than kind of co-evolving together. And it's so hard not to think like that because you give so many beautiful examples in your book, not only just morphologically, but but in terms of capability, how the neoortex, I mean, you said like with stroke victims, for example, it's not like the brain recovers those dead cells. It learns to kind of repurpose those functions in other parts of the brain. So, it seems like Mount Castle was correct in the neoortex is this just magic general purpose learning system. I mean, what do you think? So I think um well there's two different uh there's two different ways to look at uh the neoortex enabling things like mental simulation and model free or model based reinforcement learning. One is that that function and algorithm is being implemented in the neoortex. But another which is a slightly less strong claim which is the one I would make is the addition of the neoortex enables the overall system to engage in this process. So it is not saying that uh the entire process is implemented in the neoortex. I think it seems very clear that the phalamus and basil ganglia are essential aspects of enabling the pausing the mental simulation the modeling of one's own intentions the evaluation of the results etc. But it is possibly to say which is what I'm argue in the book is in the absence of the neoortex that process does not happen. Um, and so I think where my ideas would synergize with what Hawkins is saying is the neoortex builds a very rich model of the world and a model of sufficient richness that you can explore it in the absence of s sensory input. Um, and that's a really essential aspect of model-based reinforcement learning because if I have a a model of the world that has sufficient richness that I can mentally simulate actions um that I'm not actually taking and it at least somewhat accurately predicts the real consequences of those actions. And that model is really useful um uh because I can now imagine outcomes before having them. I can flexibly adjust to new situations. Um and of course there's so many really deep interesting questions that are yet to be answered about that. For example, um just because you can render a simulation of the world doesn't answer the question, okay, what do you simulate? I mean, this is one of the one of the hardest problems of model based reinforcement learning is fine, you can have a model of the world, but how do you prune the search space of which aspects of that model you explore before evaluating outcomes? Um, and so that's another really hard challenge that I think there's a lot of good evidence is actually a partnership between the neoortex and the and the basil ganglia, which is a much older structure. Um, so yeah, I'm not really of the, you know, I think the triune brain has been, you know, amongst evolutionary neuroscientists largely discredited. Um, and I think in part somewhat unfairly if you actually read McClean's writings, he was he actually is very open about the fact that, you know, this is an approximation. It's not exactly accurate. Like he he couches his claims much more than the popular culture than just converted into a dogma. Um but I think the the popular interpretation of the triune brain is not accurate which is it is clearly not the case uh that the brain evolves in three key layers. It's not the case that a reptile brain doesn't have anything limbic like uh if you the a reptile brain absolutely has a cortex that does a lot of what our limbic structures do etc etc. Um so yeah that those would be my thoughts on that. >> Yeah it it's fascinating because we as humans need to have models to explain and understand the thing itself just like active inference for example. I mean, I I'll get to planning and agency and goals in a little while, but a lot of these things are instrumental fictions. I'm not saying [clears throat] that our brains don't plan, but you know, like the way that the abstract mathematical way that we understand planning, it's probably not how the brain works. It's it's much more complicated than that. But why don't we just rewind to the beginning? So, you know, we're going to be talking about this chapter on on, you know, simulation if you like. And you kind of lead by saying that what the neoortex does is it does learning by imagining. And uh Hawkins spoke about this as well. He said we've got the matrix inside our brains, right? We're always just doing all of these simulations of of future things. And uh we're using that to kind of help us understand the world. And you you give this really interesting example of um some of the features of the brain that kind of lead you to believe that we are basically living in a simulation. And it's almost like rather than perceiving things, we're testing if our simulation is correct, but that means that we can only simulate one thing at a time. So we can't we can't see two things. We can only see one thing. So can can you talk through that? >> Sure. So um one of the um first sort of introspections and explorations into how perception works in the human mind happened in the late 19th century with all of these explorations of visual illusions that you see in pretty much every neuroscience textbook or book that you open. Um so listeners will be familiar with you've probably seen examples of triangles where you actually perceive a triangle in a picture when there is in fact no triangle. Um or uh yes that isn't that yeah you can find that picture. [laughter] Uh >> yeah sorry I hope I'm not distracting you. >> No no no. So that's um so that's a you know a standard uh finding that was observed in the 19th century which is this idea that clearly the brain observes the presence of things even though they're not actually there. So we perceive a triangle there, we perceive a sphere. We perceive sort of a bar. we perceive the word editor when in fact if you actually examine that the word E is not there. Um there's evidence that suggests E is there by virtue of showing the shadows but we did not actually write the letter E there but the brain regularly observes that. Um so that finding led uh this this uh scientist Herman von Helmholtz to sort of come up with this concept that what the brain what you actually consciously perceive is not your sensory stimuli. You are not receiving sensory input and experiencing the sensory input. What's happening is your brain is making an inference as to what is true in the world, what's actually there and then the evidence is just uh the sensory input is giving evidence to your brain as to what's there. And so you start from this prior um and then that prior maintains itself until you get sufficient evidence to the contrary and then you change your mind. And so it's not hard to imagine why this would be extremely useful in uh any sort of environment um that an animal might evolve in. You know, suppose you have uh a mouse running across a tree branch at night. Um first I see the tree branch and the moonlight. So I build a mental model of the tree branch. As I move forward, I lose moonlight. I no longer see the tree branch. As long as as I'm stepping forward, the evidence is consistent with my prior of the tree branch. it makes way more sense for me to maintain the mental model of that tree branch as opposed to all of a sudden the tree branch disappears because I no longer see the sensory stimuli of it. So because sensory stimuli is very noisy, it makes a lot of sense that we in integrate it over time, build a prior and then until something uh gives us evidence to the contrary, we maintain our prior about the world. So so that was sort of the first idea that there's some form of inference. there's some ch there's some difference between sensory input and some model of the world that we infer and then thus perceive. What's interesting that is not as discussed but is also present in the discussions amongst scientists in the late 20th century about this late 19th century about this is this idea that you can't actually render a simulation of two things at once. Um so there's lots of really interesting uh sort of visual illusions around this where you can see something. The famous one is it's either a duck or a rabbit. Yep. Exactly. And um it's interesting. Yeah. And then so you can see that staircase is either moving up to the left or you're under the staircase looking upwards um and it's actually a ceiling that's jagged. Um and why can't the brain perceive both of those things at the same time? Well, it would make sense if you have a model that there are such things as ducks, there are such things as rabbits, there are such things as 3D shapes that operate under certain assumptions. And if that's true, then you cannot see a duck and a rabbit at the same time because there's no such thing. It cannot be the case that the staircase is looking from above and below at the same time. So what your brain is not doing is just perceiving the sensory stimuli. It's trying to infer what is a real 3D thing in the world that I am aware about that this sensory stimuli is suggesting is true and that is the thing that I'm going to render in your mind. Um, and uh, I think one way this parallels nicely to some of Hawkins's ideas actually is if you hold the thousand brains theory to be true, which is the the neoortex has sort of all of these redundant overlapping models of objects, then it would make a lot of sense that we want to synergize these models to render one thing at a time. You don't want to have 15 different things rendered because then it's really hard to evaluate them and vote between these different columns. Um, so it makes sense that the brain does is say, "Let me integrate all the input across sensory stimuli and render one sort of symphony of of models in my mind so I can see one thing at a time." So that's this idea of perception by inference. Um, at the time no one really connected that to the idea of planning. So this was just this idea that what we perceive is different from the sensory stimuli we we uh get. Um but later on sort of uh as a the world of AI started thinking about things from the perspective of perception by inference um what we end up realizing is this idea of perception by inference if you're going to train a model to do that it comes with this notion of generation um so uh because the way it self-supervises is it takes the prior and it tries to make predictions and it compares the predictions in the world to the sensory stimuli and as long as those predictions are below a threshold I maintain my prior. Um, so a famous version of this is the Helm Holtz machine which Hinton devised. I think that was in the 80s. Um, it could be later I forget. Yep. And so this is this basic idea that you can build a model he called, you know, a lot of people use the term latent representation. Some people don't like that term for a variety of philosophical reasons. Um, but it builds a representation of things by virtue of building a model. Um, in other words, perception by inference. And the way you build that model is you're constantly comparing generating predictions from that model to what actually occurs. And this also has synergies to a lot of Hawkins's ideas where uh we think about intelligence as prediction. Um and so um what that means is if you if you build a model of perception by inference by virtue of generation then it's relatively easy to say okay well what happens if I just turn off sensory stimuli and I start exploring the latent representation. Well now we're exploring a simulated world. Um, I'm able to cut off sensory stimuli, close my eyes, and imagine a chair, rotate the chair, change the color of the chair. And because this model is a relatively good, uh, has relatively rich features about how the world actually works, um, I can model things without ever having experienced them, um, without ever having done them and relatively reasonably predict what what, um, is real is would actually happen if I were to do those things. So what I think is interesting and perhaps somewhat of a novel uh um [clears throat] uh proposal in the book is I think a lot of people think about the neoortex as having adaptive value because of how good it is at recognizing things in the world. Right? So a lot you know if you read a standard uh textbook a lot of what people will talk about the neoortex is how good it is at perceiving things object recognition. Um some of the best studied parts of the brain are this visual neoortex. So we understand reasonably well how we're building models of visual objects etc. But from an evolutionary perspective this is a little bit hard uh to to find convincing because if you actually examine the object recognition of vertebrates it's incredibly accurate. I mean a fish can recognize human faces. Um a fish can oneot recognize an object when rotated in 3D space. So, it's hard to find a dividing line between object recognition in animals with a neoortex and object recognition in objects without a neoortex and with uh with brain structures that seem more similar to early vertebrates. >> Um, did you have a question? Yeah. >> Well, well, yeah, I just wanted to touch on a couple things there. So um Hawkins said that the reason I mean he said first of all that we overcome the binding problem by having this kind of profusion of individual sensory motor models rather than having this [clears throat] kind of you know feed forward enrichment of representations >> and that was really interesting but he also said the reason why having let's say we've got 150,000 mini cortical columns that are just wired to different sensory motor signals and he said it's the it's the kind of diversity and sparsity that gave the robustness of recognition And then you can kind of think okay well well what do the representations look like? So if our brain builds some kind of a a model of the world some kind of topological model it it must be a representation not necessarily it's not like a a hunulous or it's not like a stapler inside the brain if I'm modeling a stapler it's actually some weird structure of the stapler but as seen by every way I can touch feel here and you know lick a stapler or whatever. So it's difficult for us to imagine what that is. But the reason I'm the reason I'm going down this road though because that's a bit weird, isn't it? So we have this very weird representation of things and then I come to Hinton's um Helm Holtzian generative model and you can get it to generate let's say a number eight if it's trained to do numbers right and Hinton would argue and I would disagree with him that that the model understands what an eight is. Now, this is weird, isn't it? Because we understand what a mouse is, but intuitively we feel that a neural network doesn't understand what an eight is. And I would argue that we're getting into semantics here. So, I think the reason we understand things is because there's a relational component to understanding. So, semantics is is about the ontology. you know our rep the way we feel about things where the thing came from what was the intention guiding the creation of the thing what was the provenence of the thing it's almost like the interconnectedness of the thing tells you more about the meaning of the thing than the actual thing itself in a weird way what what do you think >> well I do think I think uh this is where uh the word understanding I think can mean different things to different people can understand the word understanding in different ways Um I think there is absolutely uh something to the idea that uh just because you can recognize something. So that would be a feed forward network that can observe a stapler. Um uh alone is insufficient to what most people would mean when they use the word understanding. Um and just to talk about interrelatedness, if I have a feed forward network that you know let's say it's just a binary classifier, is this thing a stapler or not? Um, now I can't ask lots of things of that feed forward network that I would expect of a uh an agent or a model that understands what a stapler is. I couldn't ask it, for example, what would happen if I uh burned the stapler? What would happen if I opened it? What would you see inside of it? I can't ask um what does a stapler do? Um I can't say show show a human holding a stapler and a set of objects in front of them and what do you think the person's going to do next with this stapler? And so and so clearly understanding most of our intuitive understanding of the word understanding um contains some richness that's not included in just classifying or recognizing the presence of objects. So I think that's absolutely the case. Um my intuitions sort of fall in the direction of uh what we mean typically when we use the word understanding it comes to having something that can be mentally explored. Um, and so I think that could that requires what you're describing, which is the interrelatedness of things because when I see someone holding a stapler and you ask me the question, what are the things they would likely do? Well, I start imagining what they might do with that stapler and then I can evaluate which ones seem plausible to me. Um, and so in the imagination and evaluation of which things seem plausible, there's an interconnectedness between the thing and the the world around it. Um but yes I if I absolutely agree with you that just recognizing objects clearly lacks something that we mean when we say understanding. >> Yeah. Do do you take into consideration the memephere? So you know we've got this kind of um we've got ideas that are quite collective as well. I guess I was going to explore this with you later but um maybe maybe let me frame it like this. I I have this um intuition that a lot of our intelligence is outside of the brain. So, it's almost like if if I was in the wilderness and, you know, disconnected from society, um, I would be much, you know, a lesser human being. In a weird way, I might have more agency, but, you know, I I wouldn't have access to all of these rich cultural tools and knowledge and patterns and and so on. So, it it's almost like that's where a lot of our intelligence comes from rather than and and and also meaning as well rather than than just being able to plan in the brain and so on. And there must be some kind of interplay. So culture must shape the development of our brain and and vice versa. But culture seems to be more dynamic. But how how do you wrestle with that? >> Well, I mean there it's that's undeniably true. I mean uh the the first example that comes to mind is just writing. I mean what would humans be if you removed the technology of writing? I mean we would we would all realize we're not that smart. Um I mean writing is a technology that externalizes a feature of the brain. Um uh which is memory which brains are not that great at. Um we do a good job of like condensing aspects of a memory. So we uh for episodic like things and procedural memory we're relatively good at. Um but for semantic memories we're terrible. Um and so externalizing that with writing is one of the key technologies that enables us to be way smarter than we are in fact uh because we now have this external device that enables us to store largely infinite numbers of memories and translate them across generations. So that alone I think proves your point which is uh what humans are capable of is clearly some relationship between brains and external things that can be writing tools uh that can be other brains um you know sharing ideas and getting challenged. Um so yeah you're absolutely I agree with you in >> intelligence is like the it's the dynamics isn't it? It's it's all of the low-level, you know, dynamics of things interacting with each other. And we can take a snapshot of language and and we might say, "Oh, well, yeah, you know, language isn't isn't the intelligence, but but language itself is a form of intelligence." I'm not just talking about the words and language models and so on. We're talking about the actual language in in our culture. And I guess I I think of that as as almost a distinct form of intelligence. >> Yeah. Yeah, I mean I I think um it we all it almost gets into a philosophical territory as where do you draw the bounding boxes over the things that are imbued with intelligence and the things that are supportive mechanisms that that allow those things to have intelligence. So through one lens you could you could think about brains as the physical entity in which intelligence is instantiated and then language is a supportive tool. You could take a very perhaps odd view which is language is the thing that's evolving and it's just instantiated in these brains that sort of produce the language, consume the language, but it's language that's that's evolving. The same way that you know we don't think about intelligence on the level of an individual neuron. Um we don't imbue a neuron with intelligence but we think on the scale of the 86 billion neurons something has emergently uh appeared that we do deem intelligent. Um, you know, there's some great sci-fi books, uh, where, you know, intelligence gets instantiated in colonies of ants where each individual ants isn't not intelligent, but somehow the colony itself is capable of doing incredible abstractions and whatever. So, so yes, I think there's there's very interesting ways to and it's not obvious how one divides the lines between what are the physical entities in which these are instantiated. That said, I have a particular interest in brains because I think if we're looking for for uh what is the physical manifestation that we can learn from and thus try to uh one to understand ourselves just I think all species have an interest in understanding ourselves. Um but then also uh if we want to try and borrow some ideas from how biological intelligence works into AI, I think the of all the physical things to examine, the brain to me seems clearly the one that is probably the most rich with insight. Um but I no I agree. I I think your point is well taken. >> Interesting. Okay. Um I want to close the loop on what what you said about the brain being an imagination filling in machine. So you said that it does filling in. It's one at a time. it can't unsee visual illusions and evidence is seen in the wiring of the neoortex itself you say. So it's um um shown to have many properties consistent with a generative model. The evidence is seen in the surprising symmetry the ironclad inseparability between perception and imagination that is found in generative models in the neoortex. And you give examples like um um illusions and how humans succumb to hallucinations why we dream and sleep and even the inner workings of imagination itself. So it really seems plausible when when you kind of think of it in that way. >> Yeah. I think I think there's there's a reason why um so much of the neuroscience community I mean what I'm saying there is not really uh novel um so much the neuroscience community has sort of rallied around this idea of predictive coding which is very related to active inference um and generative models um because there's just so much evidence that what's going on in the neoortex uh the imagination of things uh episodic memories I mean it's there's been some good evidence that epis sodic memory and in other words thinking about past and imagining the future are in fact the same underlying process happening in the neoortex which is again consistent with this idea that there's a generative model um if we look at the connectivity patterns I didn't talk about this too deeply in the book is a little technical but um what you would expect from a generative model is backwards connections would be much richer than forward connections because you're modulating uh downstream um of course neo course is not perfectly hierarchal but things that are in general lower in in the hierarchy would have lots of of uh inputs from uh parts of the neoortex that are higher in the hierarchy. That's absolutely what we see. Um so yeah, there's a lot of evidence uh that these are two sides of the same coin um which is there's some form of generative model being implemented. I do think in AI one way in which this manifests is the the very clear success of self-s supervision. I mean this idea although the um actual predictive coding algorithms that people have sort of devised uh as the neo as uh neoortex implementing when we've actually modeled them they haven't outperformed any of the stuff going on in in AI world the principle of self-s supervision which is can a system end up having really interesting emergent properties and generalize well when you only train it on predicting sensory input that it receives. Um and that clearly has become the case. I mean if the transformer I think is a great example of if you just give it a bunch of data and you train it through self-supervision i.e. masking so you hide certain data inputs um it becomes remarkably accurate and good at generalizing across data that it hasn't seen before which is in principle all a gener what people are predicting or claiming that the neoortex is doing as generative model. Yeah, it it feels to me that there is a bright difference though between let's say a transformer and and the neoortex. And I think the difference is um maybe agency is not the right word, but that you can think of the the neurons I think as having some kind of autonomy. So they're sending messages to each other and then the other, you know, it's eventually consistent. So the other neuron will get the message and it will decide itself what it's going to do. and um in a transformer just because of the way they're connected and the backrop algorithm and so on they they all um they all kind of ride a Mexican wave together to use an analogy. So it feels like a difference in kind to me. Well I mean clear clearly as I uh argue in the book I don't think that the brain is just one big transformer. So I would agree with you in in the human brain unless you think there's something that's non-deterministic and sort of magical happening. Uh you know I think you would still say that there is either uh you know base firing rates of neurons uh that and then there's sensory input that flows up and then goes through the brain until eventually there's sort of it's affecting uh muscles until you're responding. So there is you know there is sort of a deterministic flow happening. it might not be as uh feed forward as what's happening in transformer which is definitely the case. Um but they might both be sort of uh you know deterministic in in a in a similar fashion. I think a lot of people I've had this exact sort of argument with a lot of people and one counterargument that people have towards this idea um I don't know if I fully agree with it but it's interesting is that attention heads uh really are doing something more magical than we give them credit for which is they are kind of dynamically rerouting and effectively resetting the network um based on the context that the prompt is getting. And so although technically it's just a series of you know matrix multiplications etc. Um, if in principle what's happening is these attention heads are doing something really clever where they're looking at the context of a prompt and then effectively dynamically reweing the network to decide what it cares about and what it doesn't. So there are people that think, you know, there is something really interesting happening in the transformer that might be analogous to certain things that are happening in the brain. Um, but yeah, clearly, you know, these these feed forward networks are are not capturing everything that's going on in the brain. Yeah, it's really interesting what you said because the way I read that is um things like chatbt and language models, they are entropy smuggling or agency smuggling. So what that means is they kind of just do what you tell them to do and all of the kind of um the agency. So my directedness comes from me. So I give it a prompt, it does the thing that I wanted it to do and then the kind of the the mapping that you were talking about, I interpret that a bit like a database query. So you know depending on the prompt you give it it'll activate a certain part of the representation space and it will give you a certain result back. But the the the brain has this thing where all of the neurons have their own directedness and and the weird thing is at the cosmic scale um agency is it seems to emerge. So even transformer models that were acting autonomously could presumably in large enough scale give rise to something that we think of as directedness or goals or purpose or or whatever. But it's almost like um in the natural world because there are so many levels, scales and scales of independent autonomous things just kind of mingling with each other independently and then like downstream mixing their information together and rinsing and repeating over many many different scales. That seems to be the thing that gives rise to all of these amazing things like agency and creativity and and etc. >> Yeah. Yeah. Yeah, I mean I think the the notion of agency is an interesting one where um I really am amenable. I mean there is sort of a a I don't know if I would call it a schism in the field but there are there are there is a debate where uh between sort of the reinforcement learning world and the active inference world where um how much of intelligence can be conceived as optimizing a reward function um and sort of the re the hardcore reinforcement learning world is like everything is just a reward and then the active inference world is you know would argue that not all behavior is driven just by optimizing a single reward function. There is some uncertainty minimization. Um there is uh trying to satisfy your own model of yourself, fulfill your own predictions. These sort of things that seem very well aligned to behavior we see. But uh you know it's unclear which of these is right. It's probably some balance of the two. But to me agency people would conceive of agency differently in these two worlds, right? So I think uh some people in the RL world would say agency is just you give something a reward function and then it just learns over time try and optimize that reward. Um in the more active inference world which I do think has legs and I'm obviously amendable to um the idea of agency is a little bit more. It's building a model of yourself um and trying to infer what your goals are um based on observing yourself and then trying to make predictions to fulfill those end goals. In other words, it's constructed. Goals are constructed. And this is sort of one of my favorite Fristen papers uh is predictions not commands. I don't know if you've read that paper, but I think it's a brilliant paper um about how you could reconceive motor cortex not as sending motor commands to your body, but actually as building a model of yourself and predicting what what will happen. And the way the spinal cord is wired is it just fulfills those predictions. Um, and I think that's a really interesting uh sort of reframe of how you could get agency and really interesting smart behavior in the absence of just a strict reward function, right? So, how that would learn is it's trying to model um the behaviors it observes and then it's trying to sort of predict those and fulfill them. Um, so but yeah, I yeah, I think agency is a really interesting concept because it sort of manifests itself in these different paradigms in different ways. Yeah, I I find it fascinating. I mean, um because the way I read it in active inference literature, it's a very principled definition of an agent and there's still a bit of a gap because I think Friston would argue well in the natural world because of the the laws of physics and particles and and whatnot, you get the emergence of things and things become agents when they have a certain, you know, um depth of planning, should we say. Yep. But um yeah, it's it's really really interesting and I guess he would argue that you get all of these um phenomena that give rise to agency like biotic self-organization and and so on. But um yeah, maybe we should do we we'll slowly go go in that direction. So you you give the example of um mice doing planning. Can you can you sketch that out? >> Yeah. So this is another real like area of neuroscience research that I just absolutely love. Um so it was the case I think it was the 40s or 50s I forget the exact decade uh where Tolman observed that worms when they uh uh mice when they would reach choice points in mazes where he was training them to navigate around mazes would pause and then they would sniff back and forth and then they would choose an action. And so he hypothesized this idea that uh they must be engaging in vi in uh vicarious trial and error. They must be imagining possible outcomes before deciding. And of course this was hugely controversial um because there's no evidence. He had no evidence that they were in fact imagining anything. And of course most people like to in the absence of evidence assume animals are as dumb as possible. Only when there's irrefutable evidence will we imbue them with any intellectual capacities which I think isn't interesting human bias but that's fine. Um and then uh David Reddish uh who is also a close friend and mentor of mine uh he did some amazing research uh with some one of his PhD students. um where they were recording hippocample play cells. So it's very quick background for for viewers. Um you can go into the hippocampus of really any mammal, but this is best well studied in rats. And part of their hippocampus region called CA1 has these things called play cells. And so if you look at if you record it as a mouse is moving around a maze, what you find is this incredible thing where there are neurons that activate only in specific locations in that two-dimensional plane. Um and it's not based on how they got there. It is egocentric is independent of their egocentric path. It is alocentric meaning in this in the plane of external space. So they can come back to the same place from any route and that same play cell will activate. And as an aside um from the evolutionary story, we find similar types of cells. It's not exactly as accurate, but we find similar types of cells in fish in in the homologous region of the hippocampus um of their cortex where they have uh place-like cells. It's not as accurate, but cells that activate in certain locations in a maze. What he found is when mice engage in this act of vicarious trial and error, meaning when they pause and look back and forth, the place cells in the hippocampus cease to only activate in the location they are, it actually starts activating down the paths of each route it might take. In other words, you can literally watch rats imagining the future. One of the most inc I think one of the most incredible neuroscience findings. Um, and so he then took this and then did a bunch of other experiments which I think reveal even further the power of sort of imagination in in rats. One of my favorites is his counterfactual learning studies where he puts rats in this thing called restaurant row where it's a sort of uh square like Yeah, exactly. It's this like a square like maze and as the rat is going uh counterclockwise at each sort of door a sound is released or made and that sound signals to the rat whether or not they can go right through the door and get food in like I think it's like 3 seconds or they're going to have to wait 45 seconds before they get the food and they're given a bunch of time to try and get as much food as they want. Rats have clear preferences. So, some rats will really prefer bananas and they don't really like the bland food. So, what happens? Well, this presents a set of irreversible choices to a rat. So, let's say they come up and they can either get uh they can either get a treat uh right now. They can they can get a treat right now that they don't really like that much. Let's say it's the bland treat or they can go to the next one and hope that they're going to get the banana really fast. If they go to the next one and the banana sound is long, meaning 45 seconds, then they regret their choice because it would have been better if they just went in and quickly got the food. How do we know they're regretting the choice? We can literally watch them imagining eating the foregone choice. We can go in a part of their brain called the orbital frontal cortex, which activates for certain types of tastence, and we know we can see them imagining the foregone choice, and they end up making different choices the next time around. They end up being less likely to forego that that choice in the future. So I think this is just such an incredible finding of what we mean when we say model based reinforcement learning is clearly in happening in these brains uh of very very simple mammals. >> There's a real challenge knowing which simulations to run because if you think about it we've got a search problem, right? There's an intractable number of simulations to run. So how do we how do we fix that in AI and how do humans fix that? So, so this is one of the I think of the many but this is one of the big outstanding questions in AI which is how do you effectively prune the search space? We do not know how mammal brains do this so well. So I can I'll give you some like highle ideas or theories but we we just don't know and this is one of the big possible breakthroughs um in figuring out you know how how do mammal brains do such a good job of this. So the thing that AlphaGo does which I think perhaps is a clue um and is clever is the selection of the search space actually is bootstrapped on the temporal difference learning model under the hood. So this is actually very clever which is let's say you train something to uh learn without a model of the world. All it's doing is it gets sensory stimuli. It gets a model of a go board and then it just predicts the right next action. It just has a policy value function that bootstraps on each other etc. So no planning. If you want to add planning to that, um, what they did, which is quite brilliant, is you say, okay, well, you know what? Instead of building some other system to try and choose good trajectories, why don't we just use the the policy network and we just don't only pick the first one, we pick its favorite move, but then we also look at what's your second favorite move, your third favorite move, and maybe your fourth favorite move. And then let's literally play the games out. Um, and let's just see let's play a bunch of games against ourselves and then see the ratios in which we win one of them. And then what we might learn is your second best guess was actually better than your first best guess. But we're not starting from every possible possibility. We're saying let's bootstrap on our best guesses of good moves, but then check them by playing out the possible futures. So if we were to analogize that to the brain, what that would suggest is perhaps it's the basil ganglia, which a lot of evidence suggests is engaging in this type of sort of model free reinforcement learning actually is the thing that chooses the moves, but there's some other system that lets us choose the second best move, the third best move, etc. Um, one way this might happen, there's some evidence for this, far from conclusive, that there's some notion of uncertainty that frontal cortex or basil ganglia is measuring and when the level of uncertainty between the next actions uh passes a threshold, pausing occurs. Um, because when we see animals do this vicarious trial and error, it almost always occurs in moments of high uncertainty when contingencies have changed, when the right answer is not obvious. Um, and so you could conceive of this as a policy network where you're evaluating its best choice, second best choice, third best choice. And when there's uncertainty about it, in other words, they're close together or um, there's some other measure of uncertainty. Perhaps you have parallel policy models that and you're comparing the similarity between them. A lot of different ways to do this. Um, that triggers a process of playing forward. This is another key thing that mammal brains do that AlphaGo does not do. Alph Go engage in planning on every move. So there was never the question of when do we pause to plan in a game of go it doesn't matter just engage in planning in every move because we can do it so fast in the real world there's so much uncertainty and noise um and we need to be so energy efficient with human brains we can't engage in planning every instant so we need some mechanism that tells us when to stop and think about what I'm going to do next and when I can just continuously go with model free choice. This is also something we don't know how mammal brains do. Um but I think you know a reasonable uh sort of speculation is that there's some uncertainty measurement that's occurring. One last point I'll make that I think parallels in an interesting way to some of Hawkins ideas um and FO's ideas is if you take the thousand brains uh model and you apply it to frontal cortex in other words we have multiple parallel models of ourselves. Um you could imagine that there's an uncertainty measurement the same way we do uncertainty measurement uh in a lot of deep learning models where you create parallel models and you just measure how similar are the predictions of the model and if multiple parallel models predict similar things we just measure it as low uncertainty. When they diverge substantially all of a sudden we measure high uncertainty. So again speculation but you could imagine if it is the case we have redundant models in the neoortex. Then might it be the case that somewhere perhaps the phalamus or the basil ganglia the the similarity or differences between these predictions are a measure of uncertainty that triggers pausing. Um Steven Gber has similar ideas. He calls this like matching and non-matching. Um >> yeah yeah yeah >> yeah like almost because um our ability to do abduction is something that fascinates me and there is some kind of a model selection or or or