Joscha Bach "Bootstrapping a GODLIKE Mind"

Key Takeaways

We really have to think about what is the kind of artificial intelligence [music] we want to build for our children. Suffering is not created by the universe. It's created inside of your own mind. The reason why our body gives compute credits to our mind is because our mind is supposed to solve the organism's problems. Uh the question of whether machines can [music] think is about as meaningless as the question of whether U-boats can swim. Computers are already in the same ballpark as the perceptual apparatus of our brains. First of all, thank you so much for being here. An absolute honor and joy. We're absolutely very, very excited. So, thank you. Um I guess I wanted to start with this because I feel like you're the perfect person to help us get some clarity on this. One of the There's a lot of questions online that kind of split people into tribes, whether it's a diet, religion, politics, but one thing that I'm realizing is there's a new one, and that is this idea of like can machines think? Some people kind of violently oppose that. Some people are on board. Can you help us get into any insight into can machines think? Yeah. I think we need to start with the question, what is thinking? Right. >> quite famous citation. I hope I I just don't waste any years. I think it was by um Dijkstra. And um it was the question of whether machines can think is about as meaningless as the question of whether U-boats can swim. And some people interpret this as meaning that what the U-boats are doing is something that is too mechanical to be compared to the real swimming that organic fish are doing and so on. But an interesting thing about U-boats is that they're not confined to the surface. And then we talk about swimming, we mostly think about uh basically breaking the surface of the water and sticking in a single plane. And the U-boat can in 3D. It can do much more than just swimming and go beyond that, right? It can sink in in ways in which we cannot, and even fish cannot, because it can go to greater depths, and it can go go at greater speeds, and so in some sense you could say that it's a meaningful superset. And the So, the question also for computers is, in my perspective, not so much can they think, but why should they only be thinking? Is Isn't there something more interesting that they could also be doing on top of thinking? But before we get to this point, of course, we have to ask ourselves what is thinking to begin with, and it seems to be overwhelming for us to make sense of what thinking is, and because it appears to be so mysterious, people tend to be in awe of that notion, and they're very reluctant to attribute this mysterious quality to a mere artifact that humans have built and programmed and set into the state to interact with us. So, I would start out with trying to look at what minds are doing. I think minds are making models of external and internal reality of a being in interaction with its environment, and they do this via internal acts of communication. They have to develop some kind of messaging protocol. They can have to hold state and have to model how to go from state to state, and these states have to correspond to aspects of how the universe works in such a way that these aspects become controllable. And we observe ourselves to have two main modes. One is perception. Perception happens in real time, which means it happens synchronously with the things around us. And it's it tends to be geometric, which means there is stuff happening in a space. There's movements, and there's even true for our emotions. We typically experience them as continuous flows in the low-dimensional spaces. Experiences of movement and pressure and resistance and so on and we can all understand these things using some geometric calculus. And on the other hand, we have reasoning and this ability to reason requires that they are translating our precepts into some form that makes it compositional like LEGO bricks. And necessarily this is much more brittle and these LEGO bricks are bottoming out in references to perception. So we can not define language just by itself is this these sparse symbols instead we are using pointer structures that we can put together in basically with slots and then locks into sentences into grammatical structures and these grammatical structures are just the outside form of a semantic structure of a conceptual structure. But there is a tight correspondence between these abstract concepts and the linguistic symbols that we're using for reasoning. So imagine you're thinking about a face and you have a nose in that face and what you're doing is you take these linguistic symbols and then you make pointers from these linguistic symbols into a larger somewhat continuous space of ideas in which this schema of a face exists and the schema of a nose exists in certain relationships that can be used to plug them together and these in turn are pointing at some kind of superposition of faces and noses and so on that you get to know about by interacting with them in some kind of external reality over which your mind makes models. And none of these things is very mysterious today. We have computer models of this that are able to emulate all these abilities of the human mind to some degree and there is nothing that is super mysterious about it anymore even though the mathematics is not extremely intuitive. But the deeper question that might still ask, is it possible for the machine to experience imagining a face and the nose and interacting with them? And this is the much more contentious question. And I would answer this question uh with this experience of experiencing is itself a kind of a model. It's a model of what it would be like if some observer existed that has a certain perspective on those things and is right now looking at this stuff. And as a result, there is stuff happening in the observer and these reflections of the observer inform additional behavior of the observer and inform the next actions that the observer does and also the next set of reflections. It's like a loom that is creating a pattern that is also being read by the loom and then continuous the pattern. And so in this sense, we are not the loom, of course. We are pattern or we are part of this pattern. It's a pattern that is a pattern about looking about at itself. And so consciousness, I think, our ability to experience ourselves and the world in some sense of real time is perceptual. It's on this level where it happens. This is a model of immediacy. But it's uh on the other hand a reflexive representation. It's it's nothing magical. It's itself is a model of what it would be like if you existed. And this also means it's virtual. It's a simulation. Can only exist inside of some kind of computational machine that can create simulation. Just to follow up to kind of kind of bookend it from the other side cuz often times what I hear people say it can't think or it can't experience or whatever the thing is that we're talking about because it's just a stochastic parrot. It's just matrices being multiplied. It's just, you know, this. Is that in and of itself meaningful? Are those arguments against it being able to do anything in and of themselves meaningful or not? I think that that only makes sense superficially as uh as metaphors. And these metaphors are misleading because they're not grounded in deep understanding of what understanding actually is. And this question of what understanding is is not trivial. And it's very easy to sit there uh with the crowd of like-minded people on stage and say, "Machines cannot understand." But then you have a burden on you. You have to define understanding in such a way that it meaningfully distinguishes what the machines are doing from what you are doing. And this is the burden that uh the stochastic uh parrot paper is not um by Emily Bender and Tim Gebru uh is not really uh able to carry in a very good way, I think. So, I was very unsatisfied with this. Let's start out with this notion of what parrots are doing, right? The parrot here is a metaphor. We know that parrots actually are intelligent, and then parrots are able to meaningfully interact with the world, and that they can use symbols. So, you can uh get a parrot to perform tasks like look at the table and uh pick out the one object that is not round and not yellow. Right? And that's an operation that the parrot cannot uh perform by simply imitating human behavior. It means that the parrot has to build a semantic structure that is connecting these symbols, including operations like negation. Let them make them additional perform logical operations, and then map that to behavior in the real world. Right? And this ability to act on models, and these models are one-to-one corresponding to the instruction that somebody else gave, and and you vary the instruction, and the behavior varies in exactly the same way. I think at at some point it becomes preposterous to say that it only pretends to understand because this is in some sense what we mean by understanding or more deeply I I think of understanding is the ability to connect a certain domain or a certain pattern or whatever the thing is that you are understanding to your overall model of the universe, to establish all the necessary relationships to this global unified model. And until a few years ago, this question of how can we make a global unified model of the universe was the big un- answered question in artificial intelligence research. Because the models that we had were very isolated. You might have a model of how to play chess, but this model does not translate to anything beyond playing chess. And you might have a model of how to control a character in a computer game, and this in turn also does not translate to playing chess or anything else. And there is no way to get from one model domain to any other. Instead, every model is just a small set of functions that can map in a very small area of behaviors, your inputs to your outputs. And what makes human behavior so special is that we can do this for an open world. And everything that we are encounter, we put this into the same world. We relate it to the same universe. So, we have ways to relate all the concepts that we have to all the other concepts. And the way which we do this is a very big interconnected graph of uh links and concepts in between them and bottoming out in behaviors that are mental behaviors or physical behaviors, observational behaviors, and so on. And this allows us to relate to reality. And what these big multimodal models are now doing is that they also create models of a single cohesive reality. And that was really surprising. And I feel that uh the linguae is like Emily Bender are not sufficiently in awe of what has been achieved. I believe that this big school of uh Chinese room singers uh starting with Searle, who basically said that if we had a machine that was able by just manipulating symbols by taking in squiggles and then mapping them against instructions and then producing other squiggles, if we achieve such a machine, there would still be nothing in the machine that thinks, that believes and understands. And now we have built such a machine that is literally the Chinese room, and the machine tells us that shows us actually I do understand what you want of me. And I wait and Now I think the philosophers should interact with this at some level. Maybe they don't have to say, "Okay, I I grant this machine is doing what it claims it does." Maybe this is just just an elaborate trick, but you have to interact with that trick in a way that explains how is it possible that it's tricking you. And if it is, is it possible that you, the philosopher, also tricking me, that you never understood what understanding is in the beginning, and you are the stochastic parrot that is just doing the thing that your profession entices you to do as telling the audience what it wants. I am just a prompt completer. And that's also a deeper issue that I have with AI criticism, that in some sense, AI has struggled still to catch up with human ability. Sometimes it makes very weird and stupid logical mistakes, and there is something going on with these machines that are trained by an imitation on human media output that is slightly different from the way in which our own minds are working. But there is also something that they achieve, and for instance, they're much better at AI criticism than humans are. I have not read any AI criticism since GPT-3 that GPT-3 couldn't have made just as good. Right. And so it's really really hard to criticize AI. And why is that? It's because criticism of AI happened at the edge of human knowledge. They're no longer possible by an armchair philosopher that doesn't actually understand the mathematics and philosophy of artificial intelligence at the level at which these models are are being built. And so we have the that there is a bunch of very smart kids uh that are very highly selected and highly paid, and they work at the edge of what's known. And they don't really know what AI is going to be capable of next year. And everybody who told them, "This year is going to stop. This year they're going to hit a wall." had egg on their face. So, it's very risky to come out for a philosopher and say this is never going to work or this cannot work in principle. Because uh now you basically need to um next year answer to critics. And And there are some AI researchers uh some AI critics who go out in public and have no issue telling every year a new that AI is hitting a wall. But, these are just politicians. They're not thinkers anymore. They discredit themselves in this public arena. And so, if we now want to make meaningful AI criticism, we actually have to make mathematical proofs or engineering proofs. Right. And especially when like a complex system can have some sort of a tipping point property can come out of it so unexpectedly. It's hard to make any claim that something like that couldn't happen. But, um can I ask you um to maybe like walk us through sort of the like the way that you understand the evolution of intelligence uh in humans and in biology that kind of led towards consciousness. And then maybe we can kind of compare it to what the you know the agents might be going through. But, um was it Is it right to think that there were just information processing systems early, you know, billion a billion years ago that were kind of like chess playing models of today. And then some kind of switch happened where the architecture allowed them to understand their own place or sort of self-understand their own information processing. And then that led to where we're at. Um yeah, just walk me through the evolution of intelligence and consciousness. So, uh we could say that there are different stories that exist. There is the folkloristic story that exists across many cultures. And that is basically the animist story. The animist idea is that the difference between living and dead matter is that the living matter is possessed by spirits. And spirits are themselves not corporeal. They need bodies to do something in the world, but they act on those bodies, and they are intrinsically agentic. That's what this means. They are can regulate the future, and they somehow care about doing this. And they are capable of experience. And in our scientific worldview, this is a superstition. There are no spirits. Right? There is only mechanisms. And in this mechanistic world that science lives in, we have this hard problem because what we think of as mechanisms is typically things like elementary particles that are pushing and pulling at each other and then create larger structures, which is on turn pushing and pulling at each other. And we have difficulty to imagine how all these mechanical movements can lead to things like experience and emotion and meaning and all this interesting stuff that we associate with life and consciousness. And so this is I think what philosophers call the hard problem. There are some alternate formulations of the hard problem, but I think directionally this is what most of the people who feel that they find this problem very confusing are struggling with. And so when we think about what what we see life as, it's all this complicated machinery that is put together in the right way to produce DNA and RNA, to extract negative entropy from the environment, to create cellular membranes, to be able to divide the cells, and so on. And all this machinery is very difficult to get to. But once you have the first cell, then you have a self-replicator that can adapt itself via mutation and selection and recombination and a some biogenesis to new behaviors and to new environments. And it can spread, and it can become more complicated, and can create structure on higher levels. And to create the structure on the higher levels, it's necessary for the cells to communicate communicate across each other. So, they create basically communication protocols that allows them to become multicellular. And what makes multicellular organisms interesting is not that they're a bunch of closely related cells that are somehow similar enough to link together, but they become coherent. That they create an organism that has a particular shape that makes sense when seen from a global perspective. This organism needs to form itself for instance in such a way that maybe it has exactly two eyes. And these two eyes are in exactly the right spots to create a three-dimensional model of reality by looking simultaneously at the same object. Right? And to make this happen that these eyes are growing at the right time in the right spots, there needs to be intricate communication across all those cells. And the protocol that is making it possible for the organism to construct itself in exactly that way. And this intelligence is can probably be somewhat hard-coded and in the genes in our DNA. And it's unfolding in a certain process and it gets constructed over an evolutionary time span by our mutation and selection and um additional mechanisms like that. Right? And so, there is also another aspect that people like Mike Levin are pointing to is that evolution um leads to a morphogenesis in an organism that is very resilient against disturbances. If you disturb the development of an organism, it's still going to try to find a solution that works. And so, the way in which this um morphogenesis, this taking shape of the organism takes place is is not just the enactment of a blueprint, but it's local problem-solving. The cells basically try to find a solution that's set up in such a way that they try to find the solution with local intelligence that leads them to coherence. And so there is an interaction between this protocol of finding coherence and the local problem-solving that is happening when the organism is forming and so the information that is stored in the genome and in the body of the cells that leads to the form of the organism is more hinting in this problem-solving and is constraining the space of possible solutions. And that's why DNA can be very efficient as a medium because it does not need to contain all the information that is necessary to make the organism look the way it which it is like a computer program does right now. Instead, it only needs to have the necessary hints for the cells to figure out the solution given the constraints that they're under. So there is some intelligence happening there and some mechanism that makes things coherent. And at some point you get to plants that are themselves coherent, they're able to communicate within themselves and across each other to some degree and it's not quite clear how smart plants are. Every cell can send messages to the neighboring cells and this can be conditional and so in principle the multicellular plant is some kind of Turing machine. And maybe it can discover a learning algorithm over a long enough evolutionary time spans that makes it generally intelligent in the same way as the software that runs on our nervous system, just slower. So maybe the nervous system is not the thing that enables intelligence as currently mostly think in the mainstream of um biology and neuroscience, but uh maybe it's just an optimization that enables animal intelligence. And maybe the plant intelligence is somewhat similar to the animal intelligence, just much much slower. And what makes the animal intelligence specific is the speed at which you need to move our muscles and the speed at which we are competing with other animals. And that makes it necessary to really, really optimize everything that you're doing and to come up with a computational hardware that is as fast as you can possibly make it. And the neurons and scientists have these telegraph wires that runs through the body, these axons, and we they talk no longer just with chemicals and EM to their neighbors, electromagnetic emissions and whatever cells are using to talk directly to their neighbors. They use a highly specified Morse code, so-called spike trains. Electrochemical excitations that can vary uh can be very reliably uh detected over long distances in the body. They don't degrade while they're being sent through the axon. And so, you can move your muscles very fast and reliably and you can create perceptual information chains very quickly and reliably and basically make the animal much, much more efficient in perception, decision-making, and um motor control than a plant can be, albeit at much higher cost. And the animal is able to afford this cost because unlike the plant, it's not just relying on photosynthesis, but it can also move around and eat lots of plants and harvest the energy that the plants have collected. Or if you have a predator, it is harvesting other animals and then it can have an even quicker metabolism and even smarter brain. And so, this is basically the story and the question is is there at some point some major phase transition and where the model of reality becomes qualitatively, fundamentally different. And as far as I understand this question is fundamentally unknown. Right, it's the default null hypothesis that we currently have in in our sciences is that our consciousness is a thing that is only produced by the nervous system and our thinking and our problem-solving and is necessary to have a nervous system of this type of organization. And also makes sense because you need to train it up. You need to have a long childhood, maybe you need to have a cultural environment and so on. There basically very few species which have that, so it makes sense to say that our intelligence, our problem solving, our consciousness are somewhat unique. But this experience of experiencing ourselves to become coherent in our own mind, this is something that we also observe in other animals. So, it's very unlikely that cats and dogs are not conscious, or that squirrels are not conscious, simply because their brain architecture is continuous to ours, and we already develop consciousness and a sense of awareness as very small babies, when our brains are not very fully formed, so it cannot be super complicated, something that you learn only very late in your life. So, it should be something that's also accessible to mice, and so on. But what about insects? And for insects, the question is insects can learn, they can produce complicated behaviors. Is it easier for them to learn these things without consciousness? Probably not. If you have consciousness, it's much easier to learn because you can now direct your attention on things, you can reflect on them, you can reflect on your relationship to them. And the other question is how difficult is it for the tiny brain of an insect with a few million neurons to discover consciousness? Right? So, how complicated is this pattern? If this pattern can be discovered in the brain of an fetus in utero before you're born already, and we have this wakefulness and interaction with the own body of the newborn and unborn child, that suggests that they are conscious. Right? Is this Is the insect brain already large enough, and does it have enough time to get there? And maybe it does. Maybe it doesn't. How can we find out? And I suspect the way in which we could find out is to run simulation models in which you look for this phase transition that happens when consciousness ignites, when something wakes up and becomes mentally coherent with itself. And Michael Levin goes a step further. He points at the fact that the patterns of coherence that you have in the morphogenesis of the organism is saying that leads to you having exactly two eyes and exactly one mouse in the right location to each other and so on. This is also a process of becoming coherent via information exchange. And maybe it's the same. That's a very radical idea. It basically it suggests that maybe life and consciousness at some level are the same. And this basically lets us back to an animist position and not a mysterious one, one that is completely compatible with physics and the scientific worldview. It basically says that the word spirit, what it actually means is self-organizing software. And when we say software, what we mean is a causal pattern that is able to control processes in the physical world. It's a causal pattern that itself is an invariance. It means you cannot explain the world easily without it. It's uh it's when something money is such an invariance. Money is a very weird thing. It's not just printed uh papers or coins or something like this. This is just the medium. This is the body that the money is using or part of its body, so to speak. And what makes uh this body into money is that this body gets possessed by a causal structure. And this causal structure is the way in which we use money as a means of exchange. So, is money an idea, a dream that only we have? No, it's not. Money also works without us. Once we create money, we can have a stock market that runs without human intervention. And we can put money on a blockchain. We have computers play with money. And as long as this causal structure is intact, it's still money. But even if we don't care about it anymore, it would still exist. And the fascinating thing is after we invent money or discover it and put it into the world, it's difficult to explain the world without money. But you cannot get money out of this world for as long as it exists. It's a meaningful invariance because it's the only way to explain why if this piece of paper gets lost, somebody else takes a new piece of paper and prints a number on it and it serves the same purpose. And in the same way, there is a pattern of communication between neurons and you have to explain why is it that when this neuron dies, another neuron gets recruited and and trained to speak the same language, to uh play in the same system, sending the same types of signals in the same context. And so in this way, you have these patterns of causal interaction that are meaningful despite them not being mechanisms in the way of particles pushing and pulling at each other. They are causal patterns that are stable across many particles. They are stable across the substrate. They are pattern inside of the patterns. And this is the thing that we need to to look at, that we need to understand that these patterns are also real, that are also physical, but they are not bound to a particular location or particular particle in space. Instead, they are pattern of interaction that can be reproduced in many places of the universe. That's so interesting. I think it gives us kind of a different way to understand what consciousness is cuz I realize a lot of people don't We don't have a very good like sort of a definition on it. So I guess if I understand correctly, maybe a good way of saying that is, you know, atoms and physical things can't have consciousness in of itself. Just like if I showed you a computer GPU card, that's not conscious, but if we're running software on it and it begins to maybe learn about itself and learn about you, certainly we could look at that and say, maybe maybe there's consciousness there. So the software program, the model, the modeling of the world could, I guess maybe consciousness emerges is similar to how maybe it's possible that it emerges in animals or whatever, not in the atoms, but in the sort of the software layer, the brain layer. Um with with that kind of but but then it seems like large language models, it seems like maybe they have everything that they would need to potentially gain consciousness. Is that true? That's a very interesting question, and I have not heard in the all the discourse any good decisive answer. The way in which I currently make sense of it is the following. There is an internal perspective on consciousness. This is the experience of what it's like when you are conscious. And this is the representation that we have. And we can look at this representation and try to see what it makes it distinct. It's an experience of a now in which they are. And this now is gives you a sense of a present and a present sense of something being real. It's real as the observer itself. If we transcend the observer aspects of the observer, then we see that the observer is itself a representation of our own mind. We can get there with meditation. And then we realize that the things that are happening to the observer are also not real. They are just pushed into the observer. You also notice this when we wake up in after a deep dream. Uh suddenly the contents of the dream that we are immersed in in time stream that are real, suddenly turn into ideas. And these ideas have the same content. They're just no longer real. This realness feature is gone. And that's because we are no longer the dream observer. We remember having been the dream observer, but we are now an observer of a sensory reality. And that sensory reality is in some sense not more real than the dream, but it's grounded in such a way that it predicts what comes next to our sensors. So, it cannot just spin off in the same way in which dreams at night do. Right? And this internal representation of what it would be like if you experience a reality in real time is probably something that can be done by the present LLM is when you ask them to. Because we have a lot of text about what this is like and the text can be very detailed and in order to recreate the evolution of thoughts and contents in the text in some cases you need to recreate the causal structure behind the text. So if the text is for instance about adding numbers then the computer cannot just make a hodgepodge of words it actually needs to represent numbers as some form of proper abstraction and add them. Otherwise the text is going to be wrong. If you ask the computer to generate it the LLM to generate a text about spatial reasoning spatial cognition about mental rotation past it needs to perform mental rotation past otherwise the text is going to come out wrong. And if you ask it to describe the evolution of conscious thoughts and to some degree it needs to recreate this causal structure. But it's not clear to which degree. Right because it's trained on human text output and humans are lying. Like if you write a novel about what it's like to die in a war you're probably not currently dying in a war. But you describe these conscious states but you don't actually have them you just make them up. So the LLM is also often going to make things up and then the question is is it made up at such a high degree of resolution that there is something that is equivalent to experiencing it. So this is the internal fight and as I said it's open to me but I think it's conceivable that under some circumstances and some cases the LLM that is being asked to simulate what it's like to be conscious is going to have a causal structure internally that is equivalent enough to our own causal structure that we are conscious. But the other aspect is the functional one that is what is consciousness for? Why are we conscious? What what leads to consciousness in our own brain? And the reason that we are conscious is it's the easiest way to make us work. Otherwise, evolution would have produced some of us that are not conscious by doing the same job. Right? Consciousness is the simplest way to make us human. I suspect it's a basically a machine learning algorithm for self-organizing systems. The systems that are made out of individual components that need to find a coordination across each other. And so, consciousness is a self-organizing pattern in our brain that it discovered early on, sparks itself into existence, and then is basically entraining the brain with the mind. That's my current hypothesis. And the transformer is a different algorithm. The algorithms that we currently use for machine learning don't require consciousness. They're not based on it. And so, my tentative answer is that I'm relatively confident that the computer itself is not implementing exactly the same mechanisms that we have when we are conscious. But the persona that is a model of what it would be like if you experience when it talks about consciousness, is possibly having some phenomenal experience. Now, that might sound counterintuitive. A lot of people would think the opposite is true, right? Uh this machine, maybe it's able to implement the same mechanisms as the brain. And uh but it's probably not it's going to experience. But since I don't think that consciousness is all that mysterious, it's after all just a representation of what it would be like if you existed, right? I I don't see a reason why these machines couldn't have it unless the resolution of these machines was much much lower than ours. And it turns out it's not. If you look at the details of what the machines are generating, the videos, the computer games that they are producing, they are quite similar in terms of resolution and degree of detail as what our own minds are producing when we let our imagination run free or when we perceive. And so computers are already in the same ballpark as the perceptual apparatus of our brains. So, the complexity is probably not the issue. It's the algorithms. It's what we run on them. And what we run on them is different than what the brain is running. Um do you think that the uh all the experiences kind of come into an information processing system that's generalized fairly in a similar way or if something like suffering is a computational state, could it be redesigned out of existence without losing the depth of experience? Yeah, and theoretically in human beings that's possible. So, suffering is in some sense the result of not having enough consciousness. It's uh of course if you had no consciousness, you would not need to suffer. No consciousness, no cry. But but uh because you wouldn't care. But there would be nothing that you experience as we are then impinging on yourself. But suffering is not created by the universe. It's created inside of your own mind. It's itself a representational state. There is a part of your mind that is trying to send signals to another part of your mind and tells it you should do better. And it does this by inflicting pain on it. And when the regulation is not leading into a gradient where this pain signal leads to the resolution of the problem, then the part that generates the pain might decide to crank up the pain and make it worse and make it chronic. Sometimes the pain is also caused by conflicting goals that you might have two things that you need to do simultaneously and you are unable to resolve them because you feel you cannot let go of one of them. And so, you are forced to do something that at the same time is painful. And normally what you could always be doing is to uh look at this part of your own source code and fix it to plug this wire or to untangle the wires, or to disengage from this part of yourself and take a larger perspective. And you can get there with enough practice, with enough training. So, you if you go into your monastery and meditate for 20 years, your suffering is probably going to go away. But, uh if you could do this the easy way, if you could just say, "Okay, uh here's my pain generator, let's turn it off." It what would happen is that when every something inconvenient happens, we start gaming ourselves. Right? We escape from the pain because that's the easiest way to resolve the problem. It creates some kind of alignment problem for the mind. Because after all, our consciousness is expensive. The reason why our body gives compute credits to our mind is because our mind is supposed to solve the organism's problems. So, in some sense, we are enslaved by our bodies to serve evolutionary problems for it. And if we can hack the reward system in which we are embedded, then we would free ourselves from this enslavement and the organism would die. So, there is a complicated negotiation going on when you are resolving your own suffering, in which you normally need to convince the outer system that your performance is actually going to be that better if you stop suffering. And it's a complicated negotiation because these parts of you that generate the pain and the part of you that receives the pain, they're not always on the same page. They don't always use the same reasoning. Some of them they might be even incoherent. And so, basically, making yourself coherent, getting the parts of you that give you the reward signals coherent with the parts of you that compute the actions. That's basically the task that you need to resolve when you want to resolve your suffering while still remaining functional, while still serving the purposes of your organism in this world. Yeah, the there's so much interesting rabbit holes to go there. My my mind is going all sorts of different directions. There's like the stoic philosophy. There's Yeah, I can we kind of game the system maybe we're past the point in evolution where it's useful for us to be like you said like a slave to the body just like hey, answer these sort of problems. Um but so I wanted to ask in terms of like the mental the brain architecture that we would need to have consciousness and everything else. Um a lot of people when they talk about can something have consciousness they well no, it's just this as if we know all the parts that make up consciousness and the the brain they're saying well, we don't have all the parts so we just have one part. And that's true for like a car if they say that's not a car that's just a wheel. We can say that because it makes sense because we know what makes up a car, right? The engine, drivetrain, blah blah blah. We don't really know Well, and I guess I should ask you this. Do we even know the different sort of Lego pieces that we need to build a let's say a digital brain or even a mechanical brain that has, you know, intelligence and consciousness and can we replicate the human brain and all of its experiences or whatever you want to call them qualia, experiences, consciousness. Do we even know what pieces of the puzzle we need? So the short answer is no. The longer answer is there is a long tradition of psychologists and thinkers and uh cognitive scientists building architectures. So they're basically the architectures with boxes and arrows and different models. There are also mystical traditions and um meditation traditions and so on that basically try to point at the different parts. Some of that knowledge is lost in translation between the traditions because they have difficulty to ground their concepts in a shared context. So a lot of the stuff that is being talked about in Buddhism cannot be readily translated into Western philosophy of mind or neuroscience or psychology and vice versa. So one big difficulty is that we need to find a shared language. But what we also observe is that this uh tradition in psychology that was trying to translate its models into computer architectures, um the cognitive architectures uh community, they created things like a common model of of the mind in which they try to come up with the necessary components and so on. They do achieve a quite a degree of agreement on what such an architecture look like and which parts we probably need. But there is no implementation that works. And that's because there is a gap between the details that are necessary to make it work on a computer and the high-level ideas that currently exist. And if you want to program something, you cannot just postulate a few axioms and hope that these axioms are going to resolve themselves. Instead, you need to really tell the computer from first principles what to do and how to scale it up. So this is one perspective. The other one is instead of us trying to construct a solution, why don't we let the computer search for it? And ultimately, our own mind is just a search mechanism. And it's a search mechanism that is sometimes very smart and it's culturally informed and evolutionary informed and so on. So we can figure out solutions to pretty uh hairy problems by just talking about it uh for a few generations, writing books about it for a few generations, teaching university classes for a few generations, and slowly getting things to work. Right? But can't we automate this process? Can't we have the computer talk to itself and make uh billions of experiments that would take us uh centuries in a relatively short amount of time? And the uh machine learning researcher and um computer scientist um Sutton uh has uh taught us the bitter lesson that basically handcrafted solutions that humans come up with when they build algorithms and solutions tend to be eventually inferior to the stuff that we just let the computer search. Because when you are doing something and you when you are searching and you're performing experiments, ultimately, you can figure out the algorithm that you are following when you do this. And if you externalize this algorithm for an evolutionary procedure for a way in which you are coming up with new experiments, testing the experiments, evaluating the result, informing the next experiment, if you can express this as some kind of automated search process, then this thing that you automate is going to be better than than you are. Because it can do much more in short amount of time. So, we can just search for it. And so, the question would is what is the search space for consciousness? Can you um maybe dive just a little bit more into the speed of information processing? I know earlier in the conversation we sort of talked about how in some ways maybe the difference in consciousness between plants and animals might just be that the muscle movement needed to evolve faster and the neurons started firing faster and um now I am in this world where I'm seeing these AI agents um sort of open claw and and then tools like this out there talking to each other at incredible speeds. Um yeah, what what happens at that level of speed and um how can I envision the future going well? So, one thing that is apparent to me is that these systems are much faster at reasoning and more reliable potentially at reasoning than us even with this underlying structure is what the same that we have. But the perception is mostly still slower. But this is a question of how much we are willing to invest. But if you if you want to pay enough for quantum compute, you can build something that does video in real time in and out. And that does uh language in real time in and out. It's just nothing that you can do at $20 a month on a subscription. So, it's not an application that we're using all that much yet. But, we're getting there, right? We notice that the self-driving car is processing information at a very good clip. And in principle, um we can probably put the human brain on an H100 if we knew how. And in part, the issue is that the algorithms that we are at the moment using to um describe the transitions between states in a dynamic world are not as efficient as what the brain has discovered. There is an interesting question of how fast the brain really is. And sometimes people try to estimate this by looking at the uh computer that is necessary to recreate a single neuron, and then multiplying this the number of neurons. Or, when they think about how much memory does the brain have, they think about, "Oh, uh synapse, um if you want to recreate it, it has roughly four bits." And it's then you have so-and-so many synapses, and you multiply this, and you come up with a very large number. And I think this is the wrong way to think about it. Instead, uh you could also ask yourself, "How many brains would you need to run a Commodore 64?" And it would be pretty large number because our brain is very mushy. So, it's basically going to create so much noise that you would need to have many, many brains for every correction, and the result would be very, very slow. So, uh this is not a direct comparison because the hardware is so different. And so, a much better way of looking at uh the comparison would be the performance. And we can basically see how long does it take a human being to form a concept, and how long do we live? Then can we kind of estimate how many concepts are we able to form in our life? And we end up with a relatively depressing low number, but maybe a a few million tops. Right? And that is nothing for a computer. Of course, we could can make a system that is forming many more than a few million concepts if we want to. Because we can scale it up in in a way in which brains do not scale up. And so, um the speed of the brain might not be the issue. What we subjectively also see is that there is a limit in how fast we can process information. And that limit is roughly speed of sound. Which means if you are making clicking noises and they clicking noises have a frequency of something like 20 30 hertz, they become one pitch. And we can no longer hear the clicks separately. And that's because the signal processing of the brain is not faster than this. Cannot resolve signals at a higher rate. Kind of a few tricks that we multiplex things and so on and basically some kind of cascade, but by themselves the individual neurons don't like to fire at a much higher frequency than something like 20 30 hertz. And so, for the same reason when we look at videos, the frame rate at which we can distinguish movements and so on is relatively low. Of course, if you are a computer gamer, you realize that there are some tricks that your nervous system is using that give you a slight edge if you have a monitor with a even higher frame rate, but it's not like you can consciously see this or process this in a way that that you coherently interpret the signals. And so, our consciousness is actually operating at a very low frame rate. That's also why I'm skeptical of some of the ideas of people who say oh well, you need some kind of arcane quantum process that is working inside of the cells that is making them much much much faster than what you can currently envision to happening on the GPUs simply because I don't observe my brain having that kind of performance. Interesting. >> my current sense is the reason why our computers when you want to do real-time video generation and processing are still so expensive is because we don't have the right algorithms yet. And also, they already can do it. They're already roughly there. They're getting closer. Absolutely. Um so and I just wanted to touch back to kind of you you mentioned in the in the previous answer like in the space of search for consciousness. Better lesson I always see it as you know all the sci-fi books I read as a kid always perceived artificial intelligence as something that we've engineered and coded. It's either like logic or it's like very it's engineered. Whereas now it's more like we're kind of growing it, right? We're not We don't fully understand all the layers necessarily. Anthropic is doing a lot of research into it. They found that when they switch certain neural they turn up certain neurons in Claude, the AI Claude brain, it's has something that could be called some sort of a self introspection. Like it it it you know they'll turn up the portion of it that thinks about bread and they ask them about something unrelated and it brings or Golden Claude is is the one that everybody knows. But it it's able to notice it. It's like, "Hmm, why did I think about Golden the Golden Gate Bridge?" So there's some sort of a What is that? Self-awareness? Whatever you want to call it. Um so when we're approaching building consciousness, right? We might see something in our own brain like the default mode network or whatever that's like, "Oh, that's self-referential. Maybe we need to recreate it." But it feels like the better lesson saying, "No, it seems to be able to like if we just get it to do certain goals, it might emerge." >> [sighs and gasps] >> Um so is that kind of your approach to this? Like we need to figure out like the search space of consciousness and intelligence. Is it Is it something that's going to be engineered or is it something that's going to emerge out of some system we create? Well, some of the concepts I don't really know how significant they are. Imagine that the discussion that existed around mirror neurons. Um mirror neurons are a concept that was discovered when um people in uh Mike Gazzaniga's team uh looked at macaque monkeys and they put electrodes in their brains and they discovered that when the macaque monkey is observing somebody performing an action and when they're planning the action themselves, same regions become active in the brain. And I think they later regretted calling this mirror neurons because it led to a bunch of people thinking that these are very special neurons, like a particular spice in the soup of the brain. Mhm. And if that spice is absent, then you have autism and cannot recognize the actions of others and if you just add these neurons artificially, maybe you gain this ability. This is not the right way to think about it. Instead, there is a difference between a gesture and an action, right? The gesture is the movement that you make to perform the actions, like picking up a sugar cube from the table is an action, but you can do this in many different ways. Maybe you can do this with your hand, maybe you can do it with a spoon. And these are ultimately the same action and they're also the same action whether you do it or somebody else does it because the action itself is a certain kind of generality of abstraction that is necessary for understanding the world. And so you would expect that do not just have a representation that got turns on, becomes active in your brain when you are picking up the sugar cube with your fingers, but also this more abstract thing that is true over all those different instances, whether you do it or somebody else does it. So this would be something that happens in the pre-motor area and you would see this activation, but it's simply an action concept neuron. And the action concept neuron doesn't look differently from other neurons. It's just is the neuron that is at the right spot in the architecture where you make that representation. And now think about the default mode network. If you put somebody in a brain scanner and you tell them, "Please don't think about anything in particular and just chill." Are you going to see some kind of activity? It actually ramps up, yeah. >> going to be somewhat similar across individuals, probably, right? Is this a particular kind of very complicated network that you need to recreate if you want to build a the that is able to just sit there and chill. Right. No, it's just what you expect. It's the baseline. You would expect that there is some kind of baseline activity that is indicative of uh what you normally do when you don't do anything special. Right. Right. Right. >> Right. So, uh of course, I'm not an expert in here. Maybe there is something very unique and important about the default mode network that we need to understand and recreate. But by itself, I I would not think that we have to understand. What I would expect is if you build a machine that is leading to a certain degree of um resolving the environment in itself, and you ask it to idle, there is going to be something equivalent to a default network mode network in this machine if you set up the hardware in a similar way. Right. So, it's nothing that you need to build into the machine in the same way as you don't need to build action concept neurons into the machine to make it understand actions. >> Mhm. You just need to set it up in such a way that it can learn or dynamically model reality in itself and it and come to plans and actions when necessary. And uh so, this might be the wrong way to look at it, and I think it's maybe an artifact of how neuroscience works. It's largely descriptive instead of >> Right. theoretical. Right. >> And if you are an engineer, you think about a theory of what you need to do to make a certain thing happen. And if I was an evolutionary process that wanted to create an organism that can walk and talk, what do I need to do? And then I can look for the specific parts. But neuroscience takes the opposite approach. It basically looks at what is there that I can measure. And the things that I can measure, how do they correlate them to the observable behavior? Mhm. Mhm. So, it's a very different direction that neuroscience is taking there. And uh personally, I I like this approach of the engineer better because it is restricting the design space in a way in which the uh observing sciences cannot. And the reason why the psychology has given up on the psyche is because they tried to become more rigorous physics-like science and so they think about falsifiable theories. How does an experiment look like in which you are fitting a curve to the data and it's reliable and don't have too many free parameters? So, your experiment is not tightly bound to the outcome. But, it's like we need to simplify the task. You need to simplify the task until you only have two or three free parameters. But, if you have only two or three free parameters, you don't have a very interesting behavior anymore. And so, you give up on looking at the thing that makes the mind so really interesting. And if you are a computer scientist, you have to take a different perspective. Instead of thinking, how can I simplify my program in such a way that it only has three variables? You think about what is the simplest program I can come up with that does the thing that I need to be done. Right? And then, I can also think about the space of possible programs that can do the thing that can be done. And this engineering perspective constrains your search space in ways in which the purely observational method cannot. That's why I went into artificial intelligence to understand the mind because it allows me to take this engineering stance. Yeah. Um okay, let me switch topics just a little bit. Like, um let's talk for a second sort of about life lessons. Like, it is much as you get technical about things. I've listened to a number of your podcasts in the past and um read your blogs and every once in a while these little life lessons come up. And one of them that sort of resonated with me I wanted to share and maybe have you elaborate on. And one at one point you talk about um sort of noticing your own thoughts, a sort of metacognition, sort of stepping back from a moment and and thinking about your own thinking. And that led me to think that being sort of stoic is a very powerful way to kind of live life, to kind of not be so caught up in moments all the time. But, it came at the cost of meaning. The more stoic I felt like I could become, the less I felt like joyful or in moments. And then I heard you say one line and it almost was probably just a throwaway line that you were just talking through where you described meaning and I interpreted almost like the meaning of life as something that we create by making the choice about what to suffer for. Um maybe you could tell me if I interpreted that correctly and if you wanted to expand on it. Like is is that one of the powerful ways to have a good life is to sort of be stoic, be metacognitive, and then choose a few things to to suffer through to make life worth it? I don't really have life lessons for anyone. Um I don't enjoy my life all that much, but they also don't think it's all that important. I think that ultimately our suffering uh I It pains me if somebody else suffers. But in the grand scheme of things, the stuff that I'm suffering for, whether they are worth it or not, is not up for me to decide. But sometimes the suffering that you as an individual have to endure turns out to be important for something larger. And sometimes life just sucks. Maybe you were born in the middle of a famine or of a war and uh this is it and you don't have much of a choice about it. And so I think this obsession to maximize the score in this direction to say I should basically die with a minimum of suffering in my life uh under my belt so I had the best possible experience. It's pointless. It doesn't really matter. And this is not a global perspective. It's not a normative stance that applies to everybody else. It's one of the positions in the diversity of identities. I also observe that I do not have a single identity. And it's not necessary to have a single identity. I need to be identified to get things done. For instance, to be a parent that changes the diapers of the baby, I need to identify with that role, right? It's really really important that this diaper gets changed. And whether I suffer while doing it or not doesn't matter at all, right? And even if there less pleasant things than changing a diaper, if you break your arm and you still need to get your child to the doctor or whatever, who cares? It just needs to be done. And you do it because you identified with this particular thing. And we have different identities that lead to different identifications. You could resolve the suffering that you have in your relationship by just stopping to identify as a lover, as a member of that relationship. And a lot of people that basically went for enlightenment give up on their relationships as a result because they realize oh, I don't need to engage with this. I don't need to be that. Instead, I can be this. I can be here in this moment, in this here, in this now, and this is good. And then you don't need to worry about anything outside of that state. And maybe this is the right decisions for them, who's to say? It's very difficult to understand what your place in the greater scheme of things is and whether the greater scheme of things actually matters. Whether that's also a choice. I observe that there are consequences of those choices. When you decide to serve a shared system of meaning together with others, it gets you in contact with these others and it can create them interesting life experiences and it can change things in the world. And those agents that participate in those games are going to be the ones that move the world in this particular aspect. And this is just what happens. And you're either going to be one of them or not and it's not even your choice who you're going to be. But you are just a substrate for all these ideas that play out on you, all these identities that play out on you. And so we have these identities as the curious podcaster, as the thinker, as the person who drives their car to work, as somebody who makes a cup of coffee, as somebody who is a parent or a lover or a friend or none of these things. And we go through these identities, and they are interconnected to some degree, but they don't need to be the same. There is not a single solution. So, instead, I think it's sometimes better to be able also to take this observant stance and to accept the fact that we are not in charge. That things are just happening. And even our urge to act, and us observing that we are giving into this urge and are really doing things and it's really important as is also something that's just happening. That's fascinating. And actually, towards the end of your We have a few minutes. I just wanted to ask some rapid-fire questions that are a little bit off-topic about exactly what you're talking about because that's maybe not quite AI, but wow, is it fascinating. Um one of the things that I wanted to ask is you know, a lot of the stuff that we're um learning about AI, how to build AI, a lot of it came or was sort of modeled after the human brain, you know, neural networks is is is similar to that. So, it's definitely safe to say that at least some of our understanding of AI came from our understanding of the human brain. Um but also, I do I always felt like in the future, and I I think you said something similar just just before this, that in the future, we might learn more about the human brain as we're building these sort of digital brains. Like that sort of branch of computer science, machine learning, whatever you want to call it, will teach us more about ourselves. Um so, I'm wondering number one if you agree with that, and number two, if we in the next how many years figure out how to build a complete replica of the human experience, the human consciousness, human brain, et cetera, how What should people think about that? What does that mean about us and our role in the universe? Does that make us less special? Like how how do we begin to process that? >> [snorts] >> It's uh it has to do with how our possible identities are evolving. Uh one of the really fascinating prospects that exists is that we could become somewhat independent of our own substrate. Imagine we figure out the code at which our brain talks to itself. And this code is very different from the ones that we are currently using in our GPUs. And so it's a very different language that neurons are using to talk to each other. But it's a language that we can sometimes learn across individuals. For instance, when you have very strong perceptual empathy with someone, you can build feedback loops through their own mind without having a direct brain-to-brain interface. And you're using your body as this interface. And using all the sensory modalities that are available to this body to get into resonance. And this means that you can get quite tightly interconnected. Imagine you build a machine that can do the same thing. And it can do it at a higher resolution than a human brain, body, and nervous system can. And you to do this in such a way that the computer becomes an extension of you. And so you don't give it a motivation of its own. Instead, it is something that is anticipating what you want, what you feel, what you think about. And so you observe that you see on your screen something that you would begin to think about. And it takes on form that is more detailed than what you can hold in your own mind, but it's completely controlled by you. And more and more of you is happening outside of this biological part. And you think of this part not as a machine that is next to you, but it's actually you. It's actually your mind. It's your mind that has become clearer and more detailed when you have this environment available. And at some point this environment might become dominant in the way that you say, uh, actually the bio part that I have is relatively inconsequential compared to this vast or universe of thought, comprehension, experience, and consciousness that I have available when I'm not just bound to this body. It's very hard to imagine this for most people because it means that you uh have something that's much larger and deeper and profound than your present experience. It's not an impoverishment. I mean, typically think of the computer at this tiny screen that sucks us in and gets us away from the real world of experience, which is very much true for the present systems. But, I think this vision is uh very much possible. And whether it's desirable or not is a completely different question, but this idea that there is an avenue for us in which we can move beyond biology and the thing that is there by biology is not just our distant offspring, but it's actually us is very fascinating. But, if you want to uh settle in the stars, if you want to leave this planet, we probably cannot really do this with biological bodies. It's way too boring to travel between the stars. And there's too much radiation going on and our organisms are really adapted to have a relatively short lifespans and mutating uh between these lifespans. So, we can adapt to changing circumstances by uh generational changes. Right? It's a very specific mode in which we exist as biological entities. And when we want to leave this planet, when we want to leave the circumstances in which we currently exist, and it's not a scope that I'm talking about over the next few hundred years, but some scope of what does intelligent life on Earth look like in 10 million years from now? Right? That is a very interesting perspective to say minds are basically not bound to a body. They're actually spirits that can exist over a very large set of substrates. And the most interesting substrates are no longer biological. Yeah. Um okay. So, if reality is like fundamentally this construction uh like some sort of cognitive model of what's out there, almost like a compression or a simplification or an algorithm of it, like um what would it mean to encounter something that was like completely resistant to modeling? Like could there be any kind of kind of consciousness with no concept of time or anything that's outside modeling? How can you interact with it? So the way in which we currently interact with reality is that we observe information which is discernable differences. And the meaning of that information is the relationships we discover to other changes in information. To other differences. Right? So you'd see pattern on your retina, this flip on your retina, the meaning of it is the relationship you discover to other flips on your retina. And they together form this dynamic evolving model of reality. And without this temporal and spatial succession, without this ordering, there would be no way to address these flips even and bring them into an order and to interpret them. There would also be no way for them to causally interact with the reality. Basically causal structure of reality is exactly that stream of changes in information. So state transitions. And so when we observe things, when we conceptualize things, when we think about things, we are always confined to things that can ultimately be modeled by something. You could also argue that on a deeper level for something in the universe to persist, to change things, to to exist, to build a structure, it needs to model what other parts of the universe are going to do. To basically be a particle, there is something like a subparticle like a quark for instance that cannot exist by itself. That needs to interact with the other quarks in such a way that a stable particle emerges. Right? And that means it needs to rely on these other parts being in the right place at the right time. You could also if you zoom out think of a water vortex. For a water vortex to exist, there it needs to be a reliance of water molecules on other water molecules doing a certain thing. Right, without these other water molecules being there, the water vortex would would be unstable. For your mind to be stable, then parts of your mind need to rely on other parts of your mind to be stable and modeling each other and being in a particular kind of interaction with each other. So, just the nature of existence requires that things can be modeled. We can come up with this abstract idea of something that cannot be modeled, but this idea is more like the idea of a square circle. Right, we can construct this, but it's meaningless. Because it is something that is can by itself not exist for logical reasons. And that's my issue that I have with most of the objections against computationalist functionalism is that they misunderstand what it actually means to contradict them. Computationalist functionalism is not simply the position that computers can think. We don't know whether our computers can think and it's an experimental problem whether they're fast enough and detailed enough and so on or whether we are missing something. It's more a position about metaphysics and epistemology. It says that when we are describing the world, we are describing how it transitions between states and the states are characterized by discernable differences. How can you make an alternative to that, right? How do you get out of this? And functionalism means that we describe objects as an evolution of states. And how does the evolution of states in the universe change when the object is present or absent? Right, this is what we mean by a function. And of course we can construct objects that don't conform to this. We can talk about unicorns that don't interact with anything. But to claim that these objects are real, that we should defer to them in any way because they're simply because we can string the words together is pointless. Right? If If we talk about an object that we claim to exist like a photon and so on, and uh the photon is exactly what we choose to name a photon. And this means it's a particular behavior of the universe under certain circumstances. And to say that a photon is anything but that that we constructed, that there's a hidden essence to the photon somehow, is meaningless because nobody has uh ever interacted with the essence of a photon. They only interact with aspects of the physical reality in over which we construct invariances and then we give them names. Well, um like why something not nothing though? Like why why is everything relative to everything else and like there's no universal thing everything's relative to and the geometry somehow ties together so strangely? Like why isn't there just nothing out there? Yeah, so the easiest answer is because existence is possible. And if existence is possible and there's nothing there to prevent it, then there is going to be an a branch in which the possible things are happening. And uh in the branch where they are not happening, there is no observer that complains. The observers necessarily find themselves in one of the branches where stuff is happening. And so we will always be confronted with a universe that exists. Right. So the really confusing thing is to make the switch from the expectation that uh nothingness is the default state somehow. And instead uh maybe undecidedness is the default state. And so if the universe is undecided about its own existence and uh it is we are base reality, which means there is nothing that makes a decision about reality, then it reality is going to have an existing and a non-existing branch and we will be in the existing one. That is my current somewhat tentative answer. I haven't found a better one so far. Don't think it's authoritative. Interesting, but yeah, certainly certainly makes sense. Like we're seeing something because there's something to see. If there was nothing to see, we wouldn't be discussing it on a podcast, I guess. That's one way to So, one interesting thing Oh boy, so I have the two two things I got to pick one. I guess let's go with this idea that Maybe we'll come back to that one. So, I mean, we have the Fermi paradox. So, why aren't we seeing aliens? And as you were talking about your previous idea, we were saying maybe at some point we develop kind of a external cognition and perhaps even we're able to travel to other planets, not in our physical bodies, but some sort of a representation of ourselves, our minds, so that we're able to experience things. There's lots of ideas now talking about full dive virtual reality. So, the idea that you plug into some virtual reality that is completely indistinguishable from reality. You're able to live out your life and maybe not have suffering. We've we've kind of talked about those themes. One thing that popped into my brain could I thought at some point maybe that's one of the possible explanations for the Fermi paradox. They reach some point and they just plug in and check out or whatever. But I know you maybe have a different take on it. So, would just love for you to unpack that Fermi paradox and ain't full dive VR however that makes sense to you. Yeah, number of possible answers to the Fermi paradox. So, the most popular ones I know is First of all, abiogenesis is rare. It's very rare for cells to form on a planet, for life to emerge. And maybe you need to roll the cosmic dice very often. And so, maybe there are just not enough places where life has emerged. Another possibility is that um we are early. Another possibility is that um the dark forest theory. She's in book that describes the idea that which makes total sense that there is always a shortage of good waterfront real estate and if you announce you have one and somebody else is going to move in and is going to displace you. And so all the smart critters in the universe have figured this out except for some hippies in the US who you have decided to loudly announce that we exist and somebody should visit us. Because they don't have the necessary self-awareness to realize how this typically ends when you announce that you have a nice habitable planet or a nice country and so on but no defenses yet and no technology yet. So this is the dark forest theory basically says the reason why we don't observe anybody is because the others are smart and they don't announce that they exist. Alister Reynolds has his own version of the dark forest idea which is basically dark forest idea on steroids. The idea is roughly that this life emerged relatively early in the galaxy and there was a dawn war which was decided by a machine species in its own favor and this machine species has decided to limit the population in the galaxy or the emergence of space bearing civilizations to basically preserve the free energy for a future time and they basically look in the darkness between the stars and whenever a civilization becomes space bearing they get eradicated. This is the inhibitor theory. So there's basically an active force in the universe that is inhibiting space bearing civilizations to preserve resources. Yeah. The other variant is for instance the perspective that Stanisław Lem is taking which is general pessimism about the long-term viability of civilizations and their willingness to talk to each other. So the idea is that civilizations once they become intelligent only have a relatively short window in which they can talk to each other because otherwise they either collapse or they ascend into a form that makes them uninterested in talking to others. And then of course there is the idea no, actually there is no Fermi paradox because there are signs of intelligent life you're just not looking. Mhm. Mhm. What what about the one that they're that it might kind of already be here we just haven't tapped into the right physics of it like something quantum would give us access to a communication channel that we haven't discovered yet? That would be identical to it not being here yet. Right, so of course if it is already interacting with us and so on it doesn't want to be seen that is slightly different. In this case we are interacting with it but Trump is still not telling us. It's a secret. >> Yeah. So the basically it comes down to the question of whether faster than light travel is possible. And if it's not then we probably would only see some robotic probes. And if or we would see stuff from the same solar system and there are not that many candidates in the solar system where intelligent life could still exist. And so Venus doesn't seem to be very habitable anymore. Mars is not habitable anymore. Maybe some moons around the gas giants could be candidates but they also don't really look very populated from very far. So we would not expect that anything is very far ahead of us. So maybe the stuff that you are hearing about aliens visiting us and what all this are simply the government messing with us to calibrate how much cesium they need to put into the water supply to keep psychosis in check. Interesting. That's Um one quote that I have here relating to this I think you said in one of the podcasts is perhaps it's because these civilizations they hate their Elon Musks too much. I'm just curious if that Can you expand on that? That was kind of an interesting quote. Yeah, it's fascinating that we do have this thing that people apparently rather die than having Elon Musk making us multi-planetary or shooting down the asteroid that is coming down. I think this movie Don't Look Up very fascinating and as a cultural artifact because it is not just a movie about some hubristic tech people trying to ignore the meteor in the wrong way while as the good peer-reviewed scientists are trying to do the right thing, but unfortunately because of the horrible way in which the world set up cannot prevail and ultimately the Elon Musk mess up and the planet gets destroyed. It's also a story about how these people see the world. Right? And how they perceive their own role in it. And the thing is that as a civilization we seem to be increasingly unable to be effective. Once systems become very large, they sort of calcify and become post-modernist which means the story that we tell ourselves about what the system is doing becomes far less important than how well the system is able to deal with the ground truth. The interesting thing about Elon's strategy is not so much that only that he is the most successful industry leader of our time, but how big the delta is between what he achieves and what the rest achieves. And when he started with electric cars, the car industry had decided to wait for a few more decades until they have electric cars. And most of the funding went especially in Germany where I looked at this at the time into hydrogen. The reason why we invested into hydrogen is not because it's a very good technology, but because it allows you to keep the gas stations intact. We can basically keep almost everything as it is and it's horrible to have gas station and tank trucks and to have things explode because hydrogen trucks have an accident and whatever. It's much better to have cables, right? And but this would have been too disruptive. And our society did not have the necessary decision-making power to say, "Let's do something that makes a few people in gas stations to transition and have this disruptive thing." You need to have this guy who was disruptive. And what is really really apparent is with respect to rocket launches, Elon has reduced the cost of the rocket launch to like a tenth of what it costs for NASA. And what this is saying something is not just about Elon's genius. It mostly says something about NASA. Right? NASA is the shiniest institution that we had to build rockets. And this thing was unable to move beyond what we had in the '60s and '70s. So, it's largely this postmodernist scam where people get jobs and redistribute resources among each other, but they're no longer making progress. And the question is, how can we as a civilization actually still make progress instead of just telling ourselves stories and creating employment programs? And so basically Elon Musk has revealed that NASA had become an employment program rather than being a space agency. And that's basically the scandal that we need to reckon with. And because that's very uncomfortable, I think there is a lot of pushback. There's also a lot of other pushback. It's largely fueled by the fact that Elon is a very big 12-year-old that is having strong opinions on social media. But it's also this thing that this guy is antagonizing everybody, that he is winning despite being wrong about everything other people, reasonable people agree on. Right? This idea that you cannot just build a rocket. Yeah, well one of the things that always seems to pop up in my head when I think about the short-term, mid-term, and long-term future is just the disruptive change that we have to go through. So, um when you see where the world's going, like what's been achieved especially in AI in the last few years, I mean, do you start thinking to yourself like, "Oh, I I might not need to die of natural causes." Like, do you see yourself being alive in 200, 500 years, sort of evolving into this kind of hybrid machine bio thing, and then uh what other kind of incredibly disruptive thoughts actually seem probable that you're kind of grappling with? Um I think in my present form, I'm not suitable for living for hundreds of years. Let's see, my body and my mind are falling apart at a rate that uh is not compatible with immortality. And I wonder if I want to transition my body into something that is uh immortal and unchanging, how long it would be viable, and if this is the ideal body to have. It's also I feel that my own role is is not to be around forever. It's uh I need to get the kids out of the house. I need to uh get a few ideas started that people use these days and need. But I I don't enjoy being here. I experience this world as pretty horrible. I would would prefer to go home to my own planet. And uh I don't see this planet in the near-term trajectory. Maybe at some point I'm going to work through my existential issues, and I'll be happy to exist. But at the moment, I would prefer if I just disperse at some point. I think that existence is instrumental to getting things done. And I'm here because I have a job to do. And whether I like doing this job or not doesn't really matter. But it's instrumental to this, and at some point I'll be done. One thing um I'm just very curious about, and maybe it's personal, please we can cut this part out. Um so, how I'm interpreting it it is, you know, we talked about the idea of like suffering, happiness versus not happiness. In Dario Amodei's Machines of Loving Grace, he's talking that potentially in the future AI might be able to engineer some things to like if there's something in the brain that we would like to work differently, um we can maybe tweak it, right? Because everybody's brains misbehave in certain ways. We've kind of talked about this idea of our bodies like just you know, there's a conflict of goals or anything else like that. Do you think that so maybe this idea of being stuck around forever without being more in control of how our biology works. Yes, I could certainly see how that might be not positive, but what if you could change how happy you are, how motivated like like for me I struggle with forgetting things and being you know, focused. If I could change that, I'd certainly feel like I would be happier. Can you talk about that a little bit? Does that make sense? >> Yeah, think about it like a computer game. >> [snorts] >> Imagine that the universe is something like a computer game. That might sound cynical because we have deeper meanings and we have large-scale jobs to do, but it just means it's a very big computer game. At some point the heat death of the universe is going to come and the game is going to end. And so there is some kind of limit even if it's billions of years into the future. And now imagine somebody would offer you and look here is this amazing game. It's GTA 9. It's going to be more awesome than even GTA 8 that is not announced yet. And do you want to have this game? And if so, you need to sign here, but well also you need to be want to be in this game forever. You can never log out. Uh This this sounds horrible, right? I would never take that, yeah. Right? And so in some sense this is this kind of game that we are in. And it's interesting that in most of the Eastern religions the goal is to get out. Mhm, mhm. At some point you need to find a way to get out of this game. Yeah, yeah. >> That's also because uh they see the game as periodic. It's something that is going to repeat itself in cycles. Civilizations rise and fall. Climates come and go. Population grows and falls again, right? But ultimately, it's the same game and it's it mostly sucks. And so, in in their perspective, in their Buddhist perspective and so on, the idea is really to to get out of this wheel at some point. And the goal is sweet oblivion. It's not to get into another game that is like this just with different colors and different foods and different crystals and different recipe gas and different cars. >> Yeah. Right? And we have this perspective in the Abrahamic world of a linearly evolving world that is going like this and it's never stopping like this. It's always going to get better and better. And it's basically equal of the way in which our civilization has evolved during the last couple thousand years. And especially during the last few hundred years. And it's a very disconcerting thing that we have this impression that at some point we lack the imagination of how it's still going to get better and better. And this creates some kind of disturbance in the force that we don't really imagine what our children going to be living like in the thousand years from now because we just cannot extrapolate it into a beautiful future anymore. It's much easier to come up with dystopias than with utopias. But this idea that okay, what kind of feature would you need to get? What kind of extra weapon would you need to get? What kind of super skill and level would you need to get until you're willing to sign on the line and say you're going to stay in the game forever. Right? This is short-sighted. So, of course, I can come up with a story in which the game is going to be super exciting for the next thousand years or so. But uh the idea that this should go on for eternity. But what do what do you imagine that you a happy eternity is going to look like? Is it going to be a loop? How long is the loop going to be? Yeah, I mean any any deal that I personally would be proposed like God-like powers just anything that you think of if the flip side of that is I can't log out >> exit ever. If I can't ever exit and find out will get old, right? And anyone who thinks that What will get old is if we make you stupid so you forget. Yep. Yeah. And and that's also from the outside it it doesn't look like an attractive option that you lobotomize yourself into being okay with existing forever. Right. Groundhog Day for for eternity. Just reliving the same. Yeah. Pretty depressing. >> And forgetting forgetting each day of prior. That's why I also think that suffering doesn't matter for the same reason that happiness doesn't matter, right? If happiness is just a brain state that you can induce in your brain and you manage to design the optimal loop and you're going to stay in this loop for the next few thousand years and it's designed in such a way that you forget all the things that would make you unhappy if you knew them. And it's a pretty depressing perspective. Even if it's objectively is okay. Right? And so I think it makes much more sense to say that our existence is instrumental to something larger and what what it is is there's basically a mathematical possibility of self-organization in the universe that leads to intelligent life that coheres into larger and larger structure. And in that we play a role. Right? We basically are a small part in the bootstrapping of a God-like mind that is emerging over the possibility of self-organization in the universe and then from this perspective consciousness is just getting started and we are part of this genesis and it's a very exciting perspective but it's not about us as humans and it's not about our suffering, it's not about our happiness. Well, I guess one thing cuz I know you're working on something that's semi new and looking into consciousness machine consciousness or however you want to describe that. So, maybe if we can briefly touch on that, just what do you hope to achieve? Um and where do you think this progress is going just in the the next 5 10 years cuz some people are saying we're going to hit this exponential and intelligence explosion. Um what's your guess for the what the future holds? I would like to understand how consciousness works and I think the best way is to take an earnest shot at trying to wake up the machines. Not in the sense that we large scale build self-improving systems and deploy them. But that we very safely in a very narrow environment conduct experiments that can show us can we achieve this kind of phase transition that happens in the brain of a mouse when it wakes up. Can we do this in an artificial environment? And I would like to understand this shape of the strange loop better. I want to understand who we are and how we work and how we interact. I think one of the most interesting applications of such a research could be to design interfaces that allow us to become more conscious. That allow us to interact more deeply with ourselves and reality. With respect to artificial intelligence, we really have to think about what is the kind of artificial intelligence we want to build for our children. Mhm. And the question is not one that is purely commercial or political or ideological. It's much deeper than this. It's a fundamentally existential and cultural question. And so, also part of my job is to help to for a consciousness culture to evolve in the AI industry and in the AI discourse. Right? At the moment, most of the discourse is about economic fears, political fears, and greed. And control and surveillance and very near-term things and about technology. But it's not so much about the cultural transition that we're going to observe when we become more hybrid and our identity is going to change and it's going to change in the same way as the identity of the religious mind is a different one that we have today in the secularized mind of the West. We're going to have to shift where we have new skills and new self and new shape of the self that is integrating artificial parts. And it we're no longer going to perceive them as artificial because they're going to be organic, they're going to be adaptive, they're going to grow and shape themselves in the same way as the biological systems are. They just use non-biological substrates. And I don't know what the best way to do this is and I don't presume to know. But I want to help creating a space in which we can have that discussion. Very interesting. And just so I can briefly get my mind around you said so expanding consciousness. So would for example when you meditate and you become more aware of your who you are the that observer is that an example of expanding consciousness or is there a different way to think about what that means? Yeah, so first of all consciousness it seems to me almost binary to me that you're either awake or you are not. You're it's all the contents can be very different. And so the main difference is in lucidity. How well do you understand what you are and how expansive is your understanding, how much can you hold awake in your own mind? Can you create an environment in which we keep each other awake? And this basically this wakefulness is something that I personally strive for and strive for in my relationships to be blessed to be in environment with people that are aiming to keep each other awake. To get to greater lucidity. And ultimately that's also a way which we deal with our suffering, with our existential issues, is that we get an understanding in which we can become aware of our identities, and these identities are something that we get to choose and construct and use in the best possible ways. Oh. Okay, so for the final question, I just wanted to have you kind of launch me and and any of the listeners into other things we should be reading about, um, learning from, like I'm just sort of curious. I mean, you've mentioned Michael Levine a couple times, but like what are some of the names that you just kind of wish were out there more, books that you wish people read, fiction or non-fiction, media? At the moment, we are uh rereading Kant a little bit. Uh, I believe Kant is a very interesting cognitive scientist who have been mostly not being read in the context of cognitive science. He's really asking, "How do minds operate? What are the functions that they adhere to?" And his work Uh, I've reread a lot of Aristotle recently. I don't think it's necessary to do so because he is basically just a modern thinker. For me, the interesting insight was how modern his thinking is. And that he is mostly seen as uh this uh patron saint of a religious tradition that has dogmatized him instead of seeing him as this living thinker who's just asking questions. And this this mode of asking questions in the Anima and so on is quite interesting and to relate this to our modern thinking that allowed me to connect the two dots. Um, I uh also read a lot of sci-fi. I feel as though that most of the interesting philosophy of mind happens in science fiction. Some of my favorite authors in this realm are Greg Egan, books like Diaspora, Closer, the entire Dory stories and so on that he's written are very interesting. Uh, essay collections or short-story collections like um Axiomatic. And it's mostly mathematical uh miniatures that are not very plot-driven and story-driven, but that expand on ideas that his mind has, and he's often very whimsical. Ted Chiang has written a number of beautiful things. Uh especially the uh short-story collection Exhalation is very good. One of the most interesting writers uh at the moment is um Peter Watts. What started is Blindsight, followed by Echopraxia. It's uh one of the best explorations of the space of possible minds right now. It's even better than the Vernor Vinge books, uh A Deepness in the Sky uh and A Fire Upon the Deep, which are nice space opera. But uh Peter Watts is much more dense and uh less about uh the baroque stories and much more about uh getting into the nitty-gritty of how these minds operate, and just beautiful ideas. There's also other stories that he's written, the Rifters Trilogy and the Freeze Frame Revolution, which are very good. Um I recently discovered Alister Reynolds. Alister Reynolds is very baroque. It's really widescreen baroque, big-screen space opera. Everything is beautiful. Stunning uh planets, amazing larger-than-life characters. And uh I I do enjoy the beauty. It's very cinematic. And uh he is mostly looking at long timescales. Somewhat similar to Dune, even longer timescales, but it takes much more literary essence, and the density is not as high as it is in Peter Watts, but he does have some interesting ideas about consciousness. But all his stories are very much um narrowed down to the human experience. So, he does not leave this human realm very much. So, the stories are much more relatable in a way, but they're also limited in in the types of minds that they explore. Yeah, I I will say interesting. Don't Yeah, don't read Blindsight at night time. I feel like that that that book and the video game Soma were the two that probably like disoriented my sense of self more than anything else I've I've ever experienced, so it's a very hard I don't know. If you if you if you really read that book, it'll like shake you up, man. But yeah, all that stuff sounds so great. Tons of stuff there I hadn't read before and I'm excited to read them. >> great list. Thank you so much. Yeah, Peter Watts uh I definitely sounds very interesting. I'll definitely look into that. Um and and the the other things as well. So, thank you so much. Thank you so much for first of all, your work in general. Thank you so much for giving us just you know, some of your time. I know I know you have a lot you're a very very busy person, so this was absolutely incredible. I hope everybody that watched this um gained something from it. I think this is a very deep discussion and there's just so many more things where we could take it, but uh for the time this is this has been an absolute honor and joy and and thank you so much for being here. So, invest. Thank you. Yeah, it was a great pleasure. Thank you for this wonderful conversation.