Fin Moorhouse - Longtermism, Space, & Entrepreneurship

Key Takeaways

the scope for like what we could achieve is like really extraordinarily large like maybe kind of larger than most people kind of typically entertain them having this kind of property of being like anti-fragile with respect to being wrong like really celebrating and endorsing changing your mind in a kind of loud and public way but you can also do a thing which is try to make a lot of money and just you know make a useful product and then use the success of that first thing to then just think squarely like how do i just do the most good if you have like 10 million in the bank and you make another 10 million does your life get twice as good obviously not right if on the other hand you just care about like making the world go well then the world's an extremely big place and so you basically don't run into these diminishing returns like at all there's this question of like if the many worlds view is true what if anything could that mean with respect to questions about like what should we do or what's important [Music] today i have the pleasure of interviewing finn morehouse who is a research scholar at the oxford university's future of humanity institute and he's also an assistant to toby ort and also the host of the hear this idea podcast um finn i know you've got a ton of other projects under a belt so do you want to talk about all the different things you're working on and how you got into ea and this kind of research i think you nailed the broad strokes there i think yeah i've kind of failed to specialize in a particular thing and so i found myself just dabbling in projects that seem interesting to me trying to help get some projects off the ground and just doing research on you know things which seem maybe underrated i probably won't bore you with the list of things and then yeah how do i get into ea actually also a fairly boring story unfortunately i really loved philosophy i really loved kind of pestering people by asking them all these questions you know why are you not why are you still eating meat red's kind of peter singer and will mccaskill and i realized i just wasn't actually living things like these things out myself i think there are some just like force of consistency that pushed me into really getting involved i think the second piece was just the people um i was lucky enough to have this student group where i went to university and i think there's some dynamic of realizing that this isn't just a kind of free-floating set of ideas but there's also just like a community of people i like really get on with and have all these like incredibly interesting kind of personalities and interests so um those two things i think yeah and then so what was the process like uh i know a lot of people who are vaguely interested in ea but not a lot of them then uh very quickly transition to you know working on research with top va researchers so uh yeah just walk me through how you ended up where you are yeah i think i got lucky with the timing of the pandemic which is not something i suppose many people can say i did my degree i was quite unsure about what i wanted to do there was some option of taking some kind of close to default path of maybe something like you know consulting or whatever and then i was kind of i guess forced into this natural break where i had time to step back and i you know i guess i was lucky enough that i could afford to kind of spend a few months just like figuring out what i wanted to do with my life and that space was enough to like maybe start like reading more about these ideas also to try kind of teaching myself skills i hadn't really tried yet so try to you know learn to code for a lot of this time and so on and then i just thought well i might as well wing it there are some things i can apply to i don't really rate my chances but the cost to applying to these things is so low it seems worth it and then um yeah i guess i got very lucky and here i am awesome okay so let's talk about one of these things you're working on which is that you're you've set up and are going to be helping judging these prizes about uh ea writing one is you're giving out uh five prizes for a hundred thousand dollars each for um blogs that discuss effective ultras related ideas another is uh five prizes of twenty thousand dollars each to criticize the ideas so talk more about these prizes why is why uh why is it now an important time to be uh talking about and criticizing ea that is a good question um i wanna say i reluctant to frame this as like me personally like okay yeah i certainly kind of helped help set up these um initiatives well so i heard i i heard on the inside that actually you've been uh you've been pulling a lot of the weight on these projects certainly yeah i've um find myself uh with the time to kind of like get these things over the line which i'm yeah i'm pretty happy with so yeah the criticism thing let's start with that i want to say something like in general being receptive to criticism is just like obviously really important and if as a movement you want to succeed where succeed means not just like achieve things in the world but also like end up having just close to correct beliefs as you can get and having this kind of property of being like anti-fragile with respect to being wrong like really celebrating and endorsing changing your mind in a kind of loud and public way that seems really important and so i know this is just like a kind of prima facie obvious case of wanting to incentivize criticism but you might also ask like why now there's a few things going on there one is i think the effective altruism movement overall has reached its place where it's actually beginning to do like a lot of really incredible things there's a lot of like funders now kind of excited to find find kind of fairly ambitious scalable projects and so it seems like if there's a kind of an inflection point you want to get the criticism out the door and you want to respond to it like earlier rather than later because you want to set the path in the right direction rather than a just course which is more expensive later on well mccaskill made this point uh a few months ago you can also point to this dynamic and some other social movements where the kind of really exciting beliefs are kind of have this like period of plasticity in the early days they kind of ossify and you end up with this like set of beliefs it's kind of like trendy um or socially rewarded to hold in some sense you feel like you need to hold certain beliefs in order to kind of get credit from you know certain people and the costs to like publicly questioning some practices or beliefs become too high and that is just like a failure mode and it seems like one of the more salient failure modes for a movement like this so it just seems really important to like be quite quite proactive about celebrating this dynamic where you notice you're doing something wrong and then you change track and then maybe that means shutting something down right you set our project the project seems really exciting you get some like feedback back from the world feedback looks more negative than you expected and so you stopped doing the project and in some important sense that is like a success like you did the correct thing right it's important to celebrate that so i think these are some of the things that go through my through my head just like framing criticism in like this kind of positive way yeah it seems pretty important right right i mean analogously it said that uh losses are as important as profit in terms of motivating uh economic incentives and it seems very similar here in a slack we were talking and you mentioned that uh maybe one of the reasons this is born now is if if a prize of twenty thousand dollars can help somebody help us figure out how to or not me i don't have the money but like help svf figure out how to uh how to better allocate like 10 million dollars that that's a steal it's really impressive that effective altruism is a movement that is willing to fund criticism of itself i i don't know is there any other example of a movement in history where that's been so interested in criticizing itself and becoming anti-fragile in this way i guess one thing i want to say is like the proof is in the pudding here like it's one thing to kind of make noises to the effect you're like interested in being criticized and i'm sure lots of movements make that another thing to like really follow through on them and you know is a fairly young movement so i guess tai will tell whether it really does that well i'm very hopeful i also want to say that this like particular prize is like you know one kind of part of a much um a much bigger thing hopefully that's a great question i actually don't know if i have good answers but that's not to say that they're right now i'm sure there are like political liberalism as a strand of thought like in political philosophy comes to mind as maybe an example one other random thing i want to point out or mention you mentioned prophets and just like doing the maths and what's the like ev of like investing in just red teaming an idea like shooting an idea down i think thinking about the difference between the for-profit and non-profit space is quite an interesting analogy here um you have this very obvious feedback mechanism in for-profit land which is you have an idea no matter how excited you are about the idea you can very quickly learn whether the world is uh as excited which is to say you can just fail and that's like a tight useful feedback loop to figure out what your whether what you're doing is um worth doing those feedback loops don't by default exist if you don't expect to get anything back when you're doing these projects and so that's like a reason to i want to implement those things like artificially like one way you can do this is with like charity evaluators which in some sense impose a kind of market-like mechanism where like now you have an incentive to actually be achieving the thing that you're like ostensibly setting out to achieve because there's this third actor that's more party that's kind of assessing whether you're you're getting it but i think that that framing i mean we can try saying say more about it but that's like a really useful framing i think to me anyway um and uh one other reason this seems important to me is if you have a movement that's about like 10 years old um like this you know we have we have like uh strains of ideas that are thousands of years old that have significant improvements made to them uh that that were that were missing before so just on that uh that alone it seems to me that the reason to expect some mistakes either at a sort of like theoretical level or in the applications that that does seem like i do have a strong prior that there are such mistakes that could be identified in a reasonable amount of time yeah i guess one framing that i like as well is not just thinking about here's a set of claims we have we want to like figure out what's wrong but some really good criticism can look like look you just missed this distinction which is like a really important distinction to make or you miss this like addition to this kind of naive like conceptual framework you're using and it's really important to make that addition a lot of people are like skeptical about progress in in kind of non-empirical fields so like philosophy for instance it's like oh we've been thinking about these questions for thousands of years but we're still kind of unsure and i think that miss is like a really important kind of progress which is something you might call like conceptual engineering or something which is finding these like really useful distinctions and then like building structures on top of them and so it's not like you're making claims which are necessarily true or false but there are other kinds of useful criticism which include just like getting your kind of models like more more useful speaking of just making progress on questions like these one thing that's really surprising to me and maybe this is just like my ignorance of the philosophical history here it's super surprising to me that the movement like long-term ism at least in its modern form it took thousands of years of philosophy before somebody had the idea that oh like the future could be really big therefore the future matters a lot um and so maybe you could say like oh you know there's been lots of movements in history that have emphasized i mean existential risk maybe wasn't a prominent thing to think about before nuclear weapons but that have emphasized that civilizational collapse is a very prominent factor that uh might be very bad for many centuries so we should try to make sure society's stable or something but do you have some sense of you have a philosophy background do you have some sense what is the philosophical background here and to the extent that these are relatively new ideas how did it take so long yeah that's like such a good question i think one name that comes to mind straight away is this historian called tom moynihan who so you wrote this book about something like the history of how people think about existential risk and then more recently he's been doing work on the question you are which is like what took people so long to reach this like what now seems like a fairly natural thought i think part of what's going on here is it it's really hard or easy i should say to underrate just how much i guess is somewhat related to what i mentioned in the last question just how much kind of conceptual apparatus we have going on there's like a bit like the water we swim in now and so it's hard to notice so one example that comes to mind is thinking about probability as this thing we can talk formally about this is like a shockingly new uh thought also the idea that human history might end and furthermore that that might be within our control that is to decide or to prevent that happening prematurely these are all like really surprisingly new thoughts i think it just like requires a lot of imagination and effort to put your self into the shoes of people living earlier on who just didn't have the kind of yeah like i said the kind of tools for thinking that make these ideas pop out much more naturally and of course as soon as those tools are in place then the like conclusions fall out pretty quickly but it's not easy and i agree that i appreciate that actually wasn't a very good answer um just because it's such a hard question um yeah so you know what's interesting is that more recently um maybe i'm unaware of the full context of the argument here but uh i think i've heard uh holden kurnowski write somewhere that he thinks that there's more value in thinking about the issues that ea has already identified rather than identifying some sort of unknown risk that for example what ai might have been like 10 20 years ago yeah 11 i mean given this historical experience that you can have some very fundamental tools for thinking about the world missing and consequently miss some very important moral implications uh does that imply that we should expect the space that ai alignment occupies in terms of our priorities should we expect something as big or bigger coming up or just generally tools of thinking like you know expected value thinking for example yeah that's a good question um i think one thing i want to say there is it seems pretty likely that the most important like kind of useful concepts for finding important things are also going to be the lowest hanging and i don't know i think it's like very roughly correct that we did in fact like over the course of building out kind of conceptual frameworks we picked them like the most important ideas first and now we're kind of like refining things and adding maybe somewhat more peripheral things um that's at least if that like trend is roughly gonna hold and that's a reason for um not expecting to find like some kind of earth-shattering new concept from left field although i think that's like a very weak and vague argument to be honest um also i guess it depends on what you think your time span is like if your time span is the entire span of time that humans have been thinking about things then maybe you would think that actually it's kind of strange that it took like 3000 years before maybe even longer uh i guess it depends on when you define the start point it took you know 3 000 people to realize hey we should think in terms of probabilities and in terms of expected impact so in that sense maybe it's like i don't know it took like 3 000 years of thinking to get to this very basic uh very basic idea what seems to us like a very important and basic idea i feel like maybe i have i want to say two things if you imagined lining up like every person who ever lived just like in a row and then you kind of like walks along that line saw how much progress people have made across the line so you're going across people rather than across time then i think like progress in how people think about stuff looks a lot more like linear and in fact started earlier then maybe you might think by just looking at like progress over time and if it was faster early on then if you're kind of following the very long run trend then maybe you should expect like um not to find these kind of again totally left field ideas soon but i think the second thing which is maybe more important is like i also buy this idea that in some sense um progress about thinking in thinking about what's like most important is really kind of boundless like david deutsch talks about this kind of thought a lot when you come up with new ideas that just generates new problems new questions actually some more ideas um that's very well and good i think there's some sense in which you know one priority now could just be framed as giving us time to like make that progress and even if you thought that like we have this kind of boundless capacity to come up with a bunch of new important ideas it's pretty obvious that that's like a prerequisite and therefore in some sense that's like a robust argument for thinking that like um trying not to kind of throw humanity off course and um preventing mitigating some of these catastrophic risks is always just going to shake out as like a pretty important thing to do maybe one of the most important things yeah i i i think that's reasonable um but but then there's a question of like even if you think that the existential risk is the most important thing um to what extent have you discovered all the uh again that like x-risk uh argument and by the way i thought earlier what you said about you know trying to extrapolate what we might know from the limits of physical laws um you know if that can kind of constrain what we think might be possible i i think that's an interesting idea but i i wonder like partly like one argument is just like we don't even know how to define those physical constraints and like before you had the theory of computation it wouldn't even make sense to say like oh uh like this much matter can sustain like so much so much uh flops uh floating point operations per second and then second is like yeah if you know that number it still doesn't tell you like what you could do with it i you know what i think um uh an interesting thing that ho karnowski talks about is uh he has this article called this can't go on where he makes the argument that listen if you just have a compounding economic growth at some point you'll get to the point where um you know like you'll have many or many many many times earth's economy per atom in the affectable universe and so it's hard to see like how you could keep having economic growth beyond that point but that itself seems like um i don't know if that's true then there has to be like a physical law that's like the maximum gdp per atom is this right like if there's no such constant then you can like you should be able to surpass it i i guess it still leaves a lot to be even desiring you could know such a number you don't know like how interesting or what what kinds of things could be done at that point yeah i guess first one is you know even if you think that like preventing these kind of very large scale risks that might like curtail human potential even if you think that's just incredibly important you might miss some of those risks because you're you're just unable to articulate them or really like conceptualize them i feel like i just want to say at some point uh we have a pretty good understanding of kind of roughly what like what looks most important like for instance if you kind of i know get stranded on a camping trip and you're like we need to just survive long enough to make it out and it's like okay what do we look out for i don't really know what the wildlife is here because i haven't been here before but probably it's going to look a bit like this i can at least imagine you know the risk of dying of thirst even though i've never died at first before and then it's like what what if we haven't cons like even begun to think of like the other i was like yeah maybe but it's kind of there's just some like you know table something practical reason for uh focusing on the things which are most salient and like definitely spending some time thinking about things we haven't thought of yet but um it's not like that list is just like completely endless and there's a kind of i guess a reason for that and then you say the second thing which i don't actually know if i have like a ton of interesting things to say about although maybe you could try like kind of zooming in on what what you're interested in there i i i kind of think of it i don't think the second thing has a big implications for this argument but the two um uh yeah we have like 20 other topics that are just as interesting but yeah but just as a i don't know as a closing note the the analogy is very interesting to me that camping trevor you're trying to like do what needs to be done to survive i don't know okay so text an analogy it might be like i don't know somebody like the laser discovers oh that berry that we're all about to eat because we feel like that's the only way to get sustenance here while we're you know just almost starving um don't eat that beer because that berries poisonous um and then then maybe somebody could point out okay so given the fact that we've discovered one poisonous food in this environment should we expect there to be other poisonous foods uh that we don't know about uh i i don't know i don't know if there's anything more to say on that topic i mean one thing well like one i guess kind of angle you can put on this is you ask this question like we have precedent for a lot of things like we know now that uh igniting nuclear weapons does not ignite the atmosphere which was a worry that some people had um so we at least have some kind of bounce on how bad certain things can be and so if you ask this question like what is worth worrying about most in terms of what kinds of risks might um reach this level of potentially posing an existential risk well it's going to be the kinds of things we haven't done yet that we haven't like got some experience with and so you can ask this question like what is what things are there in this space of like kind of big seeming but totally novel precedent-free changes or events and it actually does seem like you can kind of try generating that that list and getting at answers this is why maybe or at least one reason why ai sticks out because it's like fulfills his criteria of being pretty potentially big and transformative and also the kind of thing we don't have any experience with yet but again it's not as if that list is like in some sense endless like there are only so many things we can do in the space of uh decades right okay yeah so moving on to another topic we're talking about the for-profit entrepreneurship as a a as a potentially impactful thing you can do sorry maybe not in this conversation but like we separately we had uh one point yeah yeah um yeah so okay so to clarify this is not just for-profit in order to uh do earning to give so you become a billionaire and you give your wealth away to what extent can you identify opportunities where you can just build a profitable company that solves uh an important problem area or you know makes people's uh lives better um one example of this is wave it's a company for example that uh helps with uh you know transferring money and banking services in africa um probably has boosted people's uh uh well-being in all kinds of different ways um so to what extent can we expect just a bunch of for-profit opportunities for making people's lives better yeah that's a great question and there is really a sense in which some of the more like innovative big for-profit companies just are like doing an incredibly useful thing for the world they're like providing a service that wouldn't otherwise exist and people are obviously using it because they are a successful pro for-profit company yeah so i guess the question is something like you know you're stepping back you're asking how can i like have a ton of impact with what i do the question is are we like underwriting just starting a company so i feel like i want to throw a bunch of kind of disconnected observations we'll see if they like tied together there is a reason why you might in general expect a non-profit route to do well and this is like obviously very naive and simple but where there is a for-profit opportunity you should just expect people to kind of take it like this is why we don't see 20 bills lying on the sidewalk but the natural incentives for uh in some sense taking opportunities to like help people whether it isn't um a profit opportunity they're gonna be weaker and so if you're thinking about the like difference you make compared to whether you do something or whether you don't do it in general you might expect that to be bigger where you're doing something non-profit and like in particular this is where there isn't a market for a good thing so it might be because the things you're helping like aren't humans maybe because they like live in the future so they can't um pay for something it could also be because maybe you want to or get a really impactful technology off the ground in those cases you get a kind of free rider dynamic i think where there's less reason to like well you can't protect the ip and patent something but this reason to be the first mover and so this is like maybe it's not for profit but starting a or helping kind of get a technology off the ground which could eventually be a space for a bunch of for-profit companies to make a lot of money that seems really exciting also creating markets where there aren't markets seems really exciting so for instance setting up like amc's advanced market commitments um or prices or just giving yeah creating incentives where there aren't any so you get the like efficiency and competition kind of gains that you get from the for-profit space that seems great but that's not really answering your question because the question is like what about actual for-profit companies i don't know what i have to say here like in terms of whether they're being underrated um yeah actually i'm just curious what you think okay so i i i think i have like four different reactions to um i've been remembering the number four just in case i'm at three and i'm like i think another thing to say okay so um yeah so i i i had a draft uh an essay about this i didn't end up publishing but that led to a lot of interesting discussions between uh us so that that's why uh we might have uh i i don't know in case the audience feels like they're interrupting a conversation that was already uh preceded the the beginning here uh so one is that um to what extent should we expect this market to be efficient so one thing you can think is listen the amount of potential startup ideas are so vast and the amount of great founders is so small that you can have a situation where the the most profitable ideas are yeah that it's right that like somebody like elon musk will come up and like pluck up like all the maybe like the 100 billion ideas but if you have like a company like uh wave i'm sure they're doing really well but uh you know if it's not obvious how it becomes the next google or something and i guess more importantly if it requires a lot of context for example you talked about like um neglected groups um i guess this doesn't solve for animals and um future people but if you have somebody something in global health where you're like a neglected group is for example people living in africa right um the people who could be building companies don't necessarily have experience with the problems that these neglected groups have so if you have it's very likely that or i guess it's possible that you could come upon an idea if you were specifically looking at how to help for example you know people suffering from poverty in the world in the the poorest parts of the world you could like identify a problem that just like people who are programmers in silicon valley just wouldn't know about okay so a bunch of other ideas regarding uh the other things you said one is okay maybe maybe a lot of progress depends on fundamental new technologies and companies coming at the point where the technology is already available and somebody needs to really implement and put all these ideas together yeah two things on that one is uh good like we don't need to go in rabbit hole on this one is the argument that actually the the invention itself not the invention the innovation itself is a very important aspect and potentially a bottleneck neck aspect of this of getting an invention off the ground and scaled another is if you can build a 100 billion dollar company or a trillion dollar company or maybe not even just like a billion dollar company you have the resources to actually invest in r d i mean think of a company like google right like how many billions of dollars have they basically poured down the drain on like hair brain schemes um uh you you can have like reactions to deep minds with regards to ai alignment but i mean just like uh other kinds of research things that you've done seem to be like really interesting and uh really useful um and yeah all the other fan companies have like a program like this like microsoft research or i don't know what amazon's thing is and then another thing you can point out is with regards to setting up a market that would make other kinds of ideas possible um and other kinds of businesses possible in some sense you could maybe make the argument that that's that maybe some of the biggest companies that's exactly what they've done right if you think of like uber um it's not a market for companies or maybe maybe amazon is a much better example here where um you know like you theoretically who had an incentive before like a pandemic happens i'm gonna manufacture a lot of masks right but amazon provides uh makes the market so much more liquid so that you can you can just start manufacturing masks and now immediately put them up on amazon so it seems in these ways actually maybe starting a company is a really uh is an effective way to deal with those kinds of problems yeah man we've gone so async here i should have just like said one thing and then i'm sorry for throwing lots of things um as far as i can remember those are all great points yeah i think my like high level thought is i'm not sure how much we disagree but i guess one thing i want to say is again thinking about like in general what should you expect the real biggest opportunities to typically be for like just having a kind of impact you know one thing you might think of is if you can optimize for two things separately that is optimize for the first thing and then use that to optimize for the second thing first is trying to optimize for some like combination of the two at the same time you might expect to do better if you do the first thing so for instance you can do a thing which looks a bit like trying to do good in the world and also like make a lot of money um like social enterprise and often that goes very well but you can also do a thing which is try to make a lot of money and just just you know make a useful product that is not directly aimed at you know improving humanity's prospects or anything but it's just kind of just great and then use the success of that first thing to then just think squarely like how do i just do the most good um without worrying about whether there's some kind of profit mechanism i think often that strategy is gonna pan out well there's a thought about the kind of the tales coming apart if you've had this thought that at the extremes of like either kind of scalability in terms of opportunity to make a lot of profit and at the extreme of doing like a huge amount of goods you might expect it's better to be like not such a strong correlation again one reason like in particular that you might think that is because you might think that like future really matters like humanity's future and um sorry to be like a stock record but like because there's not really like a natural market there because these people don't haven't been born yet that is like a rambly way of saying that okay that's not always going to be true but i basically just agree that yeah i would want to resist a framing of doing good which just leaves out like also doing some studying some successful for-profit company like there are just a ton of really excellent examples of where that's just been a huge success and yeah should be celebrated um so yeah i think i disagree with the spirit um maybe we disagree somewhat on the like how much we should kind of relatively emphasize these different things but um doesn't seem like a kind of very deep disagreement yeah yeah maybe i've been spending too much time with brian kaplan or something um so by the way the tail is coming apart i think is a very interesting um very interesting way to think about this uh scarlet xander's good article on this and like one thing he points out is like yeah generally you expect different parts of different types of strength to correlate but the guy who has the strongest grip strength in the world is probably not the guy who has the biggest squad in the world right yeah okay so i think that's an interesting place to leave that idea oh yeah another thing i wanted to talk to you about was back testing ea so if you have these basic ideas of we want to look at problems that are important neglected and tractable and applied them throughout history so like a thousand years back uh 2000 years back 100 years back is there a context in which applying these ideas um would maybe lead to a perverse outcome an unexpected outcome um are there examples where i mean there's many examples in history where things you could have like easily made things much better but maybe it made it much better than even conventional morality or like present-day uh ideas would have made them so react to the first part of the question which as i understand it is something like can we think about what some kind of effective altruism like movement or if these ideas were in the water like significantly earlier whether they might have misfired sometimes or maybe they might have succeeded in that in fact how do we think about that at all i guess one thing i want to say is that very often the correct decision exante is a decision which might do really well in like some possible outcomes but you might still expect to fail right the kind of mainline outcome is this doesn't really pan out but it's a it's a moonshot and if it goes well it goes really well this is i guess similar to certain kinds of investing where if that's the case then you should expect even if you follow the exact like correct strategy you should expect to look back on the decisions you make uh made rather and uh see a bunch of failures um where failure is you know you just have very little impact and i think it's important not to to kind of resist the temptation to like really kind of negatively update on whether that was the correct strategy because it didn't pan out and so i don't know if something like ea type thinking was in the water and was like thought through very well yep i think it would go wrong a bunch of times and that shouldn't be kind of terrible news when i say go wrong i mean like not pan out rather than do harm if you did harm okay that's like a different thing um i think one thing that points to by the way is like you could take it you could choose to take a strategy which looks something like mini max regrets right so you have a bunch of options you can ask about the kind of roughly worst case outcome um or just kind of like you know default air outcome on each option and one strategy is just like choose the option with the least bad kind of meh case and if you take this strategy you should expect to look back on the decisions you make and like not see as many failures so that's one point in favor of it another strategy is just like do the best thing in expectation like if i made these decisions constantly what in the long run just ends up like making the world best and this looks a lot like just taking the highest ev option maybe you don't want to like uh run the risk of causing harm so you know that's okay to include and you know i happen to think that like that kind of second strategy is very often going to be a lot better and it's really important not to be misguided by this kind of feature of the mini max regret strategy where you look back and kind of feel a bit better about yourself in many cases if that makes sense yeah that's super interesting i mean if you think about uh back testing in terms of the the stock like you know models for the stock market one thing that to analogize this one thing that uh tends to happen is that a strategy of just like trying to maximize returns from a given trade that results very quickly and you going bankrupt because like sooner or later there will be a trade where you lose all your money and so then there's something called the cali criterion where you reserve a big portion of your money and you only bet with a certain part of it which sounds more similar to the minimized regret thing here um unless your expected value includes a possibility that i mean in this context like you know like losing all your money is like an existential risk right so uh yeah maybe you like bake into the cake in the definition of expected value the odds of like losing all your money um yeah yeah yeah that's a great that's a really great point like i guess in some cases you want to take something which looks a bit more like the the kelly bets um but if you add to your margins like relatively small margins compared to the kind of pot of resources you have then i think it often makes sense to take just do the best thing but not worry too much about what's the kind of like size of the the kelly ben but um yeah that's a it's a great point and like i guess a naive version of doing this is just kind of losing your bankroll very quickly because you've like take it to your enormous bets and forgotten that um they might not pan out yeah so i appreciate that what did you mean by add to the margins so if you think that there's a kind of a pool of resources from which you're drawing which is something like maybe philanthropic funding for the kind of work that you're interested in doing and you're only a relatively marginal actor um then that's unlike being like an individual investor where you're more sensitive to the risk of just running out of money and um when you're more like an individual investor then you want to like pay attention to what the size of the kelly bet is if you're acting at margins then maybe that is less of like a big consideration although it is obviously still a you know very important point well and then uh by the way i don't know if you saw my recent blog post about why i think there will be more billionaires yes okay yeah yeah uh i don't know what your reaction to any of the ideas there is but like my claim is that we should expect the total funds dedicated to va to grow quite quite a lot um yeah i think i really liked it by the way i thought it was great one thing it made me think of is that there's quite an important difference between trying to maximize returns for yourself and then trying to get the most returns just like for the world which is to say just doing the most good where one consideration we've just talked about which is a risk of just like losing your bankroll which is where like kelly betting becomes relevant um another consideration is that as an individual just like trying to do the best for yourself you have like pretty steeply diminishing returns from money or just like how well your life goes with that extra money right so like if you have like 10 million in the bank and you make another 10 million does your life get twice as good obviously not right and as such you should be kind of risk-averse when you're thinking about the possibility of like making a load of money if on the other hand you just care about like making the world go well um then the world's an extremely big place and so you basically don't run into these diminishing returns like at all and for that reason like if you're making money at least in part to in some sense give it away or otherwise just like have a positive effect in some impartial sense then you're gonna be less risk averse which means maybe um you fail more often but it also means that people who succeed like succeed really hard um yeah so i don't know if that's in some sense i'm just recycling what you said but um i think it's like a really kind of neat observation well and another interesting thing is that not only is that true but then you're also you're also in a movement where everybody else's has a similar idea and not only is that true but also the movement is full of people who are young techy smart and as you said dress neutral so basically people who are going to be way over represented in the in the ranks of future billionaires and they're all hanging out and they have this idea that you know we can become rich together and then make the world better by doing so um you would expect that this would be exactly the kind of situation that would lead to people teaming up and starting billion dollar companies all right uh yeah so a bunch of other topics and effective authorism that i wanted to ask you about so one is should it impact our decisions in any ways if the many world's interpretation of quantum mechanics is true i know the argument that oh you can just think of you can just translate amplitudes to probabilities and if it's just probabilities then decision theory doesn't change my problem with this is i i've gotten like very lucky in the last few months now i think it like changes my perception of that if i realize actually most me's and okay i i know there's like problems with saying me's at to what extent they're fungible most branches of the of the multiverse uh like i i'm like significantly worse often that that makes it worse than oh i just got lucky um but like now i'm here and another thing is if you think of existential risk um and do you think that even if like essential risk is very likely in some branch of the multiverse humanity survives i don't know that seems better in the end than oh the probability was really low but like it just resolved to we didn't survive does that make sense okay all right there's there's a lot there i guess rather than doing a terrible job at trying to explain what this money was thing is about maybe it's worth just kind of pulling people towards you know just googling it i should also add this enormous caveat that i don't really know what i'm talking about this is just kind of an outsider who's taking this kind of i know this just this stuff seems interesting yeah okay so there's this question of like what if the many worlds view is true what if anything could that mean uh with respect to questions about like what should we do or what's important and one thing i want to say is just like without zooming into anything it just seems like a huge deal like every second every day i mean in some sense like just kind of dissolving into this like cloud of me's and like just kind of unimaginably large number of of me's and that each of those me's is kind of in some sense of dissolving into more clouds um this is just like wild also seems somewhat likely um to be true as like far as i can tell okay so like what does this mean you yeah you point out that you can talk about having a measure over worlds in some sense you can there's actually a problem of how you get like probabilities or how you make sense of probabilities on the many worlds view and there's a kind of neat way of doing that which like makes use of questions about how you should make decisions that is you should just kind of weigh future use according to in some sense how likely they are but it's really the reverse you're like explaining what it means for them to be more likely in terms of how it's rational to weigh them um and then i think it's like a ton of very vague things i can try saying so maybe i'll just try doing like a brain dump of things you might think that like many worlds being true could push you towards being more risk neutral in certain cases if you weren't before um because in some cases you're like translating from some chance of this thing happening or it doesn't into some fraction of you know worlds this thing does happen and another fraction it doesn't that's kind of like i don't think it's worth reading too much into that because i think a lot of the like important uncertainties about the world are still like subject to uncertainties about how most worlds will in fact turn out but it's kind of interesting and notable that you kind of like convert between overall uncertainty about how things turn out to like more certainty about the fraction of the ways things turn down i think another like interesting feature of this is that so the question of like how you should act is no longer a question of like how should you kind of benefit this person who is you in the future who's one person it's more like how do you benefit this like cloud of people who are all success of you that's just kind of like diffusing into the future and i think you point out that you could just like basically salvage a lot of basically all decisions even if that's true but the like picture of what's going on changes and in particular i think just intuitively like it feels to me like the gap between acting in a self-interested way and then like acting in an impartial way where you're like helping other people it kind of closes a little in a in a way like just you're already benefiting many people by doing the thing that's kind of rational to benefit you um which isn't so far from benefiting people who aren't like continuous with you in this special way so i kind of like that as a thing huh um it's so interesting yeah and then okay there is also this like slightly more out there thought which is here's the thing you could say many worlds is true then there is at least a sense in which there are very very many more people in the future uh compared to the past like just unimaginably many more and even like the next second from now there are many more people so you might think that should like make us have a really steep negative discount rate on the future which is to say we should like value future times much more than present times and like in a way which would just kind of it wouldn't like modify how we should act it just like explodes how we should think about this this definitely doesn't seem right maybe one way to think about this is that if this thought was true or like looks kind of directionally true then that might also be a reason for being extremely surprised that we're both speaking at like an earlier time rather than a later time because if you think you're just like randomly drawn from all the people who ever lived it's like absolutely mind-blowing that we get drawn from like today rather than tomorrow yeah given that it was like 10 to the something many more people tomorrow um so it's probably wrong and wrong for reasons i don't have a very good handle on because i just like don't know what i'm talking about i mean i can kind of try parroting the reasons but like it's something i'm you know i'm interested in trying to really crock those reasons a bit more that that's really interesting um i i i i didn't think about that argument uh for uh this election argument i think one resolution i've heard about this is that you can think of the proportion of you know hilbert space or like the proportion of all the the the like the universe's wave function that could be the like the probability rather than each different branch uh you know what i just realized that the selection argument you can you made maybe that's an argument against bostrom's um boston's idea of we're living in a simulation because basically his argument is that there will be many more simulations than there are real copies of you therefore you're probably in a simulation the like the thing about like saying that all the simulations plus you are your prior should be equally distributed among them seems similar to saying your prior of being distributed along each possible like branch of uh the wave function should be your prior across to them should be the same whereas i think um in the context of the wave function you were arguing that maybe it should be like you shouldn't think about it that way you should think