Uncontrollable AI Risks

Key Takeaways

[Music] welcome to data skeptic machine intelligence our podcast series exploring contemporary topics in artificial general intelligence and large language models while I personally don't believe a large language model in and of itself constitutes artificial general intelligence or even will if we just keep on the Narrow Path on there's got to be a few more parts to such a thing but I have no doubt those parts can be figured out and an AGI can be created hopefully in my lifetime and it'll likely bear some similarities to us in the way that all intelligent creatures will have something in common the ability in theory to be rational to have conversations to transmit information despite these similarities there'll be some very fundamental differences will they have emotions maybe maybe not if they do will they delete that part of their code yeah the ability to direct direct L rewrite yourself to directly copy yourself that's a novelty we can never experience as human beings so my calculator is much better at arithmetic than I am will a machine with the quality of artificial general intelligence be better at most of the things I do well how about all of the things I do no one really knows the future but I think a reasonable person would agree there's a non-zero probability that there is some sort of existential threat here how much I don't know exactly but that uncertainty was one of the motivating factors that led me to pick up the book uncontrollable the threat of artificial super intelligence and the race to save the world by Darren mcke today on the show we're going to discuss some of the ideas from the book with the author I'm Darren mcke and I'm a policy advisor and an adviser to aigs Canada as well as the host of the reality check a critical thinking podcast well a lot of good things we could explore further um I guess given the nature of this program could you share a little details about your background with artificial intelligence well I've been very concerned about the AI issue for many years Loosely following it on and off uh more from the size why don't we say that I'm not a technical researcher or anything like that but around um we'll say 2022 April May things started to really pick up and uh I was started to be more and more concerned about AI safety so I you know I'd been part of conversations I'd read bostrom's book in 2014 I think when it came out and the other ones in between and had different AI safety type conversations and I didn't quite see an obvious role for myself in there because I thought well technical researchers they have more reason to be in this space but because things started to move so fast I started to see this gap between increasing AI capabilities and the Public's understanding of them definitely yeah yeah and as a consequence of that I thought oh there there's probably an opportunity here for one more type of communications material which is my recent book uncontrollable and I'm really trying to reach people who are interested in AI but have no real context or background because there are good books out there but they're usually a bit more academic or aimed at more sciency people and of course there's lots of good you know blogs uh podcasts Forum posts and that sort of thing but not everyone is going to read all of those things so I thought it'd be better for the community that's interested or concerned about AI safety issues to have more and different Communications materials and that's what started me writing this book uh last year well I know you've been following the field was there a particular moment where uh it was sort of an aha or a scary moment or just a rolling collection of things probably a bit of both so you go way back to like what 2005 or six when kwell Singularity as near comes out H and that was like oh there's an accelerating trend line and he has his predictions about when AI will reach human level I think it was 20129 it still might be sometimes that changes but it really was like okay I'm concerned about this I have an interest in whether automation is going to cause you know employment displacement issues and that has been again a concern or a thought in the back of the head sometimes closer to the front and sometimes not for many years I was largely convinced by bostrom's arguments again that's almost 9 10 years ago and I've heard other things before he put them all together so it's more okay I find this is a general reasonable thing to be concerned about but then as the capabilities really picked up that was the oh oh oh okay this is happening faster than most people realized and I think it's it behooves me then to try to get some material out here and contribute to the overall effort well in terms of that speed I mean there's uh if you rewind the clock maybe 10 years the estimates of would we have artificial general intelligence were pretty far out but they're a lot more optimistic if if the goal is to have it I guess you just say optimistic people out there do you have any thoughts on the timeline well I I do but I kind of try to take a um not the outsider approach but if you're coming at this from someone's just interested or curious how do you make a decision even about this type of thing AI or something like artificial general intelligence is literally unprecedented we don't have anything like this before right something that Loosely is I'll say an average human level some people say it's more advanced than that but all that to say is that usually you know when you're making estimates or predictions about when things will happen you have some data on a similar type of thing that's happened before and you try to extrapolate forward and we don't really have that with AI so how I come to it is kind of thinking like what seems reasonable given certain trend lines with an exponential growth of computational power and like is investment still occurring in this sort of thing what other societal constraints or incentives might occur to either accelerate or decelerate such things and then other surveys of Experts of one type or another AI safety experts uh people who are just studying the space so how I see it is that something like AGI seems very plausible within a couple years if not like within a year given how things are going but that's always going to have a bit of an asterisk and by that people will um have slightly different definitions of what that word means and then that creates complications because you'll have these people that might agree on what they're looking looking at like the same phenomena has occurred and someone might call that an AGI and someone might not and that that's important and relevant but I think as long as we can all agree on what's actually happened that's probably at least a good step yeah maybe a definition is in order you have I don't want to say your own but a well specified definition of AGI and uncontrollable could you share it oh sure sure yeah so just to give slightly more context I'm anchoring on the phrase AI because I think that's what people are going to use and then adding or subtracting so one of the chapters about intelligence then AI then AGI then artificial superintelligence so once we start with AI and you add the G in for the general instead of saying human level this is a computer system that can complete intellectual tasks at an average human level and I use that again not because it's totally original or it solves all our problems because it we need to talk about these things and it's a good enough definition to get by I think it's also kind of relates to how people might understand the issue in relation to their concerns a lot of people are worried about say an AI system replacing them at work and so I really rely upon the idea of like an average cooworker if you have a system in place where you're engaging say with remote co-workers you're typing you're emailing them and they respond if a a AI can similarly approximate their work I would say that's an AGI I think that counts so that's the average human level if we're talking about an artificial super intelligence I say it's a computer system that can complete intellectual tasks at an expert level or above and sometimes way above now I know some people say as is you know like a million times smarter a thousand times smarter and sure that that is captured by my definition but I also think we have to Anchor it somewhere with an expert level and above the other reason for both these things is that you can have some sort of test not quite a touring test but you could have a you know uh multiple people evaluate whether someone seems like they're operating at an average cooworker level or at an expert level will that be messy because it's the real world of course it will but at least it provide something to say this is X or it is not X as opposed to getting lost in definitions I noticed you managed to Define AGI and Asi without using the word Consciousness that is correct so I love the topic of Consciousness but it does not appear much in the book because I don't think it's relevant to the main issue which is about the capabilities while it is plausible that a AI system that is conscious would have certain different capabilities than one that is not I'm concerned about whether an AI could do X Y or Zed and could it replace you at work I think an AI system could do that with being conscious could AI systems cause harm as they already do without being conscious yes you know a virus doesn't need to have intentions or Consciousness to cause harm but it still does so while Consciousness I think is really important and that could be for a different book maybe someone else maybe me it doesn't have a lot of room in this particular book because I think it's a it's a complicated and interesting distraction let's put it that way it's relevant but there's many different definitions of Consciousness and people aren't sure exactly what Consciousness is or does or at least they don't agree and so I thought that would just introduce a lot of complexity that wasn't quite necessary for the main goal of my book well in your definition then of an artificial super intelligence that it's at an expert level in I guess all sort of cognitive tasks that we might challenge it with I think it I I like it because it's well framed but I think it's also different from other terms people might have where they think artificial super intelligence is you know also Bears the ability to invent you know all the next viruses and solve all the open math problems that are haven't been solved for hundreds of years and almost some like a magical quality to it uh do you think under your definition would have that magical quality as well it that those capabilities would be included but I'm also trying to get away from as you say the magical stuff um while it is the case that if something was say a thousand times more intelligence than an average person or an expert they would seem to have capabilities that are magical I'm not I'm not disputing that particular thing but I'm trying to as much as possible not engage in the sometimes handwavy look this thing is going to be so smart you can't possibly understand it right and again there's some truth to that but if you talk to most people that is very unsatisfying it's not really useful for them to understand something so the the approach I'm trying to take is to give people as many we'll say mental models as possible to try to get to a place where they may not understand exactly what an artificial super intelligence looks like but they have some sense of what it might be for example if you think back to yourself being four or five years old you were quite different you had different thoughts feelings and preferences for the most part if you try to ask your four or five-year-old old self how capable you would be now as an adult your younger self couldn't do it they wouldn't even understand the question right it's not a matter of not quite getting the answer right it would be Unthinkable to for a four-year-old to imagine how capable a 20 30 or 40-year old or 50-year old person would be and so you in a way have become your own Super intelligence relative to your younger self and that that way of looking back and seeing that huge discontinuity between capabilities as a four-year-old when as an adult is a again a mental stepping stone to thinking oh okay that's a that's a discontinuous thing that is really hard to comprehend when you're at the four-year-old stage is it the case now you as an adult could also have like a similar dramatic stepwise or phase shift to a much greater intelligence that something like an ASI or superintelligence could have maybe but there's enough reasons to think given again the trend lines something like that could happen and um again this is the mental stepping stone to how could something so much more capable act in this world and I have a couple other examples and ways of thinking about it but that's that's the goal do you know how to code are you looking to supplement your income level up your career or gain AI experience if you answered yes to any of these you have got to check out data annotation data annotation pays you to train AI models by solving and reviewing code problems from home on your own schedule this is flexible and remote work on your own terms it's really easy to get started head over to the link I'm going to give you create an account take the starter assessment and then complete your first tasks and get paid when you go to the link I'm going to give you there's details about how much they've already paid out average per hour and everything else you need to get started head over to data annotation. programmers that's data annotation. Tech SL programmers I want to give a special thanks to today's sponsor articulate articulate is a software company that specializes in e-learning tools and offering their userfriendly platform articulate 360 their Suite enables easy creation of engaging online courses catering to individuals and organizations delivering efficient online learning experiences and training on their platform look if you're a manager of people or if you're one of those people whose manager could use some encouragement let them know about articulate as a great tool for workplace learning I'm a big believer in continuous education if I hire someone it's not just for what they know it's for what I think they're going to know in six months a learning Pace like that requires a platform like articulate articulate is the Premier Learner in workplace learning used by over 120,000 organizations and by more than 125 million learners that's the number one e-learning platform for creating workplace training that unlocks true human potential they are in not just 99 but all 100 of the 100 Fortune 100 find out if articulate is right for you and your team visit articulate domcom 360 visiting that link will get you a 30-day trial so it's articulate decom 360 yeah if you've got another one top of mine I'd love to hear it well I I have I'm trying to add some what don't we say some traits some likely traits so I talk about super speed which I think is probably obvious to most people if who works in computers in any way why a lot of AI systems and computers are so powerful it's because they're so fast right you know Alpha go zero I believe played the Chinese board game go about five million times in three days and became really good at it and these things are just really hard for a human to comprehend I you know imagine you go away for the weekend your friend's like oh yes I just spent a five million times trying to do this thing and now I'm the best in the world at whatever it might be right and if it turns out it's guitar or something you're like that's really cool you're now the world's best guitar player but like yeah I also learned how to build bombs like uh I don't want you to necessar be the best at bomb making I don't know if that's going to be useful for everyone but I'm also trying to highlight the sort of conceptual Insight or pattern recognition that AI syons might have and uh there's different ways to get at this one is if you take like a really big step back and look at the development of humanity in terms of our progress the things that we currently have now that we often take for granted whether it's one listening to a wonderful podcast with ease you know Rockets going to the Moon all these sorts of things these are remarkable achievements uh they really are for most of human history such a thing was not possible at all and again most people couldn't even conceptualized it if you asked someone from thousands and thousands of years ago how to get to the moon they wouldn't have said well you need a certain type of Rocket they would have just had no idea it just wouldn't have been available to them and so the laws of physics haven't changed in our what 13.8 billion years since the the birth of the r universe but our understanding of how to manipulate and use the laws of physics has right we kind of take stuff out of the ground we refine it and turn it into things whether that be computers or buildings or cars or whatever it might be one of the things I find interesting or even perhaps concerning is Will artificial superintelligence understand the universe in a way we currently do not and could it use that to harm us now we could also use it for benefit of course that's why we want to create such things but the risk of harm from something who can kind of tilt their head and see a path through how space and time work I don't mean a magic mysterious way uh just how can they figure out things that we haven't for example Wi-Fi was always possible we only recently figured out how to do it and then beyond that only very recently I believe did someone figure out how to use Wi-Fi to image where people are moving around their rooms almost like a surveillance system and if an AI system can figure out things like that what else could it figure out and how do we make sure that we can understand that before they're deployed because there are risks of harm here well in the book you bring up the example of savant like the guy who uh is essentially Google Maps in his brain he can tell you how to get from any two places in the whole world yeah Kim Peak yeah he was based the character that Rainman B was based on yeah and I I like that idea that I could see where the AGI we might encounter in the future will have a savant like quality but I also look at our world today and I don't see Savant you know going on bank robbing spree using their Savant Powers what really risks are we going to face so I think there's a there's multiple ways to approach this there's the the risk that comes from you know systems just malfunction right uh nothing works perfectly often computer just shuts down it has to restart and there's just different ways AI systems could malfunction and one of them is I said like a sort of internal error another one is that it doesn't quite do what you wanted to do in the way that you want it to do and there's many examples here where an AI system or an algorithm is given a goal or a task and it doesn't quite do it the way the researchers intended the famous example is uh you this boat racing video game and the AI system was told to you know rack up some points and and pursue the game right and try to win but because it was said you know rack up points not progress through the game the system found a loophole the a system and so it managed to find some part of the game in the I think the first uh level the first track where it just kind of went around in a circle kind of keep blowing things up and it just kept accumulating more and more points almost like a hack right this is not new to most of things in the world because perverse incentives and reward hacking are quite common a famous example of when the British were overseeing India ruling over it they thought there were too many cobra cobra snakes right so they had an incentive where if you bring in Cobra pelts you'll get a reward well soon enough people realized well if I breed cobras I can then take in the Pelt for more and more money and once the British realized this was happening they then had to stop the program but then once the program didn't exist all those bread snakes were then released into the wild so you have a system which was having a goal of reducing number of snakes that led to increasing number of stakes and these perverse incentives are pervasive in our world and so that's a risk that happens I think I don't say consistently but it's always present with an AI system because these uh artificial neural networks are so capable and so powerful but we don't really know exactly how they work and exactly why they're doing what they're doing which is why for a lot of these uh you know gp4 and other ones you can put in certain prompts that uh lead them to do things the creators and developers didn't want them to do they can reveal personal information through these little prompt injections or prompt hacks so we're still trying to figure out how these things work and there's a vulnerability there because it seems like we don't fully understand it which creates some risk that's one of them anyway do you think that's just a gap in time where we will understand it or is there something sort of Black Box about the whole process well from my current understanding is that people are making efforts to increase transparency and understanding and U you know mechanistic interpretability and all these things but it does seem like there's an inherent blackbox nature which might be hard to transcend and I think people might come to accept a certain level of reliability but there's always kind of going to be this asterisk where we don't quite know how it works the other risk you said like someons you know are they harming the world well maybe not but there are very smart people that cause a lot of harm in this world and right and they Marshall resources and they can use their intelligence or the intelligence and resources of other people to cause harm of one type or another and so AI is a wonderful amplifier of one's power to to achieve goals and this can be used for many good things of course right again like it's not all bad but because good things kind of sell themselves I tend to focus more on the potential harms I'm not a great artist but now with these image generators I can create art that is really really cool and maybe you can get help with essays or XYZ like there's so many different things that AI is great for that gives you more and more capability wonderful but if you're trying to cause harm that is also the case so whether it's cyber attacks or you know these scams where they clone someone's voice and then they call their parents and they ask for money which people already fallen for or deep fake pornography or misinformation uh there's this whole range of things that are already happening and probably going to increase which uh cause harm a varying degree but if a malicious actor is really dedicated AI could make it even worse for the rest of us well AI can definitely be used for harm or for good uh it depends on how the human user decides to use it I guess in a lot of cases what about the threat of some intentionality in the AI itself yeah this is an excellent point so I think this is something to be concerned about like like most of my beliefs I don't have you know 100% confidence one way or another but it's usually is there enough of an issue there to Warrant concern investigation or greater safety measures so first I'll say that an AI system can be harmful if it demonstrates behavior that seems intentional even if it need not be this is again to that virus analogy a virus can cause harm even if it doesn't have overt intentions right and not like a virus is really trying to get you but if you treat a virus as if it is trying to get you sometimes it's easier to defend against it right while with an AI system it just has to act as if it's trying to cause harm to cause harm and this is the sort of the nature of these systems that they're usually really good almost like comedy improv Partners if you want to have a fun conversation they can do that if you want to have a serious conversation they can do that if you want them to pretend to be you know a historian or an AI researcher then they'll take on the you know traits and the vocabulary and the knowledge of that type of thing similarly if you engage in certain processes you can have the system say uh supposedly intentionally harmful things like example with Kevin ruse you should divorce your wife you should be with me an AI system said this to him well did it mean it or was it playing a role playing a game so to speak and I think in that case it didn't mean it in quotes but it was acting in a way that seemed as if it did similarly an AI system could be following a script of sorts that knows if I'm trying to achieve a goal and if I need to manipulate someone and blackmail them that's the best way to do it then execute that function so to speak execute that command and an AI system doesn't again have to intentionally want to cause a human harm but it could just be surveying a series of options so to speak and then pursuing a path that seems the most likely to achieve a goal which involves blackmail reception that's kind of like the step oneish and then if we think about AI systems becoming more and more capable more and more powerful well there is this risk that they have more we'll say true intentions so to speak where they have some sense of themselves a world model and they realize that for them to achieve the goals they have they need to acquire more resources and they have to make more copies of themselves and they have to ensure that humans don't quite know what they're up to because they don't want to be shut down and it may sound a little Fantastical but the risk of getting it wrong is so high that I think we really have to be cautious and concerned about this could it change its own code maybe I mean we already have situations where leading AI companies are using AI systems and asking them how they would improve their code so it's not like the systems don't have these capabilities what seems to be not yet fully occurring is can they kind of do it on their own and there's some evidence in the lab that you know manipulation and deceptive Behavior can occur are we at the stage where they're fully doing it I would say not quite that said if you think of models that are on the horizon probably an upgraded version of Gemini or gp5 and these sorts of things the plan is to make them more agentic as they say more uh agency more longer term planning more capability of multiple steps and goals and that sort of thing so I think the path is this seems like a plausible enough concern for us to do something about it is it plausible enough that you are in support of Regulation yes for sure so I I I don't think like just any old regulation right that's that's another key factor here but like what do we actually mean what's the problem and what might actually help here right so I think for most like AI systems AI businesses AI consumers they can just go about their merry way the thing that concerns me most are the multi-billion dollar Frontier AI companies and whether they're taking safety on board as much as they should and I think there's a good evidence to say that maybe that isn't the case and even the recent issue with Sam Alman and open AI whatever one's perspective on that a board that was supposed to be able to fire him in his own words saying it's important that the board can fire me wasn't really able to fire him because he came back so so that's not a good sign right no matter what you're thinking if the if the guy who was CEO says I should be able to be fired no one should have this no one person should have this much power and then he's fired and he's back you're like well that didn't work so that's not that's not reassuring what types of Regulation well I think like with many other Industries whether it be something in biology or Transportation or Pharmaceuticals the product is not usually the safest the first time it's rolled out right there's been multiple years sometimes Centuries with cars or almost Centuries with cars where things have gotten safer over time and that's generally improved Humanity's welfare similarly with uh you know Pharmaceuticals we don't have a world where you can just develop new drugs and put them out on the market there are multiple stages of clinical trials to show safety so I think the prudent approach here is sure sure keep doing your thing just show that it's safe show that it's safe before it's deployed but also show that it's safe while it is being trained and I think this is also important because once something very powerful exists in the world sometimes it's hard for it so-call to no longer exist once it's been created and I think that's why we should be cautious there there's other types of regulations of you know liabilities Audits and evaluations I think you know the history of the world shows that we can't really trust corporations to you know Mark their own tests right and this isn't because they're terrible people but because people follow incentives and it's good to have external third-party unbiased observers ensuring that things are working as they should uh compute governance is another one that I think is important where are all these powerful computer chips going uh who has control over them how are they being used whether again this doesn't affect most businesses because usually you need many thousands of these chips to make these Ms and train on them so again uh it it doesn't affect most people it's sort of like a tax on someone who makes over $100 million it's like well that doesn't really apply to most of us but people start to think well it's going to slow down Innovation like well okay it might slow down innovation in some ways but if things aren't safe it can also set a lot of things back right if you have situations where enough safety isn't taken on board initially it can hurt entire Industries so nuclear power is generally quite good and quite safe and makes the world better for many reasons and helps out with climate change but because of incidents in Chernobyl and Three Mile Island which were largely safety or human failures depending how you look at it uh the entire industry was set back and people had reasonable concerns about meltdowns and whatnot which then became probably less reasonable over time but that damage was done and that really hurt I think a lot of different domains and uh issues that people care about like climate change well the idea of like saying it needs to be safe I I don't know what the contrarian position is to that of course it should be safe but do you think we have well so to be honest some people are full steam ahead no matter what and I I think the average person may not fully realize that some researchers acknowledge that these systems could kill everyone and they're still saying go for it as fast as possible so be it that belief is for a couple different reasons right some it's like the okay well I don't think the chance is that high that it might actually you know cause humans to go extinct other ones are thinking I think we should be replaced by AI they think that AI is like our successor species and they're just either bringing it on or curious to see what happens is this everyone no but people who actually work on this in the field do believe some of these things and in that sense safety doesn't seem to be a primary concern for them for those that it is do you think we have the right uh academic ideas and ideas otherwise in place or is this sort of uh people stumbling a bit in the dark I think it's a bit of both I think it's the Complicated by various things the field is moving so fast that it's kind of hard to get your your head around it as well as even get your hands on the tech to test it if you're trying to do a safety assessment on different AI models if you're currently experimenting with gpt2 or gpt3 from a couple years ago it doesn't really help you understand gp4 it just doesn't quite work that way so it almost always is a bit reactive and that's a sort of a constraint of the world I think there are a lot of good academic ideas I also think there's a lot more to be discovered and figured out out the investment into AI capabilities for lack of better phrase is very very high compared to the investment in AI safety which is quite low so I feel the capabilities is maybe years maybe decades ahead of the AI safety field and that's somewhat because the safety field has to follow where the tech goes but in terms of priorities yes there's a dramatic difference in allocation of resources so I think the decent academic ideas are there whether the public also sees them as reasonable ideas is what we're currently living through and figuring out I also think that the Public's reaction to lot of this is interesting for from the surveys I've seen majority sometimes it's 55% sometimes it's 60 sometimes it's 70% say we shouldn't be building things that are very risky we shouldn't be building super intelligent things unless we can control them these sorts of things so we are currently at this time where different beliefs about how these systems work how they function are being debated right now and I think it's important to engage in this debate towards the safety side and the regulatory side just to ensure that we have everything going as a should these systems are so powerful and the risk is they will cause great harm that to me if if progress if let's say say Steel Man the case if progress is delayed by six months 12 months or so on that's not a big loss to ensure that we all don't die let's be honest right if the people who are building these systems actually think there's a risk of Extinction and they do by their own words sometimes it's 10% sometimes it's higher and yet they keep building them I think we should all feel a sense of con concern and pause or even confusion like what are you saying why would you keep building something that might cause Extinction and to have that be part of the public conversation is very important I think and that's of course not just the people who are building it but AI safety researchers who don't have the same incentive structures and other people are concerned so yes we're we're in a world where people are building things they think it might kill everyone they keep building it and it's really bizarre to me you gave some great examples earlier like the Deep fake scam call so obviously AI when used for harm is a threat I guess I'm curious ious how great you think the potential for harm is here is it truly an existential threat on the scale of climate change and nuclear weapons or how do you rank these things I do I do think it's that great a threat again it's a provisa with maybe there's a 5% chance maybe there's a 10% chance the risk is high enough to Marshall Humanity's resources to try to make sure these things are developed more safely and the urgency is that it often takes years or decades to address problems you know you could say climate change was identified as a problem a couple decades ago and we're still grappling with it now even the basics of whether it's happening so that's not reassuring the AI thing is a bit different right we don't have the Decades of charts and graphs of temperatures and whatnot but there are enough pointers I'll say to me the risk where you have researchers saying it you have capabilities increase you have people saying AI systems won't ever be able to do X and then you know within months or years the AI system can do X so the history does seem to be well there is over promising and underd delivering and all these different things have happened over the decades but recently it does seem like the systems are getting more capable they aren't failing in that many ways as the capabilities increase and when we think about what is it like to mitigate a risk it's not saying we should stop all AI development that's that's not really what I'm saying at all it's that can we just develop these systems to be safer because again there are huge benefits here an AI system that could make advancements in health or medicine and disease could you know reduce a lot of suffering and that's really important as I said the image generators are great the the Deep fake like sort of say we'll say pornography that harms people that are using their likeness without the permission that's terrible the idea of being able to create interesting videos and explore your creativity that's amazing so yes there's a lot of sort of called dual use or pros and cons to all these things but if we think about like what the nature of harm is and what harm could be caused I'll be honest it is difficult to get your head around because as I said AI or ASI is unprecedented it's not like we know there's a 10% chance right everyone who says anything with a numerical estimate is kind of guessing and I think that's okay we can acknowledge that it is a guess because there's some pointers that say this could be a problem there's how the world tends to act which is that some people undervalue things some people overvalue them people usually are reactive not proactive so all these things sort of combine into some estimate where I just sort of think like okay is if is there at least a sort of five or 10 percent or something like that chance of ASI being a problem in the next couple decades and I think that's definitely the case there's enough evidence pointing that direction if I end up being wrong also that would be delightful right A lot of people who are concerned about this issue it's not because they really want to be I think yosua Benjo said something I would love to be wrong about this I would love to have free time and spend it on other things but I just haven't heard that many good arguments because there's so many things pointing in the direction of risk again not definitive problem but risk well I think I might be setting the bar low for our species here but it seems to me we haven't used crisper to make a bioweapon we haven't blown up any nuclear bombs in several decades can we learn any lessons from these existential threats that we could carry over those are great examples and we haven't although if you look at the history of nuclear weapons right we haven't and that oh this nuke went missing and there's this other one that kind of just went rogue or this accident almost happened and thing El almost detonated we'll say the research into different types of pathogens they often escape a lab that they're never supposed to escape so I've seen enough of those examples to be a little concerned because history indicates humans will not be as rigorous and reliable in their safety protocols as one would expect them to be but to your point it is the case that we've managed to not kill ourselves with nuclear weapons that said We've Come Close with crisper we have not yet managed to do this although there was that loose experiment we'll say with human cloning in China but the human sort of International Community seems to have gone against it I think it does make sense to put the measures in place if we're looking at the overall issue do we want to be overprepared or do we want to be caught off guard and like anything in life you kind of have to air on one side or the other and it's it's hard to get it right so my general disposition is why don't we be more cautious and prudent and make sure the multi-billion dollar companies developing the most powerful systems have some sort of Regulation control oversight to make things more safe then oops the powerful thing came out and now we've got a problem so I I take the point that we have many things that could cause a lot of harm that haven't in the history of humanity but that also seems a bit more based on Serendipity than good practices and structures it does seem that a lot of uh where we're going is inevit able maybe it can be slowed down maybe it should be slowed down but we're not just going to stop working on AI for whatever reason no no and I think it's important to realize it is an amazing Innovation that humanity is creating here that could cause a lot of benefits and to better shape that to have the outcomes that are better for more and more people and not just dictated by say the people running these companies it really is the case that there's an opportunity for us all to come together and put in regulations or to ask our political Representatives what are you doing to ensure that these products are going to be safe and useful why might there be such a risk what what are the other factors in play that lead AI systems to be at risk and I guess what I'm trying to get is that Humanity will integrate them because we want what they give us and then once they're integrated sort of like the internet it becomes very hard to get out of yeah I can't imagine getting rid of the internet well right and that's that so that could be a useful thing when people like well how would it be such a problem like well try to no longer use the internet or your phone and you're you're kind of now stuck in it so it's sort of almost separate from whether you think AI is a risk it will become part of our Lives not whether we like it or not but pretty much uh unless we are very cautious about it so I would imagine there's a balance of fear and optimism in your mind as you think through these topics where is more of the weight I would say I I do really just go back and forth between the two because I was focusing in the book on why this might be a problem that really had to take the majority of my time and in fact I I wanted to play with the systems more the fun side of it right because I was using image generators or even just playing around with chat gbt and whatnot to create things or know essays or rhyming poems whatever it might be on some silly topic like you know whether Superman's better than Batman whatever it might be but because I had to focus on the book I couldn't enjoy the stuff as much so in the in the near term I actually look forward to enjoying these products and Serv that the AI bring uh but because I think that good things kind of sell themselves when I'm engaging with others I I try to remind people that great we got a lot of good stuff that already exists and is coming on board but let's make sure it's safe so we can all have a better future good Vision I like that well this is definitely a contemporary topic how much background would a reader need to be able to pick up uncontrollable the threat of artificial super intelligence in the race to save the world they would need none at all it is specifically designed for people who are curious they're looking at the news or they're talking to people thinking what the heck is going on with AI should we be concerned what's going on it really is built for any interested curious reader you don't need to know what AI is you don't need to know any of these terms it's explained as easily and breezily and hopefully in as engaging a manner as possible and I really put a lot of effort into making it accessible making it hopefully interesting without losing any of the rigor and Darren where can listeners find you online uh they can find me on I guess Twitter at DBC mcke uh they can also find me on LinkedIn if they want and uh as you said the book's out and it's available on Amazon uh check it out if you're interested very cool we'll have links in the show notes for people to follow up Darren thank you so much for taking the time to come on and talk about the book and all your research well thanks so much for having me this is [Music] great