AlphaDev - DeepMind AI Discovers Better Algorithms for Foundational Computing

Key Takeaways

so how far can AI take us in domains like scientific progress new discoveries is it only capable of trying to optimize things that humans have already built or is it capable of creating its own new discoveries well let's take a look at one thing published by Google deepmind that seems to suggest that it's able to produce its own startling new discoveries so Alpha Dev discovers faster sorting algorithms now we'll look at what that means in just a second but it's important to understand what that these processes run probably trillions of times per day in our world right now so even a small Improvement a small optimization as it gets rolled out over all the technology all the computers all the little chips that we have across the globe that makes a massive difference so new algorithms will transform the foundations of computing so as we increased our demand for computing power we are kind of approaching how far Hardware is going to take us microchips are approaching their physical limits there's only so much that you can change in manipulate atoms before it kind of stops working and so they say in our paper published today in nature we introduce Alpha Dev an artificial intelligence system that uses reinforcement learning to discover enhanced computer science algorithms surpassing those honed by scientists and Engineers over decades so really fast what is sorting so sorting is a method of how we organize data it's something that's been around for a long long time you know it's things like alphabetizing three letters arranging five numbers from biggest to smallest Etc some of the earliest examples like this alphabetizing books by hand in a library and then 1950s and on now we have computer science algorithms doing the sorting and organizing Etc so for example here's a collection of random numbers it sorts it into one two three four five that's basically what it's doing now human intelligence and Innovation it took us really far of this contemporary algorithms to computer scientists and programmers Decades of research to develop they're so efficient that making further improvements is a major Challenge and so Alpha Dev uncovered faster algorithms by starting from scratch rather than refining existing algorithms and began and looking where most humans don't the computer's assembly instructions what's interesting here is you see some parallels between this and various AIS that play chess Etc so for example for the earlier AI chess systems we used to kind of give it some hints about what we thought is the way it should play for example we told it about how much each piece is worth so for example the pawn is worth one point the Knight is worth three points the bishop is also worth three points Etc now Alpha zero which is another sort of similar AI That's made by Google's Deep Mind it also started learning from scratch humans weren't influencing how to play so we didn't try to give it hints so we didn't try to say well here's what we learned so far it started from zero and it developed a different idea of how much each piece was worth so for example here are the piece values according to Alpha zero so we thought that this is worth one these are three The Rook is five and the queen is nine and certainly we were close but this Alpha zero that's by the way better than any human player on earth it thinks that it's a little bit different we're a little bit off and certainly who are we to question it so Alpha Dev began looking where most humans don't the computers assembly instructions and so they say here that we believe many improvements exist at this lower level that may be difficult to discover in a higher level coding language so for those who are not familiar basically a lot of let's say software developers use these higher level coding languages C plus plus python etc those go into the compiler and these use Assembly Language and what does that mean so that's basically a simplified language that's used by machines you can think of it like if you're talking to a dog you're not going to use full sentences you're going to say sit or fetch or something like that similar to this the computer is going to understand those simple commands so for example one might be mov for move move something from here to there or 80d for add add these two numbers and so the problem with it is it's a lot harder for humans to read and write but it's also a lot more powerful because it allows you to tell the CPU exactly what to do so Alpha Dev is based on Alpha zero our reinforcement learning model the defeated world champions and games like go chess and shogi with Alpha Dev we show how this model can transfer from games to Scientific challenges and from simulations to real world applications what's interesting here is they're making this AI basically play games similar to chess and go which is what it was trained on what it was sort of accustomed to doing and now making a game out of improving Assembly Language so to train Alpha Dev to uncover new algorithms we transformed sorting into a single player assembly game at each turn Alpha Dev observes the algorithm it has generated and the information contained in a central processing unit the CPU then it plays a move by choosing an instruction to add to the algorithm so instead of moving let's say a pawn to a certain Square it outputs this assembly code so for example mov for moving something right and it adds it to the algorithm so this is like Pawn to E4 and so as the algorithm is built one instruction at a time Alpha Dev checks that it's correct by comparing the algorithm's output with the expected results Force sorting algorithms this means unordered numbers go in and correctly sorted numbers come out we reward Alpha Dev for for both sorting numbers correctly and for how quickly and efficiently it does so Alpha Dev wins the game by discovering a correct faster program isn't it interesting how games are like the backbone of how we train AI all those hours I spent playing video games it wasn't a waste of time it was getting me ready for this and so these Algos are now available in the sort of standard sorting Library used by millions of developers and companies around the world so these little upgrades they do have a huge impact even if you do small incremental changes you know you improve it by one percent ten percent it's huge here they improved it by up to 70 faster for shorter sequences things like this could be run trillions of times a day if you're looking at the entire world another thing that was interesting about this is that I mean here's the code that it did so here's the actual output this is how the original was and the move that it tried that worked really well and so what's interesting is that they called it the Swap and copy move Alpha Dev skips over a step to connect items in a way that looks like a mistake but it is actually a shortcut so when humans look at some of these moves or some of these breakthroughs whatever you want to call them in the beginning they might think it's a mistake they don't understand it they think something's wrong something's off and they even compare this one to alphago's move 37. now there's this uh alphago the movie which is a pretty good thing to watch if you're interested in this stuff so it's free it's from Google deepmind it's on YouTube and so it's playing one of the greatest players of go here's that person he's outside taking a Break Meanwhile alphago makes their move and uh this move is weird this is move 37 what they're referencing in the blog post and people don't really get what it's doing notice how they're kind of like looking around they're not quite sure they're looking back and forth they're like is this is this no that can't be right this is this doesn't go there that's something's off it was a mistake that was a mistake what I see this move for me it's just it's a big shock what normally human will never play this one because it's bad it's a bad move it's a mistake but it's a little bit High yeah it's Fifth Line normally you don't make a solder here on the feet right um so coming on top of a forklife zone is really unusual yeah that's an exciting this is one of the developers saying an original move here one of the Google that's the kind of alpha Zero's development uh that yeah that you play Gold so he kind of hurries behind the scenes to see like uh what happened was there a mistake what did this thing do I wasn't expecting that um I don't really know if it's a good or bad move at this point the professional commentators almost unanimously said that not a single human player would have chosen who 37. so I actually had a poke around in alphago to see what alphago thought and alphago actually agreed with that assessment alphago said there was a one in ten thousand probability that move 37 would have been played by human player so it knew that this was an extremely unlikely move it went beyond its human guide and it came up with something new and and creative and different so it did it it beat the player I think it was something like 5-0 I admit that it was a bit of course this was a big deal because a lot of people didn't think that AI could be humans at this game a lot of people were kind of crushed there's a lot of emotions involved but what's interesting is that sort of move 37 kind of shows this sort of alien intelligence that this AI has it does moves that we look at and say well this has to be a mistake this is bad this is stupid and yet later only later we understand how brilliant and foundational and important that move was how pivotal it was we are artists you know we play ours best right for cool right so please gentle with solicitor it's very very good players great players I I'm in the room I see this is the Revolt win they try everything we just we can't I mean you can see house upset this person is this by the way is um so he's the CEO of deepmind Demis hasabis and so he's a really smart guy he was uh like the best chess player in the world under 13 when he was a kid there's tons of other things about him where you're like okay this guy's brilliant and so a lot of these new technological breakthroughs with AI he's behind it he's behind sort of the applications Alpha zero alphago Alpha fold which solved the protein folding problem which was like this breakthrough in biology like one of the biggest in 50 years with that specific thing that's now getting applied to genetics and the genome and all sorts of stuff so we're probably going to be seeing a lot more from Google deepmind and uh and this fella as well anyway so this is the move that it made that was so brilliant I mean I don't really know exactly what it is and that's one of the reasons why it's so difficult for humans to to change this because of how sort of tedious and meticulous you have to be to really understand all the intricacies of it and that's why it's so good for something like this to be able to optimize it because it'll make moves that we can't see another breakthrough was um in hashing so hashing is a fundamental algorithm in Computing used to retrieve store and compress data so Alpha dev has demonstrated its ability to generalize a and discover new algorithms with real world impact we see Alpha Dev as a step towards developing general purpose AI tools that would help optimize the entire Computing ecosystem and solve other problems that will benefit Society Alpha zero could start in the morning playing completely randomly and then by T would be superhuman level by dinner it will be the strongest chess entertain has ever been so that's that's Demis has zombies again CEO of deepmind so he's like the super smart guy behind a lot of the best entity has ever been after about eight so if you didn't catch that so it can start as kind of good by breakfast and by afternoon the best chess entity that's now as it was strong enough to be able to go out and defeat stockfish the incumbent world champion a program which was vastly stronger than deep blue though program which was previously defeated cast Prof so I called up my longtime friend Matthew Sadler and Natasha Reagan my two friends from when I used to play chess myself yeah so I knew that they were great things and it did cause a big stir actually amongst the chess players these were very exciting games very attacking games I could see that Alpha zero was trying something different like this young kid from deepest Russia is sort of arriving and then suddenly beating everyone it doesn't have an engine like style plays like a human old fire so I'm curious what everybody thinks about this because there's still people out there saying that this is just you know Chad GPT is just a bunch of scripts that it's reading off that some of the stuff is just a bunch of databases they're trying to reduce it to just something that's very like logic based like if this than that but it really is beginning to get harder and harder to try to Define things like intelligence and reasoning and creativity and Innovation it's harder to Define them in a way that sort of like includes the things that we humans do and excludes the things that Ai and neural networks do it seems to be like there's just how more and more of an overlap between the two and now we're seeing things like AI that's coming out that's being trained on the human genome which DNA is basically this vast data that we kind of don't really know what a lot of it is we know certain parts here and there we're able to modify it here and there but it's we're barely scratching the surface and these these AIS these neural networks their whole the thing that they're really good at is sort of taking these vast quantities of data of just raw data and trying to figure out some patterns that we can't even see I mean that's what the protein folding problem solve showed that's what Alpha zero alphago is that's what this Alpha Dev seems to be doing a lot of smart people in the space are saying that at some point sort of human technological progress the amount of stuff that we're contributing to it it sort of stops and it gets continued by these AI systems you know question is like what point are we going to be giving Nobel prizes to AIS instead of humans at what point is it going to be responsible for the vast majority of scientific progress that occurs how close are we to that point I mean at what point do we kind of maybe not become obsolete but what at what point is it doing a lot more Innovation and creative thinking and technological progress than we're capable of that they might not be that far away anyways hope you enjoyed it subscribe for more awesome AI content join me in this AI Revolution we're witnessing something amazing happen it's the arrival of General artificial intelligence and I hope you stay with me for this ride my name is Wes Roth thank you for watching