Code is for Machines - Not for Humans to Write (MIT)

Key Takeaways

Hello community. So great that you are back. We have a brand new research and it tells us you know what code is for machines but code is not for humans. Not anymore. Why? Now remember 2016 Alph Go. We had a policy network and a value network. And now 10 years later you would say what we have now currently we are on the verge of a revolution in the scientific discovery. And you know why? Because Google, MIT and Harvard in their new publication they showed us that they turn the art of programming of coding into a game and you say ah so the old style today many of us are in a dialogue with our AI code editors. We prompt get suggestion and then we tell the hey do it again AI try a different approach optimize this parameter and we are playing already the game. Huh? And the AI is our incredible assistant. But guess what? The human is not needed in this loop. So let's play a code science game without a human who just presses do it again button on the keyboard. So here we have it here. Beautiful. This is here 20 2599. An AI system to help scientists write expert level empirical software. Science software. Google Deep Mind, Google Research, Google Platform, MIT, Harvard University, Google Cloud, McGill University, California Institute of Technology and a lot of beautiful, highly intelligent authors of this paper. Now, what do they tell us? They have now built a system that AI system stop being simply an assistant for us in coding and AI now starts to be the player itself. So the system doesn't wait for a human prompt. It tirelessly explore thousand and thousands of potential solutions given a specific task and trying to code scientific code for scientific experiments. And it is guided here by a single imperative if you want in its reward function. Just win the game. Win the game like an alpho 10 years ago. Now 10 years later, finally, finally we're able to implement this. And here you have it. It's as simple as can be. There's nothing new to this. We have problems. We have researcher ideas. And you know, you can have a human expert write something or system just goes here on the internet. So scientific papers, archive or in the memory there were some prior ideas. You combine this or you have a deep research by Geminy or any other AI system. You have here your data collection and then you generate a prompt. You have an LLM writes a code code execution in a sandbox and then you just try out thousands and 10 thousands code sequences. And you know what the intelligence is just in the search algorithm and it's a tree search that we know now for centuries. Nothing new. Here you see the result in bold. You see here the new um algorithm they outperform currently everything else. So what is the idea? By framing now this new scientific software creation for real scientific problem as a scorable task. Say hey listen from 1 to 10 this is a 4.7. They have turned scientific programming now into its own game of gold. They can now apply everything they learned now to this scientific programming. How is it done? Easily. You know this an LLM acts as a creative engine proposes your novel moves you on the goboard or code changes code mutation with a genetic algorithm. And then we have a strategic tree search algorithm that decide which line of acquiring are most promising for the next node exploration. You know what this means? This means we still do not have the intelligence to find an simple intelligence illusion. But you know what we have now? Unlimited access to all the GitHubs that were created by humans or already by AI and humans. all the GitHub in the world, all the repos in the world, and we have almost unlimited compute capacity. If you're one of the global players who were investing tens or hundreds of billions of US dollars for new compute centers, data centers all around the globe. Now if you think about this and please read the paper this refraraming is the brilliant idea itself because what it does is on a deeper level it converts a complex open-ended scientific goal. I want to understand this experiment. I want to have a computer simulation of this scientific experiment. It converts this into a formal mathematical optimization problem which is exactly at what our machines our probability machines excel at and we already discovered if you want new mathematical algorithms exactly for those task with those machines. If you would have a further deep dive I would recommend this video here for you. So let's have a look what is now the engine of this brand new methodology of this brand new idea. It is what you already know. There's nothing new. It's just a reframing done in a beautiful way. And LLM proposes here a new code solution. After reading thousands of AAV papers and all the repos that are combined here with the arrive explanation, scientific explanation, the code is then run in a sandbox to get a particular score of course judged by an LLM and a tree search algorithm uses this score to decide which solution to explore next in the code mutation process. So this LLM, so its task is to generate just a new version of code that might achieve a better score at the end. and it can generate thousand code sequences, 10,000, 100,000 code sequences. It doesn't have to be intelligent. It just has to be fast. So this iterative refinement that you see here is far more powerful than any oneot generation and drag and whatever as it simply allows the system to build upon its success and learned from its failure. And of course, it's an agent with memory and everything else. So now you might say wait a minute but the solution space is with code almost unlimited you cannot just go and trial and error absolutely so the system needs a strategy to intelligently quotation mark navigates now this immense space of all possible program code sequences and this is here where this beautiful and simple idea of a tree search comes in and inspired from the alph code Google now and Harvard and MIT have now a new methodology booked predictor and upper confidence bounds applied to research. Now you know upper confidence bounds no this is this is almost all I don't know 9 10 years old no predictor ah that's so easy look this is all the mathematical formula that you need this is it and if you're a little bit into AI you say hey I know this no this here the first term is simply the exploitation term it's the normalized rank of the notes quality score and you say then the second term guess what this is the prior here. So this here is the exploration term. The bonus n is the total number of explorations so far. V is the number of times the a particular node has been visited. We have a hyperparameter to control here the trade-off between exploitation and exploration. But this is from you're familiar with this. This is nothing new. If you want to see this, if you want employer code, if you like to have here an upper confidence boundary search here, this is exactly the same thing. Easy. And then I said, but wait a minute, this can't be there. There must be something different. No, so what is if you go a level deeper with me, what is it? What is the genius idea here? And I think it is a flat prior in our statistical analysis. It's a formal way of declaring an initial impartiality. The system doesn't judge on a policy network which node to start with. So there's no a priority judgment which of the existing code solution let's say just downloaded I don't know 100 GitHub repos is a better parent for the next code mutation. You know what it treats at the beginning in the start phase every node in the tree as equally good starting point before it considers here their actual performance in the mutation. Now remember alpha go we had an inform prior because it was only a 19 * 19 board. So it was a finite set of possible moves from any state and we had a neural network a policy network to predict you the best notes. Now in this scientific software it's not possible because the state if you want is a complex code base and they move in those space almost an infinite set of of possible semantic changes to that code. So we can't just go here with if you want a policy network. Now so what is the solution to this? As I told you a flat prior, this is the initial starting conditions of the complete system that completely sidesteps this impossible task because it requires no domain specific knowledge and you know it's universally applicable to any scorable problem where an LLM just gives a score to a certain linguistic complexities and they decided you know what we try to apply it here to six different categories from general economics through numerical analysis and starting now with an impartial prior. The system places all the burden here of guiding here the complete complex search into the two other terms in the pocket formula the exploration and the exploitation term. And it is this exact mechanism that allows here to if you want transform the winning ideas of Alph Go, the UCB search principle out of Alph Go into this new area of scientific discovery, the boundless unstructured world of scientific coding. Just to make clear, there's no policy network anymore with this. And it's complete unlike like alpha zero which uses your complex neural network to predict which moves are the most promising hit. We have we operated here 10 years ago with an informed prior. No, this is not necessary anymore because we have let's say quotation mark unlimited compute power. So this new puck search bootstraps itself entirely from the empirical scores it receives. Of course, if you have an intelligent start condition, it has a much better convergence. But in theory, you could start anywhere. It doesn't matter anymore. You just have to compute power. So, what do we have? We have a code mutation system that is now a mathematical optimization problem. And this is now science for an AI system. And you just need compute power. Now we talked about here genetics. So let's go with genomics. They have here six different areas in the paper detailed analysis. I just give you here the headline. So they went with genomics and they had a beautiful result. If you want to have a deep dive, this is just a screenshot of one of the results of the paper. They went to geospatial analysis and they had a great result. And they went to numerical analysis solving here difficult integral systems. No. And they had a beautiful result. So they showed it is working. And this paper was just published today and I wanted to jump to the code but unfortunately it is not yet uploaded. So whenever you will see this video please they tell us here code availability will be here. Go and play with the code for sure. I will do the same. So isn't this amazing? We don't need intelligence. We just need trial and error. This is now in a computer infrastructure. This is now essential for the scientific pro progress. No, both for the humans and for the automated approach that we just outlined here for the next AI systems. Those systems will generate expert level compute code solutions so extraordinarily quickly. It will remove here the discovery process from weeks and months where human have to use an EI code editor just to hours or maybe days. So it is quite a a breach here in our understanding you know it's quite a shift here because up until now we had 8 billions or maybe 9 billion human brains and quite some quite a lot of them had some brilliant ideas did research invented new things and so on and now with all those investments of tens and hundreds of millions of billions of whatever in new data centers around the world this is the new economy if you We just have almost unlimited compute capacity if you are a global corporation. Just read what Openi says how many more than hundred billion dollars they will burn just operating costs in their new data centers. And you know what is amazing? You know what is the consequence of all this? We don't need AGI. We don't need super intelligence itself. Because we do have a robust algorithm on an almost unlimited computer infrastructure. And if this is an algorithm that really searches here the complete mathematically defined solution space of our physical chemical biochemical system. Then there is no need for a super intelligent for an elegant solution because with absolute precision this mechanism will scan all of the available mathematical solution space and will find therefore all possible solutions. Is this the truth? Is this the way forward? What do you think about it? Hey, why not leave me a comment, subscribe, and I see you in my next video.