S* for AI CODE Generation: Plus 100%

Key Takeaways

hello Community today we want to increase the AI coding performance at least 100% we call it SAR so let's have a look you know you have your local computer I don't know you have a GPU let's say 16 GB of v and you want to use here this typical code llm so to help you to code and you want to do it locally on your machine imagine you would have here a Q and coder that has a normal performance here of about 20% and you want to say hey can I at least double here my performance and yeah if we have here a particular s star methodology you can right jump up here to the normal performance of a coda of a 14 billion free trainable parameter so from a 3B to a 14b model and you have almost the same performance and this this is really something because if you if you are limited here on your local compute this is the way to get a better better performance without going to the cloud so what we do we run here code 3B model with s star and we have a performance of a code 14b model now how do we achieve this this is easy you remember we have test time scaling TTS and now we applied for code generation and you remember we already open the video now and we have here a star and this is the first hybrid test time scaling framework to improve it accuracy of eii generated code but you know the principle are identical to what we had in mathematical reasoning because you remember test time scaling it was extensively and we discussed it multiple videos here to improve you the causal reasoning the mathematical reasoning and we had more or less three ideas now that we implemented the parallel sampling that increases your the solution coverage the evental refinement that improves your the individual sample to the resyncing and revising and my last focal point was here reward models that guide here the search process and the search space much more efficiently talking about one of my last mod we have here three insane TTS models that I showed you here even with a new Transformer architecture to have this for a language model in this video we discussed it here on Vision language model so anything in robotic or anything that you have with this computer use where you simply have or the AI has to understand your computer screen and click on what button this year we have a look at the latest reward Vision language model and it was called arm up and in detail here 48 minutes I explain you everything about this methodology and now comes the step now we move to the third area and this third area is code so what is here our spark of Genius now this is easy this is is here publication from UC Berkeley great team beautiful idea February 20th 2025 published here and they tell us here SAR test time scaling now for code generation AI code generation they tell us here this extends here the existing parallel scaling Paradigm with a sequential scaling to push you the performance boundaries of eii code generation so finally we have test time scaling also for our code optimization now here you have not a complete visualization of the Performing gain that you can achieve you see the higher you go and when we have model that are already inherent with the rising model like an o1 mini the gains are not that massive but you see for the Q Coda we had an 80% standard performance and then we got more than double the performance now if you really go to this rather huge unfortunately propietary model like 01 and 03 you see dat increase is not 25% but at least around 10% which is also impressive for a model that has such a high performance so you can apply this here for the non-classical reasoning model for the normal models that we have then for the R1 reasoning model for the qw Q32 B preview reasoning model for the o1 and O3 reing models and whatever so this is a methodology you can apply and this is just great so let's come to the core idea and the core idea is simple if you remember what we have done for the language mod and for the vision language mod for all the robotics now guess what we do more or less the same we have a two stage a two face approach in stage number one what we do we have here yeah a problem description let's have a look at this so give him a positive integer number represented as a string return the integer number without trailing zeros as a string so if you have to input this and we have here some trailing zeros just get rid of the trailing zeros and we have the output and we have here some public test that you see beautiful and now what s star does in stage one or phase one is just generates here parallel samples so as star enhances here the parallel samples to an iterative debugging beautiful so this is the beauty with code with a debugger we immediately see where is the mistake each sample is tested using your public test cases executed via an interpreter with outputs and the error messages used here to guide the next round of the sample generation so couldn't be easier with iterative debugging going here multiple rounds you define the max round that you want to invest your time Budget on great and then stage two phase two simple EST star select the best sample by prompting an LM to to generate inputs that differentiate a little bit between the pair samples and then leveraging it actual execution result to inform the llm to determine the optimal choice you see this is in code so easy because iterative debugging then we have our interpretor and we immediately know hey is it working yes or no do we achieve what we set out to do yes or no couldn't be easier now I give you a complete code here in a minute it's ready for you you can do it immediately yeah let's talk about the facts facts is here and I have here from the study just want to show you three Benchmark data you have a q and2 a 7B instruct plus here this new s star code Improvement this 7B now outperforms a 32b instruct model on life code bench by quite interesting amount or if you go with gbd4 Omni mini and as star this s PES here the o1 preview now mind on one preview nobody uses this anymore but also if you have open reasoning mod to achieve performance competitive to stateof thee art proprietary closed models I show you dpse R1 distill to q1 32b with s store comes close here to stateof the-art the openi 01 High model so you see the achievements are there and they're really impressive and especially if you want to act locally or even if you're in the cloud hey if you get a 10 a 20% boost in your actual in your accuracy performance for your code generation this is something you should use so this is it more or less let's have a look at the end just want to show you here some beautiful ration studies here and they looked hey if I look here never mind whatever model it is but if we just have here the sore generation and then the selection what what is the most important action here now I told you this is a two-stage process so from the first one you see we got a 6.7% Improvement and the second is 13 see interesting the second one is so much more interesting now and here r one is still 14b we have a plus 3% and plus 16% so you see this second one is really interesting so this adaptive input synthesis is what you could call the the core the performance core of s star what does it mean in detail now for each pair of samples here and llm is prompted to generate here distinguishing test inputs in this inputs and of course executed we in Python here where the outputs are further provided to ground to llm to select you the best samples so it's rather easy know this adaptive execution grounded approach with the code ensures here really a robust identification of the correct solution so you see this is interestingly here for code a really important step yeah as I told you we do have a GitHub page Nova sky sky sword beautiful remember this is the same team from UC Berkeley that give us here this specific Sky T1 this was similar here to the 01 model to the 01 preview model here you have the GitHub bio beautiful and as you see I'm really early because just one hour ago sky sword they updated here the code for the S star approach so you have everything available here if you go there and you saw okay what is Nova sky or Sky sour whatever different teams from UC berley I show you them in a minute here you have here the beautiful re-release as star so February 21st 2025 code paper simple extended with test time scaling framework for code generation it really works with quite a lot of model I would surprise if you find a model where it's not working at all your Pyon notebook jupyter notebook everything is there for the team I don't know if you notice you see berley C Computing lab real nice you have the all the people this is just the the first thirt of them so you have the faculty core faculty professors and Associated professors here and faculty and then yes beautiful you could scroll down two three times to see all the student and whatever so amazing team great code this is here if you have a look back where is it Apache 2 license so it is for you why not use this I just recommend if if you use this because go and upgrade your local code llm or maybe even then in short time some code Vision language model to have a more physical understanding of the physical surrounding of a robotic system but this would be the content of our next video If you like this kind of videos hey why not subscribe and I see you in my next video