Mistral just announced Codestral Mamba and Mathstral | *HUGE POTENTIAL*

Key Takeaways

hi everyone I am Elvis and in this video we're going to cover a few of the announcements from M AI so they recently announced two models mat Str and cod STW Mamba so we're going to go through what they announced some of the details and what I'm going to do in this video as well I'm going to go through a demonstration of how to use the kosal Mamba using their apis so let's get to it so the first one is kosal Mamba we can see here that kosal MAA is their latest model for COD generation so this particular model has achieved some results are very comparable to their previous cultural 22 billion parameter model and specifically this one is focusing on the Mamba architecture this is the type of architecture that the community is leveraging today to build faster language models so faster in terms of inference these are competing with Transformer models now so it's great to see that mishell is actually releasing a model that's based on Mamba specifically a code generation model where it could really unlock a lot of good use cases our own code generation they don't really give you a lot of details here but they're saying that this one is based on Mamba and they also present some results here so you can see the comparison with with other code specific models like deep seek code Gemma and code Lama you can see the performance here on human Evol which is is a widely used code generation Benchmark this one as well is highly used and you can see the Gap there right this is a 7 billion parameter model and this is a 22 billion parameter model and these are the instru models so these what you see here specifically these two AR INR models as they mention here at the bottom and this specific model is you know 7.2 billion parameter model they say that Coastal Mamba was tested on in context retrieval capabilities up to 256 tokens now I saw the documentation says that it's 256k Max tokens so that's the support that they have here in terms of tokens and they mention here is could be a great local code assistant so I can imagine a lot of companies could leverage a model like this that has faster inference and also has those code and reasoning capabilities that you will need to build code generation tools or even tools that are already enabling some form of code generation capability right so we also covered a few of these tool tools like the Jupiter pretel tool and also the other tool Saturn where it's supporting openi code generation models but it be great to have a code specific model like this enabled there and we can leverage it for all the error fixing code generation and all these different functionalities that we need for those type of assistant tools performing on par with sort of Transformer base models as we can see from these results now I think the best way to know how good this model is is by actually testing it they made it available in different ways so you can use the Mell inference SDK I believe you have to download the mod for that and there's also a way to deploy to tensor rtlm as another popular way of deploying mods you will eventually be able to use it for local inference with support in Lama CPP and they also have the weights available on hugging face you can use cultural M as well on lab platform and this is available as this model here so we are going to go through a few demonstrations or few examples of how to use the model itself and just testing the capabilities that it has and some more details here about the license kosal Mamba is available under Apache 2.0 license so that's it that's there's not a lot of details there I didn't see a technical report for this we are going to cover a few examples of how to use it in this quick demonstration what I want to do is I want to show you how to test kosol Mamba model the one that was recently released by mol you can install the client first then you install python. M this is very standard for configuring the API so you have to set this API key right you get an API key using Li platform and once you have that set up then we can test it so the first thing I want to test again we're using the codal Mamba 247 that's the new model right that's the model we're using and this stuff is very standard it's very similar to like the open a models it is an instruct model we can just prompt it like this where we set a rule in this case user then we write a Content this is going to be the prompt and you say write a function for Fibonacci and then the standard request is right here so here is the code and I did this in the notebook here because I really want to test this right away and I'm just going to test it see code add a piece of code here and what I'm going to do is I'm just going to test it here I'm going to say then okay so it looks like it's going to return yeah the number at that particular position so if we do again another test okay that looks correct or yeah there we go so that's the Fibonacci for you and again it Returns the end number in the Fibonacci sequence it's going to be the end number that you specify here we're going to do another test here in this one write a python function that multiplies two numbers and add a constant of one to it so I love to always test this this is one of my first test that I do on code completion models and we can see here it has multiply and add one so I always look at the name of the function that it's using okay that looks very standard and then we have the result and then it returns result plus one as well sometimes we see that these models prefer to use like num 1 * n 2 plus one it adds it in one go here so that's more for conciseness but I guess this looks okay and it also gives you some code example for how to run this particular function this looks okay and it even gives you the output of this particular one so 5 * 3 15 + 1 16 so that's great to see you can see that this model is very capable of generating like these python functions now a test that you can try specifically maybe on a programming language that you're familiar with I not sure they didn't really mention what programming languages they're supporting but you can test on different programming languages to see how robust it is for different programming languages now I'm not going to do that test here I'm going to focus on python I'm more familiar with python so this one is WR of python class that performs matrix multiplication a bit more complex I would say but I'm very curious how it go about performing this particular task okay so let me just go here some models for this particular task they tend to generate very long classes this looks super concise but now I'll test it out and see how it's performing if it actually is running it gives me also some example usage that that's nice I'll just copy this and then I'm going to paste it here very interesting that it also gives me the the output so I can cross check that let me just check this um so you can see it's 1922 4350 it seems to be doing it okay and this particular class works and you can see that the usage example also works as well this is great for doing documentation so if you had like classes and you wanted to generate documentation or example usage for that that I think that would be really nice to use it that way so this one is right a python game without any library that lets me guess the number between one and 100 and gives me clues so a bit harder than the other tasks but here I'm looking at you know the functionality of that particular program that it's going to suggest let me just hit enter here all right so it did generate some code again it's an instruct model so it's giving me some explanation as to how I should use this particular program you can see at the bottom here to play the game and so on so I'm not going to do all of that since I have this already on my notebook I can just copy and paste it right here here we go I'm just going to run this hopefully this runs all right so she says I'm thinking of a number between 1 and 100 try to guess it so it prompt me for numbers at the top so I'm just going to say six too low I'm going to say 60 too high going say 40 too low I'm going to say 50 too low want to say 55 there we go I guess it in five attempts pretty good at this game um but yeah that's that's cool it gave me exactly what I asked for which is to give me clues so these are the Clues and then eventually I guess the the number um and it's working that's the most important thing right the code is actually working now I don't know about the actual code I'll need to check to see you know how good the code is I don't see commments in the code which is something I would have liked in the generation of the code and that's not something I should have to specify that should be automatic right that's the only thing I would say but besides that I think this is great and I'll just keep testing this if you're interested in me going through a deeper dive into its capabilities this particular Cal Mamba M since I have it here already running and I'm already starting to test it uh let me know in the comments and I'll probably do a longer video of that to test it out the next release Here is masteral so this is a mat specific mole so we have a code specific mole no we have a mat specific model um and that's interesting right because I know Mell has these general purpose models like the Mell model that they are releasing they're family of Ms that are basically more general purpose for all sorts of tasks but these weren't our specific and I think what they're seeing is that a lot of like applications and products are using customized model so if it's for code generation for instance for like maybe IDs that need some code generation capabilities out of these llms you would typically use a code specific model and that's the same thing for matal I guess so for this model what you want is you want really good at Mat capabilities right problem solving uh multi-step logical reasoning as they say here and what they mention here is that Mall stands in the shoulders of michell 7B and specializes in stem subjects now I don't think this one is using Mamba there's no specific uh mention of architectur changes or anything like that but what we get is it's probably based on the Mel 7B apparently this is also a 7 billion parameter model and you can see the performance here the deeps mat is something that I covered in my YouTube recently and this is a very powerful model for matte zseek also released one for code as well you can see how good deeps is and that's a good comparison that they're doing here with this particular model but this is a 7 billion parameter model right so you can see how it all performs most of the other models we also have another competitor here qu 2 7B which you will see how it's performing now this particular Benchmark something I'm taking a closer look at because this is a really hard task and I think with upcoming models we'll do more in dep analysis on this particular Benchmark cuz I think this one even the opening a mold struggle with this but even the latest general purpose models the bigger ones struggle with this particular task so we'll take a look at that closely later on but these are the results for now you can see on mat how it's performing it's performing 56.6% I think in one of the announcements they say or mentioned that this particular model op PR firms minurva the I think 500 plus billion parameter model from Google Mall can achieve significantly better results with more inference time computation Mell 7B scores 78 30 7 on mat with majority voting and 74.5 N with a stronger reward model among 64 candidates so quite a capable model on mat reasoning and this one isn't available for testing with your apis yet I guess probably they'll make it available at some point but it's available on huging phas the ways you can try it as well with the missal inference SDK and you can also find un name which is also great right with these custom models you want to maybe do some more fine tuning on them this chart here I guess shows the breakdown of the results how it's performing compared to the Mell 7B and these are kind of the boosts that you get on the different categories so you have like elementary mathematics apparently there's a huge boost in elementary mathematics by these results I think you can see that these custom models can be more powerful than their general purpose counterparts that will be it for this video I'm excited about this particular release I've done a few recent videos on these custom models I think custom models have a lot lot of potential especially when you are building some type of experience or application I'm seeing more adaption of these custom models in different places in fact we are also designing applications where we leverage custom models and combine them with general purpose models like cloud and these Gemini models and even the open a models as well so that'll be it for this video thank you so much for watching consider leaving a like And subscribe to the channel if you haven't see you in the next one