MIXTRAL 8x22B INSTRUCT and more!!!

Key Takeaways

so the French Open Source llm company mistol has launched three different things and in this video we're going to understand all the three different things the first thing starts with a very brand new model the second thing is a tokenizer and the third thing is a new python library that would let you work with mral models to start with the first model is something that they announced through torrent couple of days back probably a week back but they have launched the instruct version of the model so a few days back mral launched a base model mral 8 into 22 billion parameter model it's Ane mixture of experts model but now they have released the instruct fine tune model of the same one so what is this model this model stays right at the top of mlu so if mlu is a benchmark that people still use that if you see MML U you could see mral 7 billion parameter model here mral 8 into 7 billion parameter which is another mixture of experts model now mistal 8 into 22 billion parameter model is right at the top doesn't take a lot of cost in terms of the active parameters because it is Ane model I'm still not sure why people compare it things with llama so that's that's uh that's a different thing but you have got command R by coher and command R plus by coher and it's a very interesting choice of models that they've compared leaving the other models out but if you just see the open models with open completely open license or restricted license you can see Mixel 8 into 22 billion parameter model is a Nob brainer for anybody to use why is it so because this is a model that is fluent in a bunch of languages including English French Italian German Spanish the model has got good math and coding capabilities which can be very well understood from uh the benchmarks like GSM 8K and mbbp human eval in all these benchmarks you can see once again I'm not going to take Lama 2 70 billion parameter model but when you compare it with Mr 7 billion parameter model which is still the model that I recommend to a lot of uh let's say microsof creators because this is a model that is really cost efficient doesn't require a lot of compute and it can run on like 4090 3080 and all the consumer grade gpus very well with the quantized version does really well when you compare it with mral 7 billion parameter model the 8 into 22 billion parameter model are 8X 22 billion parameter model has got 41.5 it's almost like more than like 40% improvement from the mral 7 billion parameter model thanks to thee architecture which is spars mixture of exports where 39 billion parameters are active one for one token so every token goes through two exports you've got eight experts every token goes through two exports and that two exports is what you see 39 billion parameter model this helps you to maximize the knowledge or like from whatever the model has learned while minimize the compute that is required so which means you can fit a lot lot of things in single GPU lot of different advantages that we have discussed aboute in the previous videos we've got human evil which is completely rocking mbbp which is like a python test which is completely rocking you've got GSM 8K um then you have got GSM 8K with uh eight shot and youve got math the other Benchmark so in all these benchmarks you can see mistal 8 into 22 billion parameter model really is lot better than the previous Mr models that Mel released once again it is very surprising for me that they did not compare this with the mral large model that they've got they did not compare it with mral medium model that they've got um I can understand that they wanted to compare it with only open models but again it's a very strange Choice when you have got a lot of models like let's say Gemini um probably use CH gbt gbd4 and a lot of other models that we have got like Quin so a lot of other models there but they've made a conscious choice to compare it with only these models but when you compare it with these models as well you can see that this model is a pretty good model for example if you take command r+ which does not let you use the model for commercial purpose even when you compare it with that model that has code 70.7 on um GSM 8K 5 shot while mistal with 39 billion parameter active parameters it scored 78.6 a similar thing that you would see for other benchmarks as well and and this being a French company and heavily invested in the EU European Union region they've got a bunch of European languages covered and you can see across all these European languages uh they are pretty good in terms of the benchmarks the standard benchmarks for reasoning and knowledge you can once again see mral 8 into 7 8 8X 22 billion parameter model is way above all the previous mile models and a lot of different benchmarks there is an improvement in mlu there is an improvement in hsw there is an improvement in natural Qs Tri QA and all these benchmarks now the main thing that I'm quite excited about this model is not because of the bench marks I mean we know at this point that the iPhone released with let's say like a 5 megapixel camera and the new iPhone is going to release obviously with 7 megapix camera I mean maybe Apple would not do that because they are Apple but you know you get the point every new model is an improvement from the existing model but what kind of improvements that they are bringing to the table is what I started looking into it not NE necessarily uh 10 point Improvement in mmu or 20 point Improvement in hsag I mean that is good I can understand that is good but that is not enough for you to make a name in the llm world these days I would honestly say that Mixel 7 billion parameter model is really good for a lot of different tasks now what matters is when you take this models and you want to build agents what matters is when you take these models and you want to build rag Solutions Enterprise Solutions when you want to actually use it in real production use cases and that is exactly where I think this model is going to shine a lot why because this model natively supports a function calling so function calling uh simple function calling 101 uh large language model typically responds you back in text but can you make the model respond back in a Json what kind of Json a Json that can actually call a function so the Json that is coming out of the model can be the arguments for a function so that you can call a function for example you want to make a large language model and that should call a weather API so what kind of arguments that you might need you might need the location probably that's the most important name like the type of temperature Celsius or fenit so these are like two important arguments that you would need so can you make the large language model output this so that it can call a with API it can for now you can extend this for email you can extend this for a bunch of other things and that is how truly you can put together a slightly more sophisticated an agent system and this model according to them is really good capable of function calling along with the new 64,000 token windows so there are two things that I absolutely love about this model one function calling seems they're saying it is good I've not tested it the second one is the model is capable of 64,000 Contex window which means for rag retrieval augmented generation this model is going to be be really good because it is going to have a high level Precision SL recall when you use a lot of documents give it to this model so for Rag and also for function calling this model might be good now that is a really good Segway for you to go to the next section which is to help support function calling they have done two things one they have introduced their own tokenizer and they have also introduced mral common which is a python library now if you're not familiar with tokenizer tokenizer is the most fundamental thing about large language models anytime you do NLP natural language processing or machine learning with text tokenization is the most fundamental thing if you do not know anything about tokenization would strongly encourage you to watch the latest video about tokenization by Andre karpati none other than and karpati I link the video in the YouTube description for you to easily find it out so what happens is when you want to build a large language model you take a lot of text like take books of books from libraries the first thing what you would do you would not like use all the books you would just pick one of the books and you would take one page of the book you would go to one paragraph you would go to one line you would go to one word and you would read this is how humans would read accumulate knowledge the same way large language models accumulate knowledge by tokens and this token is important a lot of different things like for example how the model understands whether the model um learns different things and also at the same time this might have an impact in the cost later on when you have got a token based pricing so they have introduced their own tokenizer uh here and uh also the reason that they are saying the reason they have introduced their own tokenizer is to have additional special tokens what kind of special tokens special tokens for different sets of models so there are three versions of tokenizer the first version of tokenizer supports the open m 7 billion parameter model the 8x7 billion parameter model thee and Mixel embedding and then they have got the I don't do they have their own embedding I don't know if it is a medium model or embedding I don't I don't I don't I couldn't recall whether they have their own embedding my bad the version two of the tokenizer supports mral small latest model Mr Large latest model the version three of the tokenizer supports Mr 8X 22 billion parameter model which we are discussing in this video and as you can see it turns out that the to Tozer can support function calling you can see what are the available tools and what kind of calls it can make I'm not going to go deep into the tokenizer but you can see the the normal regular tokens you can see then you have got the tool calls available tools available tools tool result tool result so it makes this model to be a compelling case a compelling model for function calling that's what this actually does you can see here um and you can see like lot lot of more information on token call tokenizer and how function calling is being used here and you can see the example code and the next thing is that they have released a python library that is called mral common and you can do pip install mral common and what is the reason once again this Library can help you with uh first the chat template still there are a lot of discussion with you know what is the right chat template for a large language model where should the system context go where should the user message go where should the assistant respond go how do you store the conversation history this there is no standard uh format people mostly started using the chatl format that was introduced by opena but this is an evolving case so Mr has coming up with their own thing by using with this helper function called Mr common uh do protocol instruct messages then they have got the request and Tool calls where they have function and Tool uh classes and they have got their own tokenizer so the three things that Mr introduced today the one the Mr 8x 22 billion parameter model which is a sparse mou model so it has got 39 billion parameters active for one token for every token and this model is one of the best the best open source model open source open weight model that is available today you can access the model according to them you can access the model on their Le platform a um which is their own developer APA platform the second thing is they have introduced their own tokenizer which I've not seen a lot of people doing it but this is really good especially because they dealing with foreign languages like French uh Spanish which might have like different characters um this might be really good step to reduce cost rather than using other tokenizers which might be not doing proper tokenization for other languages like I said if you want to know Advanced about tokenizer I would strongly encourage you to watch Andre kpa's tokenizer video the third thing is they have introduced a python Library called mral common which is helping you to do function calling and all the other stuff like Leverage the latest toiz when you are using myal models it's like a new library that can help you make use of the fullest of the mistal models now to before I wrap up the model is also available on hugging faces model Hub you can straight away go start using it but if you were to use the tokenizer sorry if you were to use the function calling it is little bit different from what you would probably regularly do so um I would strongly encourage you to check the documentation but otherwise it's an apachi 2.0 like licensed model completely open you can do anything with the model this should probably increase uh the base model is already available and people have started fine tuning it I would strongly encourage you to check out other fine tune models that are coming out of mral 8X 22 billion parameter model and once again thanks to ml for advancing accelerating open AI development and see you in the another video Happy prompting