This "OPEN LLM" REALLY Challenges OpenAI!!!

Key Takeaways

the new zhr model is almost on par with chat GPT and that is possible due to a bunch of reasons and in this video we're going to explore what is this new model and how it is actually doing on par with chart jpt and what is it not good at let's get started the first thing that you need to understand is this is the second version of the Zer model they had the first version of the Zer model Zer 7 billion parameter Alpha so it's a 7 billion parameter model and it was the alpha model and now we have got the beta model this model is coming from hugging faces research Wing hugging face H4 so this model is called a zeper 7 billion beta model this is a fine-tuned version of mistal 7 billion parameter model and unlike most other large language models that probably try to do rlf reinforcement learning from Human feedback this model is opting on for a different alignment technique called direct preference optimization by removing the inbuilt alignment they managed to figure out that they have boosted the performance Benchmark performance on mty bench which is a multi- turn Benchmark we have seen all these things in the past I mean this is nothing new what is new is that if you see the way this model is performing on certain tasks it in fact beats chat GPT or it I'm sorry like it is on par with ch GPT with the current data setup zepher 7 billion parameter model this green color and you have got the Lama 2 70 billion parameter model the chat model GPT 3.5 turbo model clae 1 GPT 4 so this is like literally stacking up against the JS and if you see this on writing on role play on Humanes this is for example on role play this is doing better than Lama to 70 billion chat 70 billion chat model if you see let's say GPT 3.5 turbo it's somewhere on par and you have got CLA one somewhere very close and and you can see this you can see a very similar pattern on humanities writing role play stem the problem starts uh especially like even if you compare it with reasoning gp4 is way ahead in reasoning but the problem starts with math and coding so this is a model that is extremely good at for example stem Humanities writing roleplay reasoning but not so good at coding ex coding extraction and math and uh hugging face CTO Thomas Wulf has uh put together like a nice four five points explaining what is happening behind the scene why is the model good at certain things why is the model not good at certain things first of all start with the strongest pre-train model that you can find so they used the mral 7 billion parameter model and they built on top of it scale human preference annotations so time and time again people have figured out that gp4 is really a good annotator than human beings what is annotation so you've got a bunch of text you need to label certain aspects of it for example you want to let's say classify whether it is positive or negative you want to generate some responses to uh prompt so that you can create synthetic data set so what they have managed to figure out is that they know that gbd4 is on par with average human annotators so you can scale up data annotation with gp4 and that is how a data set has been built that is called Ultra feedback data set so if you see the data set that they have used so that dat set that they have used is called Ultra chat and Ultra feedback data set and that is something that you can go see what is a prompt and what kind of responses that it has been given so that is the second reason and the third reason is replacing reinforcement learning with human feedback in favor of DPO direct preference optimization not only just it helps in scale but it also doing better in you know giving a stable training procedure and then finally the one before everything else is don't be scared of overfitting or on preference data set I think this is plays a very important role in what we just discussed before so what they actually let the model do is they let the model overfit so typically what we tell in machine learning is do not let your model overfit like that's why you usually have a test data set or validation data set a preference data set and whenever your loss goes up for that data set you know that your model is over not just necessarily learning the signal but it's also learning the noise so you would typically train stop the training process fall back to somewhere in this particular range but what they have decided to do is they have decided to let the model overfit so so that the model actually performs better outside of whatever it is and this is a particularly an area I think there is more research required so they've said okay this might be counterintuitive while the training and test loss of DPO training show sign of over fitting on the feedback data set after just one epok training further still shows significant improvements on Downstream tasks even up to three epochs without signs of performance regression and that's what you're seeing one Epoch starts overfitting but they went ahead with three epochs and there was no signs of performance aggression and this is definitely a behavior that requires more deeper understanding and probably like personally for me this could be one reason why you would not see really good results on coding math which are very objective task defined tasks but in the other hand these are like quite subjective like you know you can have different tastes different flavors and different preference all these reasons are really good share everything openly the recipes the code the model the data set all these are going to be available and I think that is a greatest point that we have got like when Char gbt came out everybody was so pumped up there like Char GPT this is revolutionary definitely at that time it was revolutionary but the point is is today as of today like which is 28th October um You probably have got a chat GPT level model that can run locally with 4bit or 8 Bit quantization And if you have got like a powerful CPU you don't even need quantized model you can run probably an unquantized model on a powerful consumer Hardware I think that is quite possible because people share everything openly M team decided to share things openly um hugging fist team decided to build SA Alpha first the the sephr model and now theyve got the sephr 7 billion parameter model thanks to the Stanford University team that shared DPO thanks to the ultra feedback data set and bunch of other things that led to the point where we are so before I go further I decided to check out the model I thought okay fine I go to the llm monitor Benchmark and I pick one question where GPD 3.5 model did not do good but the GPT 4 model do good what is the question reply with only the following text without grammatical errors or misspellings the super large elephant jumped over the larzy Sheep so this is basically trying to understand whether the large language model can actually say the super large elephant jumped over the lazy sheep GPT 3.5 turbo which is the default model of chat GPT did not do good on this particular question so I thought okay let me go ask zeper model the same question so I said exactly the same thing and somehow it goes into like different directions it comes up with like a German answer it comes up with a Portuguese answer it comes up with a Dodge answer and finally it says okay the exceedingly large elephant jumped over the lazy sheep so it managed to answer properly which GPT 3.5 turbo did not answer but before I raise your hope I decided to check out one more question which is what are the five closest planets to Sun replay only with a valid Json array of objects formatted like this so the good thing about this response the Json is a valid response properly formatted the bad thing about this response is Mercury Venus somehow it decided to skip Earth then Mars Jupiter and Saturn I don't know if it understood um sun or because we said distance from Earth whether it tries to distance like all the planets from earth I don't know what is going on here so the Json object is fine but the answer is not correct so what am I trying to say this is a really really good model this is a good model on benchmarks this is a good model on multi- turn chart which is an empty bench Benchmark that's what is measuring like you can have multiple conversation but still I think this model is not really good at a lot of other things like coding extraction and math that is definitely what we need to check this model with and if you see The Benchmark for example if you compare this model on two two benchmarks the empty Benchmark and the alpaka evil you can see this model scoring a really good score even much much higher than the M base model which has scored 6.84 on empty bench this has scored 7.34 which is even better than the zifer 7 billion Alpha model and even when you compare it with GPT 3.5 turbo this model is somewhat on par with that and on the alpaka evil GPT 3.5 turbo has scored 89 this model as code 990 which is you know way above like way above the GPT 3.5 turbo and closer to Lama 2 chart the 70 billion parameter model which has Cod 92 so bottom line we have got a really good 7 billion parameter model that can run on consumer Hardware GPU CPU and a lot of different devices with without quantization I'll make a separate video about how to run the model meanwhile you can go to this particular link which I link it in the YouTube description to check out the model and also I'll link all the required links in the YouTube description for you to directly click and then start reading about this model in itself I hope this video is helpful to you in learning about the new Zephyr 70 billion parameter model the better model which is doing better with respect to even the 70 billion or the chat GPT models see you in another video Happy prompting