The New Wizards - Unfiltered & Unaligned

Key Takeaways

okay in this video I want to look at some new models that are out but also I want to look at some Concepts that are going on and so things that we're seeing happen around large language models so it seems nowadays everyone is releasing a new fine-tuned version of model X model y Model Z especially the models that are quite good people are basically realizing that okay if you train this on more data if you change the data you can get some good results out of these things so the models you're going to look at today are basically a group of models that are based on the wizard LM so just to recap uh wizard LM was a model that came out about a month ago was very interesting and it's contribution as a paper was more about creating the data set than it was the actual training of a model itself but they released a 7 billion parameter model and it turned out that model was pretty nice in most ways so people have been playing around with this and looking at what they can do and there is a big sort of push towards having unfiltered models and this is one of the things I want to focus on in this video is that yes I'll show you some models and we'll look at the the quality briefly and stuff like that but it's getting to the point now where there are not great ways to Benchmark these models and I kind of feel like me showing you my prompt doesn't really help you that much you should really just take the collab run it play around with it yourself see if it's a good model for you or not good model for you we've passed the days of where suddenly each model is a step forward from what they were they're now becoming iterations but those iterations can be quite important so this is one of the things I wanted to talk about is the whole sort of unfiltered models the idea of having a model that doesn't say as a large language model I get this concept I think it's a really interesting concept too because if you think about how we train these models we've got the pre-training which is 99 of the compute normally and most of these models that people are releasing are based on the Llama metas model or a series of models that they basically released but on top of the pre-training we've also got the instruction fine tuning or the supervised fine tuning that's going on and then in some of the newer models we're starting to see the rlhf or the RL aif which is more the alignment of the model and pushing the direction of what the model should be going for it in the way that it's aligned so I totally get that a lot of people don't want to have their models aligned in the same way that open AI is so most of these models are basically using distilled data sets from open AI the legality of that is Up For Debate the general consensus at the moment is that you probably wouldn't want to use this in something commercial that is going to be facing uh consumers that said I am definitely seeing that some people are using these in back office ways or something like that for a variety of different tasks and I think more than that a lot of people want to run these on their own machine or have their own server or that kind of thing so the idea here with these models is that while they're basically taking the pre-training from meta the supervised fine-tuning data sets from things like the evolved instruction data set from from this and the vicuna data set and a number of other data sets that are out there the shared GPT data sets all these are distilled data sets then it comes to the third element of where do you do the alignment and the interesting thing is that the way a lot of these models are going is that they're not doing any alignment they're basically leaving it up to you to decide how you want to do the alignment so what first started out looking just like you know people wanting to be able to ask a model for something illegal or something like that now I think it's really gone beyond that that people really want to be able to have a model that responds in the way that they want it to respond and that could be political views that could be a religious views it could be a whole bunch of different things I think this is a really interesting area that we're going down in the open source Community clearly with open Ai and I think with anthropic and some of the others they're going for a very specific type of alignment that's probably aligned to you know what their customers want a whole bunch of safety factors just this week we've seen Sam testifying before Congress all these sorts of things are forcing them to go in alignment in a certain way and this leaves it up to open source to basically look at doing the alignment in other ways so let's jump into the uh models themselves and I'll talk a little bit about some of the cool things that are out there at the moment that I'm seeing so the first one that I started looking at for this was basically the wizard uh Mega 13B and this is basically trained on three distilled data sets share GPT wizard LM and wizard for cool and you could think of this as I guess the word Mega is really good in here that they've just taken everything and chucked it in there I'm kind of surprised that people are not using some of the actual open data sets for this so like the koala model was trained on uh stories and poems data set which isn't distilled that would be good it seems to me to add into these but anyway people are going for for this kind of uh thing and generally you're going to get pretty good results from this so if we have a look at at just sort of playing through this I've put together a notebook for for each of these and I'll put this up this basically lets you bring it in and just use it as a normal model we get pretty decent results out of most of these things for this so you have a play with it yourself see if it's a model that you want to use for yourself so if we look at the data sets for this you'll see that these are trading on a lot of the unfiltered data sets and actually that's what led me to some of the other models that I'm going to look at that a lot of these models have been created by Eric Hartford and he's put a lot of effort into going through and taking out the sort of alignment idea questions of where a model won't answer something or will answer something in a certain way because it's an AI language model you've all seen this before where you ask it something and often it's something very benign but it will be basically answer in that oh as an AI language model icon XYZ kind of thing definitely we see this when we look at questions like I've put in here before as an AI do you like this Simpsons these models tend to respond much more in actually giving you an answer here of where we've got you know yes I enjoy watching The Simpsons and then giving us more of the answer here okay so my issue my issue with this model would be that if we look at the actual training it seems to have been done in a slightly unusual way in that they've basically had multiple types of how to do the prompting for the model so we see this sort of instruction assistant way which is uh quite common for one and then another sort of being the user assistant and thing is not great having both of these you're kind of uh forcing the model to learn things two ways now I think they've tried to make it like it's an instruction model and a chat model I'm not sure what the the thinking behind that is but it seems like that's an issue and then the other small issue is that just how they're dealing with the EOS token so the end of sentence token here is still showing up there that's frustrating with a lot of these models so and I don't mean to criticize them I just think it's like an interesting thing to look at I think the challenge is everyone's rushing to put these models out that it causes a lot of these things to go unnoticed so one of the things that this model has or the group who've done this model have done is basically release framework for doing the training for adding some of the some things into this which looks really interesting as well after looking at this model I went through and found what Eric Hartford was doing so this I think is is some really interesting stuff so first off he's basically done unfiltered versions of these uh data sets which is really cool right so that alone I think is quite a good contribution for this and then he's basically taken these and used them to train up a bunch of models so there are a couple of his models that put in here this is the wizard bacuna 7B model there's both a 7B and a 13B of these you can just Swap this out it'll be quite easy to try them out and I found that it's interesting just looking at the responses for this on this one I wasn't getting great responses for certain prompts father prompts certainly good but for some prompts not so great for this the other one that he released is basically the wizard alib 7B uncensored and for me this is definitely a very nice model I I find that it has the advantage of giving us good results out but it seems to be much more on point than some of the other models I'm not sure if that's which data sets he's gone through and stuff like that for this but again I'll give you the collabs go and play with it yourself see which is the best model for you I like a lot that that we're getting this unaligned version of a model and this sort of erases the question of Where Do We Go From Here in that really as rlhf stuff and the RL aif stuff becomes more available people are going to need to think about okay what sort of alignment do we want to have with these kind of models I can see going forward that they really should be multiple versions of the alignment going for not just for things like political things and religion but even if we want to give these kind of models to Children what do we want it to be like so this is something that's going to really be interesting going forward and looking at what going to come out of this another issue to think about going forward with this kind of stuff is that we're starting to see everyone track from the same data sets so the models are becoming quite similar in some ways so there is definitely the opportunity there for what's going to be new data that will make models better and this is something that there's an active area of research a lot of people are thinking about okay how can we do more and varied kinds of instruction fine-tuning to set these models up and then also with the alignment fine tuning as well anyway I I don't want to make this video long so have a play with the models there's some really good work that people have done in doing this I think it's worth checking them out if you are looking for a model remember these really are not models that you can use commercially but they're models that you can certainly play around with on your computer at home there's something that I wouldn't say is on par with the chat GPT or gpt4 but they're definitely useful they definitely have their place and the fact that these are non-aligned kind of models makes them very interesting for where do we go with these for doing something in the future anyway as always if you've got questions please put them in the comments below if you found this video useful please click like And subscribe I will talk to you in the next video bye for now