InstructLab Project - Applying open source methods to model alignment, Carol Chen, Red Hat

Key Takeaways

my name is Carol Chen I'm from Red Hat I'm here to talk about instruct lab it's an open source project uh jointly developed by IBM and red hat it's very new it's like a couple of months old I joined the project only last month so I'm learning as I'm I'm going along so you're here to learn with me so no slides but I just wanted to show you the GitHub page github.com instruct laab is the organization and there's a lot of reposit oops no internet wow there's a lot of repositories in there and um we're just going to like here you you go to the main or page if you want to jump right in head to the iight documentation to get started and that's what we're going to do today so good thing I loaded this page before I lost my internet connection so um I'm going to show you the uh instruct lab IAB command uh on the command line and what this does we're going to quickly go through this wow I actually have to make it smaller now okay um this workflow you download the model you check with the llm you add you try to add some new knowledge to the taxonomy generate synthetic training data retrain and try to with it again to see if there's any Improvement it does sound like uh it will take a while so I did a lot of I I did some PR preparation in my hotel room last night so I did a you know the synthetic data generation and stuff like that so you don't have to sit here to wait 20 minutes for that which we don't have 20 minutes anyway so all right let's see some oh no all the windows are messed up now so I actually don't have to enlarge them like I did earlier so because of the resolution okay anyways let's get started so I go to the instruct lab uh directory i' I'm already in the python virtual environment that's information on the GitHub um repo to how to set things up uh I've installed instruct lab but you can do pip install instruct lab it's on pii so let's see the command Let's see we just recently released 0.17.1 two days ago and I upgraded did last night so okay we're on the right version then again I haven't fully tested this version so hopefully it won't mess mess up my demo so with the instruct lab command you can um check with the model well first we have to serve the model check with the model download and so on so you can see a list here actually let's take when you installing for the first time the first thing you do is iLab in it but we already I've already initiated it so I don't do it this time we can take a look at the config yo file uh the default model is a merlinite model which is instruct lab tuned uh mistro model so you can specify different models but uh that's if you don't do that that's the default model you will take so if you do like liab download it will try to download that default model specified oh no internet connection but fortunately I downloaded it last night as well it's a quantized model uh that's about 4.4 gab or something so I I did not plan to do it here so after you've downloaded it you can IAB serve serve the model and then I'll pass in the model family which is um Merlin KN and then then model path in the models directory Merlin night so while I'm waiting for that to start okay I think it's ready so I can do a IAB chat to chat with the um model again all right so welcome to instruct lab chat hello okay it's awake and then uh my topic of Interest today what I want to train is to get some information about the Delan um um time traveling car so let's say typing full sentences is harder than typing commands because I can't tap complete but anyway so it knows a little bit something about it it goes on like you know probably something it you learned from uh Wikipedia or whatever so um what is the speed required to do time travel okay you know something actually last night it was telling me no you can't do time travel forget about that but um you know now it's tell yeah maybe but uh it it is you know um it's not based on real real world physics debatable but anyway we'll see what happens when we you know give it some concrete information so I'm going to quit this right now uh stop serving it so uh as we saw from here you can generate synthetic uh new synthetic data for the training and then retrain each of this is like an iLab command um you can use generate to generate the data and then um but each of these takes time so I last night I generated uh 100 uh number of instructions and and then trained it with uh 100 iterations that took about 45 minutes total so now we have in in this trained folder the uh quantise model of that trained model which is uh supposedly updated so let's see so now we go IAB serve it of the new model this time oops yep that's the one and then the same thing now we chat with it instruct lab trained so now we're now chatting with a different model if you go up just now we were CH chatting with the model that I downloaded but it was not trained uh with the uh synthetic data so now we ask similar questions uh what is the speed required to do time travel ah okay it's not it's actually giving me the wrong data demo effect last night it was giving me yes it's 88 miles per hour no question about it well what do you know right let's let's try again oh come on you're not being nice to me okay let's try a different question how often should oil change be done okay that's good uh I should have done uh shown actually the before the prein tuned model it said every 5,000 miles I was guessing it was like 5,000 to 8,000 or something like that now because we trained it with um I forgot to show this Back to the Future uh data which said um every 3,000 miles or 3 months so you got the information from here and updated it to say yes it's not every 3,000 miles so again I I'm probably already running over my time but I just want to show that you don't have to be uh we want to make it easy for people to contribute to large language models and I I'm personally not a developer at least uh I was in my past life but not now uh you don't have to be a data scientist but you can take the knowledge that you have uh running on uh you know normal consumer laptop and get some results from it by trying out so please check out the project instruct laab um. a or github.com instruct laab and uh if you have any questions don't come ask me ask Mark who's wearing the same t-shirt as I am he's he's the brains uh he's he's a developer who who knows a lot more about the details of how this works so I welcome yall to talk to us and uh and definitely also join us for drinks at restaurant W is that it yes thank you very much excellent