Fine-tune my Coding-LLM w/ PEFT LoRA Quantization - PART 2

Key Takeaways

hello Community welcome to finetune co-pilot part two great now we have this now we have the data set we know all the optimization now step three is easy we just code the most intelligent model in the most intelligent coding implementation great so let's see how we do this we will have a block that takes care about all the quantization topics then we will load our pre-trained code llm this is our typical star coder and then we will have here a port with the p and the Laura now we can combine this in different ways if we want to go with a quantized Laura easy we have this activated we have this activated and then we apply this on our pre-trained model if you say hey I do not want to go with a quantization I go with 16 you deactivate the quantization and you go with just the classical PF Lura on a pre-trained stock coder or you say hey neither I do not want quantization I do not want PFT I just want to go with the fine the classical fine tuning on my pre-trained model then you deactivate the blue you deactivate Lura and you just run here with this tiny segment and now we look again at the hugging face wizard I told you who are the owner of this code and leave you the link of course in the description of this video and you see here is now our most important code sequence this here from the line one to this is about 32 we have everything about quantization this is just handling quantization then from line 34 to 4 before we have here our classical load model from the specified Prof and then the rest here is just fft laow low rank approximation great now let's have a look here at the first half what we do this is a function called create and prepare the model and this we can use for the classical fine-tuning for the Laura tuning and for the quantis lower tuning this is a beautiful implementation and this is why I want to show you this code so here the function check the device map and the bits map byes and bits and bytes configuration then we check if we want a 8 bit quantization or if we want a 4 bit quantization then you create your bits and B configuration and you check if the gpus that you apply support brain flow 16 great you create a device map if a 4bit or an 8bit quantisation is used and this is already it for the Quan session now comes the classical fine-tuning path you say Here auto model for causal Alm from pre-trained hugging face default you have your model you say okay I go with 8 bit you have your cotization you have your device map you have your cache you have your flash mention and this is it so what is the rest the rest is p Laura so we say prepare the model for the PF Laura training we say hey use a 4bit or an 8bit quantization yes yes yes use gradient checkpointing great if we Now activate PF Laura yes we create as I've shown you in my last video in detail the Laura configuration you remember I told you we have 16 parameters that are important but I already focused here on the specific R the rank here for our Lura on the alpha parameter which is our magnification or amplification here of the lower layers then we have the Dropout to prevent overfitting we have the task definition and most important our Laura Target modules our b and a matrix element that we want to apply to each and every rate tensor and you choose which tensor to choose in what layer of our Transformer architecture and all the details are in my latest video before this video then we decide if you want to enable gradient checkpointing yes or no classical way we create now our PFT model we print the trainable parameters and we have have our PA model created and we can go with def find unit great so now we have the understanding we have the data set we have the theory we have any optimization and we know we're going to build one model for all three cases of fine tuning so let's start with the First the full fine tuning the classical fine tuning I will show you the training file but the command is rather easy what we say we use of course hugging face accelerate module so we say accelerate launch then since we are here in a distributed computing for the full fine-tuning we need here to have here our configuration for our data parallelism and we simply say execute train. P so we Define the model PFF we go with star coder or pre-trained code llm we have this particular training data set that I showed you we have split in training the sequence length maximum step size the gradient accumulation steps the learning rate the schedule type the weight Decay you know all of this so you defined yes and we use of course flash attention here in the last line so this is it to execute here a full fine tuning if isn't that beautiful now if you compare this now if you say hey wait a minute I want to go with deep speed in this classical full fine-tuning approach beautiful for a distributed computing here you have your deep speed configuration file and here you have your data parallel configuration file so decide with what optimization you want to go with but you see it is really really really close yeah output directory is a different name but we use here the same training data set hugging face stack version one hugging face stack version one you see I think the parameters are more or less all identical so it is clear how to do this with accelerate launch then you define here in the Deep speed yaml file all your particular parameters train and those are parameter for the train python file great yeah an idea if you do this on this exact command if you use exact here the config that I just showed you and you run here exactly with this parameter the training time on this particular data set here was about 9 hours on a GPU note with 8 a 18 180 GB gpus so remember 9 hours and 8 80 gpus great this was the classical fine tuning quite expensive quite a lot of infrastructure but you know we can do better we have Lura our parameter efficient fine tuning Our PA Lowa implementation let's see how FAS it this is and let's see if we lose any accuracy now this is the command to execute now here our train. python file The Identical training file Now with an additional set of parameters and as you can see here flashing is here the block on the P Laur file well of course we doing now here the parameter efficient fine tuning if you want to compare this two here the classical data parallel full fine tuning you see it is almost identical where we stop here at line 24 here 24 now Laura starts at 25 and goes down to 29 so we just add here the Lura block if we want to apply Laura and this is beautiful but of course we can do better we can also have the quantized Laura version so we app apply a quantization and those last three lines is all that we need to apply quantized Laura so you see we have one train. python file that we can use now for all three different cases of fine-tuning which is beautiful and which I think is really a professional code implementation now training time as I told you before now we've PF lower is 12 and 1 half hours compared to the 9 hours but and this is the beauty instead of having 8 a100 80 gabt we can do this on one GPU on one 40 gab Nvidia data center GPU so this is much cheaper takes a little bit more time but really really cheaper great now we are looking here at the most important file the training file the fine-tuning file for all our models so we have here now fine-tune our code llm star coder on a particular data set and we start here of course with our classical import notice we have sub process we have here the torch utility the data loader the Model Auto mod for causal alarm to tokenizer the hugging fist training module and its training orament and of course for the quantization fits and bite configuration for the P we have your import Laura config the get the PA model and prepare the model for K bit training 4bit or 8bit training plus from the PF tuna we have the loraa layer FR is something similar to a singular vality composition but let's jump right into our ments and here our default parameter so we use here for the data set here exactly the data set I showed you before we split here our data set into training data set the size of the data set we have a sequence length of 8K maximum steps are defined batch sizes defined gradum accumulation steps maybe reduce this a little bit we have here then the learning rate the learning rate schle type the warm-up steps everything that you know and you love then you have all your formats no floating Point 16 brain float 16 no gradient checkpointing the number of workers you want to have evaluation frequency and here we have now our lurer part so we say here use PFT lower we have here the or the rank here of our Matrix decomposition then we have here the alpha parameter amplification is important please remember this the dependen is between or and Alpha as I shown you in my last video and we have to drop out to prevent overfitting and one of the most important modules here are the Laura Target modules my last video I showed you here you can apply here the Lura methodology to almost all the different tensors in almost all the different layers of your Transformer architecture so this is where it can get performance then we decide use Flash attention yes or not and then we have your our quantization never mind that there's a little error here if it's consistent it works you in the code so 4bit quantization nested quants I showed you in my video what this is and F4 compute type float 16 or you go with an 8bit quantization you can push it to the H to final model R so let's start first is here help a function that estimates the average number of characters per token in the data set so you get a an idea about your vocabulary quality and then we have to bring of course everything to a constant length our complete data set have now constant length of chunks of token are returned yes beautiful you know this and then we create our data set now first we load our data set great then we create here a constant length data set command here on the training data set on the valid validation data set and we come now to the function I already showed you create and prepare the models here yes you notice we just went through this and then we have the function run the training beautiful so first we check is deep speed path enabled yes or no the strategy and then we just jump for the trainer to the training arguments of the trainer the hugging face trainer module and you have your output directory data evaluation strategy the maximum steps evaluation steps everything that you know the learning rate this is classical hugging face trainer like we fine tune any other fine-tuning model then we say we apply our function we say create and prepare the model great if we use P Laur yes we print the trainable parameter so you get a feeling about them and then here finally we have our hugging face training module and we insert our model we insert our training arguments just show you here all our training arguments the training data set and the evaluation data set you can do some post processing for faster yes yes yes but here is now our Command you have been waiting for trainer. train now it takes about I don't know 10 hours and of course you can save you can save the last checkpoint of the model if you use deep speed path you have here something to take care of great you can save the model and the tokenizer you can push it to the hub you know this beautiful yeah if deeps with safety model is a single P torch model bin file so we need to short it to run this subprocess to avoid interference from the accelerator you can short this R and then you have your main function and you execute it and this is this is it ladies and gentlemen this is our training python file that does all the work for us and that is really beautifully written so if you have a chance go there we are here at Pacman 100 the workshop personal co-pilot training train.py have a look at this it is really great stuff to learn from so let us talk about the performance characteristic of full fine tuning versus have Laura tuning and here we have it in red you have the full fine tuning and in orange yellow something you have the Laura fine tuning and as you can see here after 2,000 steps I would say well yeah it is really really close so we got a lot of performance Improvement a lot of memory reduction we can go on a single GPU only on a 40 GB GPU and I have to tell you it is really really close to the full-fledged fine tuning now if you are in the lucky position you say hey I have so much money I don't care I will always do the full fine tuning I have no Financial limitations beautiful if you are not in that position here Laura is is some beautiful alternative that really comes close to the original fine tuning a lot of code implementation is happening but let's summarize this now in a example that was given in the literature look we have now fine-tuned a code llm on the most recent GitHub code that is available maybe just from yesterday and we have chosen here particular those do not know how to code and we say here Laura config and then we say fill me so we want here a code implementation by our code LM now the funny thing is that the original GitHub cpilot at the time didn't give any completion and the authors tell you this is due to the fact that half is a recent library at Le published on GitHub that was not yet part of the Microsoft GitHub co-pilot training data so no way that this thing knows about PFT about this particular Li python Library published on GitHub and this is the reason why they have chosen to address this problem here that we are running about one year or half a year behind what is already published and available on GitHub with here the commercial Microsoft GitHub co-pilot schemes or any other GitHub that you might use so doing this fine tuning what we can do so official Microsoft GitHub co-pilot nothing at all fails completely if we look here at the full finding model you see here yes you get here the Laura config you get an idea what happening and if you go with the PA wantest lower completion you can see it is only almost identical so this is able to do it of course I would say gpt2 as an example is also not up to date but remember this code is also already old so it just gives you an idea that when the classical GitHub copilot is not able to perform you are now in a position to fine-tune your own code llms on the latest data and you have really a code AI assistant that is really helpful and particular in my task where I'm now focused here on paft and lower and further optimization it is great I have a highly specialized system I can update if I want every month so now let's put it all together let's code collab I leave you the description in the video and we have here we need an Nvidia GPU beautiful we're going to get clone here Pacman 100 we install some packaging we uninstall ninja and we pip install ninja again we install Flash attention beautiful then we have our git pool we install all the requirement here for this particular repo takes a little bit of time we log into our weights and buyers and we log in if you want to put it on the arguing phas up we have to provide our hugging face token if the token is valed we're ready to go so we change now here on this locally cloned repo in the code assistant to the training and then we have here a g p and we say exactly not the command as you know it this is the command I showed you already so many times ago today python train the python file this is the identical training python file that we already had a detailed look at then we say the model name the pre-trained model we go here with the star coder plus we Define a maximum sequence length of 2K we go with brain floating 16 the number of the training epox and and and in the output directory we do now say listen this will be the PFT Laura star coder plus fine tuned with PFT we will have here the chat data set we will have here the coding data set and we will do this on an a140 gab in a collab notebook this is exactly it gradient accumulation step two data set yes contents and we will use now the Laura adapter so we will have a rank of eight we will have an amplif ification the alpha module of 32 of our lower weights our Target modules remember in my last video I talked in detail what targets module to choose for and of course we will have here a 4bit quantisation beautiful we will use Flash attention and of course the data set that we train everything on now this is an intelligent one because look we do not just have a coding data set but we also have a second task if you want the chatting the Q&A integrated so what we have we have a code and a chat data set an assistant here you find this also on hugging phase and this is it and then you just started and I think about 12 to whatever 15 hours later you get here your result and you have here everything that you need and the job runs and this is great so here a very simple notebook now you know exactly what is happening behind the scene you understand every module every Python program you understand exactly how we do here our Laura configuration how we do the quantization and this is it so this is a very easy example here for a cab notebook you can write on a 40 GB Nvidia GPU and this is it for today I hope it was informative you found valuable information in this video and it would be great to see you in my next video