Running "CODE LLAMA" on Free Colab [Full Code Inside]!!!

Key Takeaways

nope I didn't write this code I don't know rejects much but I can use code Lama and do this rejects in this video I'm going to show you how you can run core llama 7 billion parameter model the instruct fine-tuned model on your free Google collaboration let's get started with the video all you need is use this Google Cloud notebook that I'm going to give in the YouTube description click and get started I'm going to explain you line by line the first thing that we need to make sure is we need to make sure whether we have got a GPU so you can make sure it quite easily by just typing Nvidia SMI and they're running the code click runtime Click Change runtime and then you have got the T4 GPU I mean if you are using this Google app notebook most likely it should be already running on GPU but still it is a good practice to make sure that you have GPU the next thing is we need to install the latest Transformers accelerate these are the two libraries that we need to install Transformers and accelerate once we have installed transformation Accelerate from Transformers Import Auto tokenizer input Transformers input torch then the next thing is you need to specify the model this code is especially written for the instruct fine tune model but if you want to use the base model you can still use it I've just commented it and I'll tell you what are the changes that you have to do once you specify the model then you are going to use the tokenizer from pre-trained model and also you're going to build the pipeline if you're not familiar with pipeline pipeline is one of the easiest way the highly abstracted way using Transformers to build a text generation use case so pipeline is also available for a lot of other things like X classification image classification but in this particular case we are going to use pipeline for the particular task which is text generation and we're going to use the same model and we're going to use the model with float 16. here there is no quantization happening just for Clarity on the device map Auto will help the accelerate library to manage memory between CPU and GPU memory this process will take couple of minutes in my case it took two minutes which will download all the required models you can also see that it is a quite a bit of like 12 to 13 gigs of model so just make sure that you have that much of memory for you to run it you're running to some Google collab this should ideally work fine without any issue then the next thing is we are going to give a system prompt and then we are going to give a user question and we're going to create that in the template that Lama model usually accepts that is in this particular format we have got the system prompt and also the user message once you have the system prompt under user message specified in this particular fan part then we're going to use the pipeline to create this particular final step which will take the prompt and also take higher extra other hyper parameters and finally create the sequences which we are finally going to print it out and in this particular case you can play with temperature also you can play with maximum length just to make sure what kind of output on how long do you want it to be once you have specified this all you have to do is now you have to print the sequences and you will have the code ready so let me go and show the question that I've asked I'd also show the output and then we are going to try this ourselves the first question that I've asked is provide answers in Python that is a system prompt and the user messages write a function that detects a pattern that matches the style of 23 0 1 2023 from the given text so what I'm expecting the model to do is I'm expecting the model to create a rejects not necessarily hard code the state but something that looks like this and the code is quite fine I mean it generates a lot more than that but the code is working quite fine so I can just copy this entire thing paste it a Google grab notebook and then you can see that it works completely fine even if I detect the if I change the pattern something like this and run this it works pretty much fine because it manages to write the rejects properly so let's go ahead and ask a different question in this case write a function that detects a pattern I'm going to just give this as an option now so it should be like this uh Jan and I'm going to just give this I'm going to run this code first we have specified the system prompt user prompt then run this code this will take more than few seconds and then finally I'm going to rank this once I run this this should I really given me the code that I can use this in Google collab notebook further sections to practically run and then see if Lama is hallucinating or if it does the job properly meanwhile another thing that I wanted to highlight here is that if you're using not using the InStep model if you are using the bills model then the prompt template that you are going to use here should be slightly different this prompt template is specifically given as a an instruct prompt template if you're using the base model you will use a different method like a prefix and suffix method if you're interested in that I can make a separate tutorial all about how to use that particular model as well but I'm also trying to put together like a web application that will be easier for you to use anyways now this has been done now I'm going to run the final one the that place where we're going to print the result so I write a function that detects a pattern that matches the style of this from the given text now we have got an answer I can copy that come back down and I can paste this after I paste this I can use the function basically and I can paste this and I can run this just to make sure whether it works it didn't work because you know we are still sticking with the world what but let's say in this case I'm going to say Feb the same date Feb and then do this either it should work fine because as you can see the rejects it looks for two digit three letters and then also four digits I think this works completely fine and that is the power of code llama seven billion instant model I don't want to stop with only python even if I don't understand JavaScript necessarily I'm going to go and then say typescript and ask it to do the same thing so I'm going to give the same thing but instead of asking it to do with JavaScript instead of asking you to do it with python I'm asking you to do its typescript and then again this entire thing runs so the system prompt user prompt grows into the prompt template that prompt template goes into the pipeline and the pipeline basically runs the model for inference once that is done we're going to finally print the result unfortunately I do not have enough skill set to evaluate the output but let's see if the rejects matches so meanwhile all you have to do is you have to just go click the link in the YouTube description which I'll give the collab notebook and get started with this little new llama world the model has been run successfully and then print the sequences and now all we change this we change the system prompt typescript and you can see that in typescript it is looking for the constant pattern I think in the typescript probably it didn't do proper good job because it is hard coded it here so maybe we need more prompt Engineering in that particular case it was really good to test so overall I hope this video was helpful to you in learning how to run code Lama 7 billion parameter model on Google collab it's a very simple set of instructions install the latest Transformers from GitHub install accelerate which will help us in memory management and then from Transformers Loadout organizer and improved Transformers which we're going to use for pipeline then input torch which is something we're going to use for the data type and then specify the model and then downward the tokenizer specify the pipeline which is a highly abstracted class for us to do certain tasks using hugging phase Transformers library and in this particular case we are doing text generation this is not a quantized model it's with fluid 16 precision and then provide the system prompt the user prompt and create the prompt template and send the prompt template to Pipeline with the certain hyper parameters and finally get the result and get it printed and you are going to experience a new AI writing computer program for you I hope this was helpful to you if you have any question let me know in the comment section otherwise make sure you check out the description for this Google collab notebook and try it out yourself and let me know the comments what do you feel about this latest llama model