Llama - EXPLAINED!

Key Takeaways

in this video we're going to talk about llama 2 what is llama 2 how is it trained how does it compare it to GPT variants and chat GPT and its architecture and code so let's get to it what's important to understand first is that llama 2 is a suite of pre-trained language models while llama2 chat is a fine-tuned chatbot that uses reinforcement learning through human feedback and this is analogous to gpt3 which is a suite of pre-trained language models and chat GPT which is also a fine tune chatbot that has reinforcement learning through human feedback so llama 2 is comparable to gpt3 and llama2 chat is comparable to chat GPT let's take a look at these individual buzzwords starting with pre-trained and language models with a language model we take in previous words in order to predict the next word and this is exactly what we want from llama and GPT and in order to do this we would feed it hundreds of thousands of examples of input sequences as well as their next token and in this way we tune the parameters of the model and this training is known as the pre-training and so we pre-train an architecture to understand how to perform language modeling now that we got pre-trained and language model out of the way let's talk about the fine-tuned chat bot so we have a chat bot and for a chatbot we'll take an input question and it will deliver a response to that question and this is exactly what we want from llama to chat and chat GPT now in this case we are still training the model to understand how to perform question answering but this training phase is known as fine tuning now that we got those two concepts out of the way let's talk about reinforcement learning through human feedback so let's say now we have this chat bot this chatbot which is either a llama2 chat or chat gbt is already trained to answer questions now this chatbot though can produce multiple answers for the same question and you can actually see this in action with chat GPT where we ask the same question it gives us multiple answers now at this point we want to check which is one of the better answers here and in this case we would check let's say the second one over here and in order to determine which answer is better we have a group of human reviewers take a look at well for this question of these three answers which is the best so essentially we have humans determining the best answer and this is then acted as feedback to the chatbot where we're basically saying hey we want you chatbot to produce more answers that look like this for this kind of question we quantify this to the chatbot via reinforcement learning and so this entire process is reinforcement learning with human feedback and this chat bot is now tuned to answer questions more like this now that we kind of tackle all of the major buzzwords here let's actually go through the entire training overview end to end so first we have an architecture that's just some untrained model we'll talk about details after this section we then train this model to predict the words that come after the input sequence this is language modeling and we introduce multiple hundreds of thousands of examples so that it truly understands how to model language now this language model is now Lama two so llama 2 is a pre-trained language model now we take this llama2 which is pre-trained on language modeling and then we further train it to answer questions and we feed it in you know hundreds of thousands of questions until eventually this model becomes Lama 2 chat this model is now fine-tuned to answer questions and llama to chat is now made to produce for a single question multiple responses we then have human reviewers determine which is the most appropriate response and then we then make sure that llama to chat understands that hey we want you to produce more answers that look like this via reinforcement learning and we again train this on multiple examples and this is how llama 2 chat becomes safe helpful and non-toxic now that we've taken a look at the overall training overview let's take a look at the model architecture and its code side by side this here is the approximate model architecture for how llama 2 looks now this is a decoder only Transformer architecture and what that means is that these are the stack of the decoder part of the Transformer neural network and it's in a slightly more simplified form note like I mentioned this is an approximate version of llama but it is very close the only main differences are that this normalization layer happens a little before the feed forward Network and this normalization layer also happens before the attention layer to see this in action let's actually take a look at the code so this here is the Facebook research repository for llama and specifically this is now updated to be llama 2 at the time of recording this video and you can see this model architecture is really only just like 300 lines of code and we can take a look at the Crux of this architecture right here in this Transformer block so looking at the forward pass here you can see that first we perform some form of normalization followed by attention and then we'll add it to the input and what we saw in this original figure was attention followed by addition and normalization so this normalization actually comes before next we'll take the output that we received we will again perform normalization feed forward and then addition whereas here we perform feed forward addition and normalization so once again this normalization operation should have come here this is just one Transformer block but the architecture consists of a stack of such blocks and it's preceded by some input embedding layer which will perform some sort of positional encodings too you can kind of also see this in the code where we go to this Transformer section where we're iterating over the layers of the Transformer block and then we want to also perform other operations now let's now look at the model forward pass so this is the code that will be executed when you want to make a prediction given some input what is like the next word or token for llama because llama remember is a language model so we first take the input we'll then try to get its token embeddings this is done by Computing the original embeddings of each token that is each word or byte pair encoding and adding it to or applying it to I should say in this case to the positional embedding to get this final embedding H for every single by pair encoding we'll then create a mask this masking is required because over here you see we're performing masked multi-head attention this is done so that the decoder cannot cheat especially during the training time because it has access to information that comes before and after a current word but it should not be able to look at the information after that current word and hence we need the mask there with inference we don't really have that information so we're just adding masking for the sake of uniformity and so we'll add that masking over here in code then we will make sure that the input goes through all of the layers all the Transformer blocks and then we'll perform some no realization and get the output if you want me to do a much deeper dive on llama code and also a little more deeper dive on the architecture do comment down below let's not compare these llama architectures with these GPT architectures the first here is that llama is open source that is we know the code that actually runs the core model for language modeling that is llama itself whereas GPT is more proprietary code and we can only make assumptions of how the model looks and how the code looks llama is trained on public data whereas GPT is not Lama has limited GPU requirements for fine-tuning at least in comparison to GPT it's also a smaller architecture with more training data so the Llama Suite of Architecture is contained between 7 billion parameters to like 70 billion parameters whereas gbt3 contains with its largest model 175 billion parameters which is multiple times more than that of llama however it is still able to have comparable performance and sometimes even much better performance than gpt3 in multiple places and this is owing to the more training data that it uses it I think Lama 2 uses of the order 2 trillion byte parent Coatings now owing to its small architecture we can also say that llama has a faster inference time than gpt3 this is because the architecture is smaller so there are more operations that need to be done in that forward inference pass and so if you have like a real-time production grade system llama 2 is faster than gpt3 now where do I start coding if I want to make some inference from these models so you can go to hugging face repository there'll be a bunch of these pre-trained llama models as well as some that are fine-tuned and they'll give you some code Snippets where you can see how you can actually use it in your own projects now if you want to go a little deeper and fine tune your llama models you can use certain tools like Colora which provides tools for efficiently training models using like CPU and memory constraints and here's the entire code where you know they're loading data sets loading their models and you can create like a little trainer and eventually just start training this model and you can see that you know over time you will actually be able to train and also you know push this to to hugging face Hub once you're done now this is not my code but I will link to the YouTube channel that actually created this and provided a good schematic for this code and the link to this will be down in the description below another really fun tool that I found here is called Auto Train where you can fine tune any language model or any supported language model and I believe llama2 should be supported in this case and they provide this one line code to start fine-tuning your llama models even on your local machine so I'll link to all of these resources down in the description below now hopefully all of these resources are good for you to understand this wonderful 77-page dissertation on llama2 and I hope that we were able to demystify llama 2 for what it is thank you all so much for watching and I will see you in another video bye-bye