Fine Tuning LLM Explained Simply

Key Takeaways

Let's understand LLM fine-tuning in a very simple language. When I moved to US from India, one of my friends took me to play baseball. And I was actually comfortable playing baseball. Why? Because in India, I have already played cricket. Now, in both of these games, you have bat, ball, you have to track the ball, and move your bat, run. So, whatever raw skills I had from cricket, I deployed that while playing baseball. Essentially, I'm transferring the skills from one game to another. So, in AI, there is a term called transfer learning, where you take the model, and then you retrain it, or you fine-tune it on new data set, task, and format. So, that is what LLM fine-tuning is. When you look at LLMs such as GPT-5, Llama, etc., they are trained on a vast amount of internet data. But let's say you are an AI engineer working in a private company. Let's say you are working for Johnson & Johnson or Reliance Jio, and you are given a task to build a chatbot for your customers that has knowledge of the internal data of that organization. Also, it should respond in a specific format, tone, etc. At this time, you will use LLM fine-tuning. So, LLM fine-tuning is a process of retraining the pretrained model such as Llama on a specific task, data set, tone, and format. Let's take an example of an imaginary company called Loki Phones uh to understand this thing better. You're building a chatbot for this company using models such as GPT and Llama. And when your customer asks a question, "My Loki Phone 12 screen is cracked. What are my options?" The base LLM will give some general answer, right? But the right answer in this case is "You should use this Loki Care Plus plan if you already have it." Now, let's say this data is not available on internet. In that case, plain LLM will not be able to answer this. There might be other questions, okay? How long battery last, whatever. And assume this is the kind of data which is private to organization, and LLM has not seen it. So, using plain LLM, it won't be able to answer. So, what you do is you use a simple technique called retrieval augmented generation. In this technique, you point your LLM to external source of knowledge, right? It can be a database, PDF files, whatever. This is private data. It's not available on internet. But when you're building a chatbot internally, you can point it, and it will be able to answer it, okay? Now, the benefit of rag is that you are not retraining the model, right? So, LLM is nothing but there is a neural network here, right? There is a neural network uh which you're training, and the network looks something like this. It's a huge network. Let's say it has 70 billion parameter. Now, what do I mean by parameter? Each of these edges have weight. Let's say this has a weight of 0.8, 0.7. And how many edges are there? Maybe 40, 50? In actual LLM, there are 70 billion parameters that you have to update. So, the benefit of rag is that you're not updating everything, and it is very cost-effective, okay? But there are disadvantages. Rag may not be able to produce the best possible answer in your company's brand tone and the format that you expect. For example, for this question, rag may produce this answer, but ideally, you want this answer. Similarly, this is a second question. Rag may produce this, and but fine-tuned model will produce this. And you'll notice that here, there is more empathy. See? And let's say empathy is one of the values of your company that you want your chatbot to follow as well. So, overall, fine-tuning gives this benefit that it can handle emotion slang. It can answer with brand tone, and overall, it can produce more precise answers, whereas rag has its own set of benefit, which is it is low cost, okay? So, if I have to fully summarize pros and cons, then these are the pros and cons, okay? You can go over it. Overall, rag is cheap, fine-tuning is expensive, but fine-tuning produces better answer compared to rag. So, what people do in the industry is they combine both rag and fine-tuning, okay? I'm giving you an example of a base LLM called Llama 3.2. And by the way, you can go to Hugging Face and find both of these models. So, Llama 3.21B is a base LLM, whereas 3.21B instruct is a fine-tuned model. So, this model is fine-tuned on this particular base LLM. And the base the foundation model is just an autocomplete. Uh you all know, right? Like how how large language model works. The way it works is you're given a sentence, and it is just doing uh autocomplete of that sentence. So, when I say, "The capital of India is", it will autocomplete Delhi. Okay, and then it will autocomplete next sentence, next next word, next word, and that's how it produces a big paragraph as an answer. But when you look at the instruct model, that is more similar to the response you get in ChatGPT. Your question-answer pair, okay? So, summarize this article, then it will summarize. Translate this sentence to Hindi, it will summarize, right? So, this is this fine-tuned model is more like ChatGPT, more consumable. There are two types of fine-tuning. One is full fine-tuning, where you will retrain the entire network, and it's going to be costly, right? You're updating 70 billion, 100 billion, whatever is the parameters of your LLM, you're updating all of them. Second one is parameter efficient fine-tuning, and there are two popular methods, LoRA and QLoRA. Here, you are not updating all the layers, actually. You keep these layers frozen, and you add some new layers on top of it. If you know about transfer learning in deep learning, there is a technique called transfer learning, where you keep certain layers frozen. You don't update them. And you update only some other uh layers. So, uh PFT is similar to that, and these are the two popular methods. I hope this video gave you some understanding of what is LLM fine-tuning. I have a plan to produce more videos on this topic, including technical tutorials. So, keep an eye on this channel. If you have any question, there is a comment box below. Thank you very much for watching. [Music]