Model Optimization in Microsoft Foundry: Supervised Fine-Tuning

Key Takeaways

Imagine a retail company wants to build a chatbot and they come to you with a request of wanting the chatbot to behave in a very specific way as well as have it grounded on the product data. What do you do? The first step will be prompt engineering where you try adding in a system prompt and give it examples of how the model should behave. We'll prompt our model saying I want to paint my wooden deck. What will you suggest? And wait for a few seconds for it to respond. You will see it will come back with a very wordy response like giving us tips recommendation from Zava. using around 445 tokens of an input of 45 tokens. I'm using GPD41 as a model here. So the next step will be okay can I ground this on our own data using rag or can I find ways to have the model respond in a short-term manner right you can also use rag where you ground your own data to the LLM but then it doesn't solve the question of reducing token usage and also improving performance so what do you do finetuning fine-tuning is where you customize an already pre-trained LLM to work on a very specific task or on a new data set to just enhance its performance and improve accuracy. You can use fine-tuning when you want to one improve the quality two reduce the cost in terms of token usage and lastly you can use it to reduce the latency by training smaller models meaning faster responses or even in Microsoft Foundry you have various fine-tuning options including supervised finetuning vision finetuning and reinforcement fine tuning In this session, we'll focus on supervised fine tuning where the model learns from your examples. How do you do this? So, let's check out the code. We're using variables. So, first thing first is to ensure our model is loaded. And then we get our training and validation files. For training, we have about 40 examples. And for validation, we have a set of 10. For this, we've given the system message, your cora, polite, factual, and helpful. And if someone asks a question, politeness shows the emoji as well as an acknowledgement note. Factual means they give you the prize. And then helpful means it follows up with a question at the end. So with that, you're aiming at reducing the tokens used as well as being factual, polite, and helpful. The distribution of the tokens you can see is about a minimum or a maximum of 62 and 78 tokens. So in average it's about 68.15. That is for both the assistant tokens, the validation as well as the question tokens. We then upload our maintaining job to the cloud and we add it to Azure. You can see we've added our training and validation files. We now specify the fine tuning job. We give it the training files, validation files, the model as well as the seed. That's the only parameter that we using for this. We have the rest of it as defaults. And it will take some time for the fine tuning job to get started once it's submitted. So you can see this is a fine tuning job submitted and then we're tracking the fine tuning job waiting until it's done. So it takes um the time it will take and then once it's done it gives us now the output and from the output you can see it lists us what were the events that took place. We can see the training tokens build were about 8,000 and then it did an evaluation once the model was done training and the evaluation passed [snorts] and also we can see the different checkpoints. So what do we do next? we deploy our finetuning model. So once you see the different checkpoints, you can see it gives us the loss that we had at the end and then the accuracy loss was 1.2 accuracy was 0.6. Uh and then that was the first checkpoint but then at the end we had yeah same accuracy same loss and then we deploy our fine tuning job. So we want to deploy it directly to Azure so that we can use it for reference. We deploy it with a capacity of eight and we're using the developer tier so that we just experiment and not get charged for our deployment. The deployment takes about 12 minutes 45 seconds to complete. And once it's complete we want to also test it out. So we have two prompts. So the first prompt is what kind of paint should I buy for my outdoor deck? And you can see the responses are good. The other option of testing is going to the playground and actually testing out the fine tune model. So you can see this is our fine tuning job. This was the progress. It took about 1 hour 34 minutes. You can see these are all the metrics that we have. So you can see how the loss decreased and the accuracy increased over time. And then you can also check out the logs what happened at each stage what was the steps and then you can also see the model evaluation past. The other thing you can do is checkpoints. This is where you can deploy very specific parts of the model. So instead of deploying the end of it in case like it has overfeitted or anything you can deploy a specific checkpoint and then you have a model that's been deployed and then we tell it your cora an assistant that is polite factual and helpful and we ask the question again I want to paint my outdoor wooden deck so what are the suggestions that you have you'll see from the response it will be much faster. than what we had earlier and it will be much shorter. That's it. So, it's telling us the emoji. Perfect. The acknowledgement note giving us the facts of how much it cost and it's asking do you need any application tips the amount of tokens used very little 73 tokens from the range of about 500. So, we walked through how you can do a simple supervised fine tuning in Microsoft Foundry. You've seen how the model moves from basically having an output that is very unstructured to structuring the output and to being what exactly you want the customer to see. Next, we'll see how you can use faint tuning for aentic tool calls. So, see you in the next video.