Instruction Fine-tuning GPT: Supervised Training, Prompt Formatting & Evaluation | Chapter 7

About this lesson

Download the source code from here: https://onepagecode.substack.com/ In Chapter 7 of "Build a Large Language Model (From Scratch)", we perform instruction fine-tuning (also known as supervised fine-tuning) on a pretrained GPT model to make it follow human instructions and generate helpful responses. We start by preparing a custom instruction dataset, formatting it using the popular Alpaca prompt style, and organizing it into training batches with a custom collate function. We then load the pretrained GPT-2 medium model and fine-tune it to follow instructions. Finally, we evaluate the fine-tuned model by generating responses on a held-out test set and scoring them automatically using a larger Llama 3 model via Ollama. What you’ll learn in this chapter: • The difference between classification fine-tuning and instruction fine-tuning • Formatting instruction data using the Alpaca prompt template • Creating a custom PyTorch collate function for instruction datasets • Padding, masking, and preparing training batches for instruction fine-tuning • Loading a pretrained GPT-2 medium model (355M parameters) • Fine-tuning the model on an instruction-following dataset • Generating responses with the fine-tuned model • Automatically evaluating model responses using Llama 3 via Ollama By the end of this chapter, you will have a GPT model capable of following instructions and generating coherent, task-specific responses. #InstructionFineTuning #SupervisedFineTuning #LLMInstructionFollowing #BuildLLMFromScratch #Chapter7

Original Description