Developing an LLM: Building, Training, Finetuning

Sebastian Raschka · Advanced ·🧠 Large Language Models ·2y ago

Key Takeaways

Developing an LLM involves three stages: Building, Training, and Finetuning, utilizing tools such as LitGPT and TinyLlama, with a focus on pretraining and finetuning techniques.

Original Description

REFERENCES: 1. Build an LLM from Scratch book: https://amzn.to/4fqvn0D 2. Build an LLM from Scratch repo: https://github.com/rasbt/LLMs-from-scratch 3. Slides: https://sebastianraschka.com/pdf/slides/2024-build-llms.pdf 4. LitGPT: https://github.com/Lightning-AI/litgpt 5. TinyLlama pretraining: https://lightning.ai/lightning-ai/studios/pretrain-llms-tinyllama-1-1b DESCRIPTION: This video provides an overview of the three stages of developing an LLM: Building, Training, and Finetuning. The focus is on explaining how LLMs work by describing how each step works. --- To support this channel, please consider purchasing a copy of my books: https://sebastianraschka.com/books/ --- https://twitter.com/rasbt https://linkedin.com/in/sebastianraschka/ https://magazine.sebastianraschka.com --- OUTLINE: 00:00 – Using LLMs 02:50 – The stages of developing an LLM 05:26 – The dataset 10:15 – Generating multi-word outputs 12:30 – Tokenization 15:35 – Pretraining datasets 21:53 – LLM architecture 27:20 – Pretraining 35:21 – Classification finetuning 39:48 – Instruction finetuning 43:06 – Preference finetuning 46:04 – Evaluating LLMs 53:59 – Pretraining & finetuning rules of thumb
Watch on YouTube ↗ (saves to browser)
Sign in to unlock AI tutor explanation · ⚡30

Playlist

Uploads from Sebastian Raschka · Sebastian Raschka · 0 of 60

← Previous Next →
1 Intro to Deep Learning -- L06.5 Cloud Computing [Stat453, SS20]
Intro to Deep Learning -- L06.5 Cloud Computing [Stat453, SS20]
Sebastian Raschka
 2 Intro to Deep Learning -- L09 Regularization [Stat453, SS20]
Intro to Deep Learning -- L09 Regularization [Stat453, SS20]
Sebastian Raschka
 3 Intro to Deep Learning -- L10 Input and Weight Normalization Part 1/2 [Stat453, SS20]
Intro to Deep Learning -- L10 Input and Weight Normalization Part 1/2 [Stat453, SS20]
Sebastian Raschka
 4 Intro to Deep Learning -- L10 Input and Weight Normalization Part 2/2 [Stat453, SS20]
Intro to Deep Learning -- L10 Input and Weight Normalization Part 2/2 [Stat453, SS20]
Sebastian Raschka
 5 Intro to Deep Learning -- L11 Common Optimization Algorithms [Stat453, SS20]
Intro to Deep Learning -- L11 Common Optimization Algorithms [Stat453, SS20]
Sebastian Raschka
 6 Intro to Deep Learning -- L12 Intro to Convolutional Neural Networks (Part 1) [Stat453, SS20]
Intro to Deep Learning -- L12 Intro to Convolutional Neural Networks (Part 1) [Stat453, SS20]
Sebastian Raschka
 7 Intro to Deep Learning -- L13 Intro to Convolutional Neural Networks (Part 2) 1/2 [Stat453, SS20]
Intro to Deep Learning -- L13 Intro to Convolutional Neural Networks (Part 2) 1/2 [Stat453, SS20]
Sebastian Raschka
 8 Intro to Deep Learning -- L13 Intro to Convolutional Neural Networks (Part 2) 2/2 [Stat453, SS20]
Intro to Deep Learning -- L13 Intro to Convolutional Neural Networks (Part 2) 2/2 [Stat453, SS20]
Sebastian Raschka
 9 Intro to Deep Learning -- L14 Intro to Recurrent Neural Networks [Stat453, SS20]
Intro to Deep Learning -- L14 Intro to Recurrent Neural Networks [Stat453, SS20]
Sebastian Raschka
 10 Intro to Deep Learning -- L15 Autoencoders [Stat453, SS20]
Intro to Deep Learning -- L15 Autoencoders [Stat453, SS20]
Sebastian Raschka
 11 Intro to Deep Learning -- L16 Generative Adversarial Networks [Stat453, SS20]
Intro to Deep Learning -- L16 Generative Adversarial Networks [Stat453, SS20]
Sebastian Raschka
 12 Intro to Deep Learning -- Student Presentations, Day 1 [Stat453, SS20]
Intro to Deep Learning -- Student Presentations, Day 1 [Stat453, SS20]
Sebastian Raschka
 13 1.2 What is Machine Learning (L01: What is Machine Learning)
1.2 What is Machine Learning (L01: What is Machine Learning)
Sebastian Raschka
 14 1.3 Categories of Machine Learning (L01: What is Machine Learning)
1.3 Categories of Machine Learning (L01: What is Machine Learning)
Sebastian Raschka
 15 1.4 Notation (L01: What is Machine Learning)
1.4 Notation (L01: What is Machine Learning)
Sebastian Raschka
 16 1.1 Course overview (L01: What is Machine Learning)
1.1 Course overview (L01: What is Machine Learning)
Sebastian Raschka
 17 1.5 ML application (L01: What is Machine Learning)
1.5 ML application (L01: What is Machine Learning)
Sebastian Raschka
 18 1.6 ML motivation (L01: What is Machine Learning)
1.6 ML motivation (L01: What is Machine Learning)
Sebastian Raschka
 19 2.1 Introduction to NN (L02: Nearest Neighbor Methods)
2.1 Introduction to NN (L02: Nearest Neighbor Methods)
Sebastian Raschka
 20 2.2 Nearest neighbor decision boundary (L02: Nearest Neighbor Methods)
2.2 Nearest neighbor decision boundary (L02: Nearest Neighbor Methods)
Sebastian Raschka
 21 2.3 K-nearest neighbors (L02: Nearest Neighbor Methods)
2.3 K-nearest neighbors (L02: Nearest Neighbor Methods)
Sebastian Raschka
 22 2.4 Big O of K-nearest neighbors (L02: Nearest Neighbor Methods)
2.4 Big O of K-nearest neighbors (L02: Nearest Neighbor Methods)
Sebastian Raschka
 23 2.5 Improving k-nearest neighbors (L02: Nearest Neighbor Methods)
2.5 Improving k-nearest neighbors (L02: Nearest Neighbor Methods)
Sebastian Raschka
 24 2.6 K-nearest neighbors in Python (L02: Nearest Neighbor Methods)
2.6 K-nearest neighbors in Python (L02: Nearest Neighbor Methods)
Sebastian Raschka
 25 3.1 (Optional) Python overview
3.1 (Optional) Python overview
Sebastian Raschka
 26 3.2 (Optional) Python setup
3.2 (Optional) Python setup
Sebastian Raschka
 27 3.3 (Optional) Running Python code
3.3 (Optional) Running Python code
Sebastian Raschka
 28 4.1 Intro to NumPy (L04: Scientific Computing in Python)
4.1 Intro to NumPy (L04: Scientific Computing in Python)
Sebastian Raschka
 29 4.2 NumPy Array Construction and Indexing (L04: Scientific Computing in Python)
4.2 NumPy Array Construction and Indexing (L04: Scientific Computing in Python)
Sebastian Raschka
 30 4.4 NumPy Broadcasting (L04: Scientific Computing in Python)
4.4 NumPy Broadcasting (L04: Scientific Computing in Python)
Sebastian Raschka
 31 4.5 NumPy Advanced Indexing -- Memory Views and Copies (L04: Scientific Computing in Python)
4.5 NumPy Advanced Indexing -- Memory Views and Copies (L04: Scientific Computing in Python)
Sebastian Raschka
 32 4.3 NumPy Array Math and Universal Functions (L04: Scientific Computing in Python)
4.3 NumPy Array Math and Universal Functions (L04: Scientific Computing in Python)
Sebastian Raschka
 33 4.7 Reshaping NumPy Arrays (L04: Scientific Computing in Python)
4.7 Reshaping NumPy Arrays (L04: Scientific Computing in Python)
Sebastian Raschka
 34 4.6 NumPy Random Number Generators (L04: Scientific Computing in Python)
4.6 NumPy Random Number Generators (L04: Scientific Computing in Python)
Sebastian Raschka
 35 4.8 NumPy Comparison Operators and Masks (L04: Scientific Computing in Python)
4.8 NumPy Comparison Operators and Masks (L04: Scientific Computing in Python)
Sebastian Raschka
 36 4.9 NumPy Linear Algebra Basics (L04: Scientific Computing in Python)
4.9 NumPy Linear Algebra Basics (L04: Scientific Computing in Python)
Sebastian Raschka
 37 4.10 Matplotlib (L04: Scientific Computing in Python)
4.10 Matplotlib (L04: Scientific Computing in Python)
Sebastian Raschka
 38 5.1 Reading a Dataset from a Tabular Text File (L05: Machine Learning with Scikit-Learn)
5.1 Reading a Dataset from a Tabular Text File (L05: Machine Learning with Scikit-Learn)
Sebastian Raschka
 39 5.2 Basic data handling (L05: Machine Learning with Scikit-Learn)
5.2 Basic data handling (L05: Machine Learning with Scikit-Learn)
Sebastian Raschka
 40 5.3 Object Oriented Programming & Python Classes (L05: Machine Learning with Scikit-Learn)
5.3 Object Oriented Programming & Python Classes (L05: Machine Learning with Scikit-Learn)
Sebastian Raschka
 41 5.4 Intro to Scikit-learn (L05: Machine Learning with Scikit-Learn)
5.4 Intro to Scikit-learn (L05: Machine Learning with Scikit-Learn)
Sebastian Raschka
 42 5.5 Scikit-learn Transformer API (L05: Machine Learning with Scikit-Learn)
5.5 Scikit-learn Transformer API (L05: Machine Learning with Scikit-Learn)
Sebastian Raschka
 43 5.6 Scikit-learn Pipelines (L05: Machine Learning with Scikit-Learn)
5.6 Scikit-learn Pipelines (L05: Machine Learning with Scikit-Learn)
Sebastian Raschka
 44 6.1 Intro to Decision Trees (L06: Decision Trees)
6.1 Intro to Decision Trees (L06: Decision Trees)
Sebastian Raschka
 45 6.2 Recursive algorithms & Big-O (L06: Decision Trees)
6.2 Recursive algorithms & Big-O (L06: Decision Trees)
Sebastian Raschka
 46 6.3 Types of decision trees (L06: Decision Trees)
6.3 Types of decision trees (L06: Decision Trees)
Sebastian Raschka
 47 6.5 Gini & Entropy versus misclassification error (L06: Decision Trees)
6.5 Gini & Entropy versus misclassification error (L06: Decision Trees)
Sebastian Raschka
 48 6.6 Improvements & dealing with overfitting (L06: Decision Trees)
6.6 Improvements & dealing with overfitting (L06: Decision Trees)
Sebastian Raschka
 49 6.7 Code Example Implementing Decision Trees in Scikit-Learn (L06: Decision Trees)
6.7 Code Example Implementing Decision Trees in Scikit-Learn (L06: Decision Trees)
Sebastian Raschka
 50 7.1 Intro to ensemble methods (L07: Ensemble Methods)
7.1 Intro to ensemble methods (L07: Ensemble Methods)
Sebastian Raschka
 51 7.2 Majority Voting (L07: Ensemble Methods)
7.2 Majority Voting (L07: Ensemble Methods)
Sebastian Raschka
 52 7.3 Bagging (L07: Ensemble Methods)
7.3 Bagging (L07: Ensemble Methods)
Sebastian Raschka
 53 7.4 Boosting and AdaBoost (L07: Ensemble Methods)
7.4 Boosting and AdaBoost (L07: Ensemble Methods)
Sebastian Raschka
 54 7.5 Gradient Boosting (L07: Ensemble Methods)
7.5 Gradient Boosting (L07: Ensemble Methods)
Sebastian Raschka
 55 7.6 Random Forests (L07: Ensemble Methods)
7.6 Random Forests (L07: Ensemble Methods)
Sebastian Raschka
 56 7.7 Stacking (L07: Ensemble Methods)
7.7 Stacking (L07: Ensemble Methods)
Sebastian Raschka
 57 8.1 Intro to overfitting and underfitting (L08: Model Evaluation Part 1)
8.1 Intro to overfitting and underfitting (L08: Model Evaluation Part 1)
Sebastian Raschka
 58 8.2 Intuition behind bias and variance (L08: Model Evaluation Part 1)
8.2 Intuition behind bias and variance (L08: Model Evaluation Part 1)
Sebastian Raschka
 59 8.3 Bias-Variance Decomposition of the Squared Error (L08: Model Evaluation Part 1)
8.3 Bias-Variance Decomposition of the Squared Error (L08: Model Evaluation Part 1)
Sebastian Raschka
 60 8.4 Bias and Variance vs Overfitting and Underfitting (L08: Model Evaluation Part 1)
8.4 Bias and Variance vs Overfitting and Underfitting (L08: Model Evaluation Part 1)
Sebastian Raschka

This video provides an overview of the three stages of developing an LLM, focusing on building, training, and finetuning, with a emphasis on pretraining and finetuning techniques. The video covers topics such as tokenization, LLM architecture, and evaluation of LLMs. By following the steps outlined in the video, viewers can develop their own LLMs and finetune them for specific tasks.

Key Takeaways
  1. Build an LLM from scratch
  2. Prepare a dataset for pretraining
  3. Tokenize the dataset
  4. Design an LLM architecture
  5. Pretrain the LLM model
  6. Finetune the LLM model for specific tasks
  7. Evaluate the LLM model
💡 Pretraining and finetuning are crucial stages in developing an LLM, and the choice of pretraining dataset and finetuning technique can significantly impact the model's performance.

Related Reads

📰
LangChain cost audit: what ConversationBufferMemory actually costs you at scale
Understand the true cost of ConversationBufferMemory in LangChain at scale and optimize your usage
Dev.to · wartzar-bee
 📰
Generative AI Doesn’t Know a Single Fact. So Why Does It Sound So Sure of Itself?
Generative AI models don't know facts, but sound confident due to their training data and algorithms, which is crucial to understand for effective use and development
Medium · LLM
 📰
Hollywood Promised Me JARVIS. Reality Gave Me ChatGPT.
Explore how AI has evolved from fictional concepts like JARVIS to real-world applications like ChatGPT, and understand its everyday uses
Medium · AI
 📰
Calibrated Selective Fact-Checking via Evidence Chain Evaluation
Learn to improve fact-checking accuracy with Evidence Chain Evaluation (ECE) for large language models (LLMs), enabling calibrated selective fact-checking via uncertain verdicts
ArXiv cs.AI

Chapters (13)

Using LLMs
2:50 The stages of developing an LLM
5:26 The dataset
10:15 Generating multi-word outputs
12:30 Tokenization
15:35 Pretraining datasets
21:53 LLM architecture
27:20 Pretraining
35:21 Classification finetuning
39:48 Instruction finetuning
43:06 Preference finetuning
46:04 Evaluating LLMs
53:59 Pretraining & finetuning rules of thumb
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch →