How to Fine-tune LLMs with RLVR (OpenAIโ€™s RFT API)

Shawhin Talebi ยท Beginner ยท๐Ÿง  Large Language Models ยท3mo ago
๐Ÿค Work with me: https://aibuilder.academy/yt/k-94oCJ_WJo ๐Ÿš€ Ship AI apps in weeks, not months: https://aibuilder.academy/courses/yt/k-94oCJ_WJo This is the 3rd video in a larger series on reinforcement learning (RL) with LLMs. Here, walk through a concrete example of fine-tuning GPT-o4-mini to detect HDFS anomalies using RLVR. ๐Ÿ’ป GitHub Repo: https://github.com/ShawhinT/rlvr-hdfs-classification ๐Ÿค— Dataset: https://huggingface.co/datasets/shawhin/HDFS_v1_blocks โ–ถ๏ธ Series Playlist: https://www.youtube.com/playlist?list=PLz-ep5RbHosU_UY8NtZAMaraz74sMHo2W References [1] arXiv:2509.16679 [cs.CL] [2] arXiv:2509.04501 [cs.CL] [3] arXiv:2501.12948 [cs.CL] [4] https://platform.openai.com/docs/guides/reinforcement-fine-tuning Introduction - 0:00 RL with LLMs - 0:15 RLVR - 1:42 SFT vs RLVR - 2:23 Example: HDFS Classification with RLVR - 4:09 Step 0: Imports - 6:37 Step 1: Train-Validation Split - 7:40 Step 2: Format Data - 10:23 Step 3: Create Grader - 12:27 Step 4: Fine-tune Model - 15:38 Step 5: Evaluate Model - 19:07 Limitations - 22:40 What's Next? - 25:00
Watch on YouTube โ†— (saves to browser)
Sign in to unlock AI tutor explanation ยท โšก30

Playlist

Uploads from Shaw Talebi ยท Shaw Talebi ยท 0 of 60

โ† Previous Next โ†’
1 biometricDashboard2 DEMO
biometricDashboard2 DEMO
Shaw Talebi
 2 biometricDahboard3 DEMO
biometricDahboard3 DEMO
Shaw Talebi
 3 Time Series, Signals, & the Fourier Transform | Introduction
Time Series, Signals, & the Fourier Transform | Introduction
Shaw Talebi
 4 The Fast Fourier Transform | How does it (actually) work?
The Fast Fourier Transform | How does it (actually) work?
Shaw Talebi
 5 The Wavelet Transform | Introduction & Example Code
The Wavelet Transform | Introduction & Example Code
Shaw Talebi
 6 Principal Component Analysis (PCA) | Introduction & Example (Python) Code
Principal Component Analysis (PCA) | Introduction & Example (Python) Code
Shaw Talebi
 7 Independent Component Analysis (ICA) | EEG Analysis Example Code
Independent Component Analysis (ICA) | EEG Analysis Example Code
Shaw Talebi
 8 Kmeans-based Blink Detecter DEMO
Kmeans-based Blink Detecter DEMO
Shaw Talebi
 9 Shit Happens, Stay Solution Oriented
Shit Happens, Stay Solution Oriented
Shaw Talebi
 10 Why Conflict Is Good & How You Can Use It
Why Conflict Is Good & How You Can Use It
Shaw Talebi
 11 Causality: An Introduction | How (naive) statistics can fail us
Causality: An Introduction | How (naive) statistics can fail us
Shaw Talebi
 12 Causal Inference | Answering causal questions
Causal Inference | Answering causal questions
Shaw Talebi
 13 Causal Discovery | Inferring causality from observational data
Causal Discovery | Inferring causality from observational data
Shaw Talebi
 14 How to Be Antifragile | 7 Practical Tips
How to Be Antifragile | 7 Practical Tips
Shaw Talebi
 15 Multi-kills: How to Do More With Less (no, not by multi-tasking)
Multi-kills: How to Do More With Less (no, not by multi-tasking)
Shaw Talebi
 16 Topological Data Analysis (TDA) | An introduction
Topological Data Analysis (TDA) | An introduction
Shaw Talebi
 17 The Mapper Algorithm | Overview & Python Example Code
The Mapper Algorithm | Overview & Python Example Code
Shaw Talebi
 18 Persistent Homology | Introduction & Python Example Code
Persistent Homology | Introduction & Python Example Code
Shaw Talebi
 19 What Is Data Science & How To Start? | A Beginner's Guide
What Is Data Science & How To Start? | A Beginner's Guide
Shaw Talebi
 20 How to do MORE with LESS - multikills
How to do MORE with LESS - multikills
Shaw Talebi
 21 Causal Effects | An introduction
Causal Effects | An introduction
Shaw Talebi
 22 Causal Effects via Propensity Scores | Introduction & Python Code
Causal Effects via Propensity Scores | Introduction & Python Code
Shaw Talebi
 23 Causal Effects via the Do-operator | Overview & Example
Causal Effects via the Do-operator | Overview & Example
Shaw Talebi
 24 Causal Effects via DAGs | How to Handle Unobserved Confounders
Causal Effects via DAGs | How to Handle Unobserved Confounders
Shaw Talebi
 25 Smoothing Crypto Time Series with Wavelets | Real-world Data Project
Smoothing Crypto Time Series with Wavelets | Real-world Data Project
Shaw Talebi
 26 Causal Effects via Regression w/ Python Code
Causal Effects via Regression w/ Python Code
Shaw Talebi
 27 5 Reasons Why Every Data Scientist Should Consider Freelancing
5 Reasons Why Every Data Scientist Should Consider Freelancing
Shaw Talebi
 28 An Introduction to Decision Trees | Gini Impurity & Python Code
An Introduction to Decision Trees | Gini Impurity & Python Code
Shaw Talebi
 29 10 Decision Trees are Better Than 1 | Random Forest & AdaBoost
10 Decision Trees are Better Than 1 | Random Forest & AdaBoost
Shaw Talebi
 30 Dimensionality Reduction & Segmentation with Decision Trees | Python Code
Dimensionality Reduction & Segmentation with Decision Trees | Python Code
Shaw Talebi
 31 How to Make a Data Science Portfolio With GitHub Pages (2025)
How to Make a Data Science Portfolio With GitHub Pages (2025)
Shaw Talebi
 32 My $100,000+ Data Science Resume (what got me hired)
My $100,000+ Data Science Resume (what got me hired)
Shaw Talebi
 33 How to Create a Custom Email Signature in Gmail (2025)
How to Create a Custom Email Signature in Gmail (2025)
Shaw Talebi
 34 I Spent $675.92 Talking to Top Data Scientists on Upworkโ€”Hereโ€™s what I learned
I Spent $675.92 Talking to Top Data Scientists on Upworkโ€”Hereโ€™s what I learned
Shaw Talebi
 35 Lessons from Spending $675.92 to Talk to Top Data Scientists on Upwork #freelance #datascience
Lessons from Spending $675.92 to Talk to Top Data Scientists on Upwork #freelance #datascience
Shaw Talebi
 36 A Practical Introduction to Large Language Models (LLMs)
A Practical Introduction to Large Language Models (LLMs)
Shaw Talebi
 37 The OpenAI (Python) API | Introduction & Example Code
The OpenAI (Python) API | Introduction & Example Code
Shaw Talebi
 38 The Hugging Face Transformers Library | Example Code + Chatbot UI with Gradio
The Hugging Face Transformers Library | Example Code + Chatbot UI with Gradio
Shaw Talebi
 39 Why I Quit My $150,000 Data Science Job
Why I Quit My $150,000 Data Science Job
Shaw Talebi
 40 Prompt Engineering: How to Trick AI into Solving Your Problems
Prompt Engineering: How to Trick AI into Solving Your Problems
Shaw Talebi
 41 The REALITY of entrepreneurship. #entrepreneurship #startup #smallbusiness
The REALITY of entrepreneurship. #entrepreneurship #startup #smallbusiness
Shaw Talebi
 42 Fine-tuning Large Language Models (LLMs) | w/ Example Code
Fine-tuning Large Language Models (LLMs) | w/ Example Code
Shaw Talebi
 43 How to Build an LLM from Scratch | An Overview
How to Build an LLM from Scratch | An Overview
Shaw Talebi
 44 I Have 90 Days to Make $10k/moโ€”Here's my plan
I Have 90 Days to Make $10k/moโ€”Here's my plan
Shaw Talebi
 45 I Spent $716.46 Talking to Data Scientists on Upworkโ€”Hereโ€™s what I learned.
I Spent $716.46 Talking to Data Scientists on Upworkโ€”Hereโ€™s what I learned.
Shaw Talebi
 46 Pareto, Power Laws, and Fat Tails
Pareto, Power Laws, and Fat Tails
Shaw Talebi
 47 Do NOT become an entrepreneur #entrepreneurship
Do NOT become an entrepreneur #entrepreneurship
Shaw Talebi
 48 Detecting Power Laws in Real-world Data | w/ Python Code
Detecting Power Laws in Real-world Data | w/ Python Code
Shaw Talebi
 49 How Iโ€™d learn data analytics (if I had to start over in 2024) #dataanalytics
How Iโ€™d learn data analytics (if I had to start over in 2024) #dataanalytics
Shaw Talebi
 50 4 Ways to Measure Fat Tails with Python (+ Example Code)
4 Ways to Measure Fat Tails with Python (+ Example Code)
Shaw Talebi
 51 Fine-tuning EXPLAINED in 40 sec #generativeai
Fine-tuning EXPLAINED in 40 sec #generativeai
Shaw Talebi
 52 How Much YouTube Paid Me in My First 6 Months of Monetization (as a Data Science Creator)
How Much YouTube Paid Me in My First 6 Months of Monetization (as a Data Science Creator)
Shaw Talebi
 53 5 Questions Every Data Scientist Should Hardcode into Their Brain
5 Questions Every Data Scientist Should Hardcode into Their Brain
Shaw Talebi
 54 AI for Business: A (non-technical) introduction
AI for Business: A (non-technical) introduction
Shaw Talebi
 55 LLMs EXPLAINED in 60 seconds #ai
LLMs EXPLAINED in 60 seconds #ai
Shaw Talebi
 56 3 Ways to Make a Custom AI Assistant | RAG, Tools, & Fine-tuning
3 Ways to Make a Custom AI Assistant | RAG, Tools, & Fine-tuning
Shaw Talebi
 57 What is #ai? โ€” Simply Explained
What is #ai? โ€” Simply Explained
Shaw Talebi
 58 QLoRAโ€”How to Fine-tune an LLM on a Single GPU (w/ Python Code)
QLoRAโ€”How to Fine-tune an LLM on a Single GPU (w/ Python Code)
Shaw Talebi
 59 How to Improve LLMs with RAG (Overview + Python Code)
How to Improve LLMs with RAG (Overview + Python Code)
Shaw Talebi
 60 Text Embeddings, Classification, and Semantic Search (w/ Python Code)
Text Embeddings, Classification, and Semantic Search (w/ Python Code)
Shaw Talebi

Related AI Lessons

โšก
Moonshot AI and the Rise of Beijingโ€™s Open-Source Frontier: What a $20B Valuation Signals forโ€ฆ
Moonshot AI's $20B valuation signals a shift in the AI landscape, with Beijing emerging as a hub for open-source innovation
Medium ยท LLM
 โšก
โ€œLLMs Do Not Remember Anythingโ€: They only process the context we give them.
LLMs don't have memory, they process context given to them, and bigger models won't solve context accumulation problems
Dev.to AI
 โšก
Why My Coding Assistant Started Replying in Korean When I Typed Chinese
Explore how coding assistants can unexpectedly switch languages due to embedding space overlaps, and learn to analyze such phenomena using vector databases and language models.
Towards Data Science
 โšก
Claude AI vs ChatGPT: What I Noticed After Using Both for Real Projects
Compare Claude AI and ChatGPT for real projects to determine their strengths and weaknesses
Medium ยท ChatGPT
Up next
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)
Watch โ†’