We Fine-Tuned GPT OSS 20B to Rap Like Eminem
Links + Notes ๐ https://www.oxen.ai/blog
Join Fine-Tune Fridays ๐ง https://oxen.ai/community
Discord ๐ฟ https://discord.com/invite/s3tBEn7Ptg
Use Oxen AI ๐ https://oxen.ai/
Oxen.ai offers one click fine-tuning or fine-tunes for you! Built on top of the worlds best data versioning tool, we offer tools to automate model evals, generate synthetic data, and effortlessly fine-tune models.
--
Chapters
0:00 Intro: Fine-tuning GPT OSS 20B-120B
4:05 The Task: Rapping in the Style of Eminem
6:15 Where we Kept the Data and the /Explicit Tag
8:08 How We Auโฆ
Watch on YouTube โ
(saves to browser)
Chapters (13)
Intro: Fine-tuning GPT OSS 20B-120B
4:05
The Task: Rapping in the Style of Eminem
6:15
Where we Kept the Data and the /Explicit Tag
8:08
How We Automize Fine-tuning
10:35
Understanding the Internals of the Models
15:13
The Attention Sinks in 20B and 120B
17:37
OpenAIโs New Harmony Format
23:45
Double Check your Templates
27:20
Question: Is the Harmony Format Only for Agentic Use-Cases?
28:25
How the Training Runs Went
32:40
Comparing against Llama 3.2 1B
34:10
Deployment Gotchas: How to Deploy After Training
37:48
Next Up: Supporting GPT OSS 120B
DeepCamp AI