[Full Workshop] Reinforcement Learning, Kernels, Reasoning, Quantization & Agents — Daniel Han

Why is Reinforcement Learning (RL) suddenly everywhere, and is it truly effective? Have LLMs hit a plateau in terms of intelligence and capabilities, or is RL the breakthrough they need? In this workshop, we'll dive into the fundamentals of RL, what makes a good reward function, and how RL can help create agents. We'll also talk about kernels, are they still worth your time and what you should focus on. And finally, we’ll explore how LLMs like DeepSeek-R1 can be quantized down to 1.58-bits and still perform well, along with techniques to maintain accuracy. About Daniel Han I'm building Unsloth and we're an open-source startup trying to make AI more accessible and accurate for everyone! We have 40K GitHub stars, 10M monthly downloads on Hugging Face and worked with Google, Meta, Hugging Face teams to fix bugs in open-source models like Llama, Phi & Gemma models. I was previously working at NVIDIA making TSNE 2000x faster. Recorded at the AI Engineer World's Fair in San Francisco. Stay up to date on our upcoming events and content by joining our newsletter here: https://www.ai.engineer/newsletter Timestamps 00:00 Introduction and Unsloth's Contributions 03:25 The Evolution of Large Language Models (LLMs) 09:47 LLM Training Stages and Yann LeCun's Cake Analogy 16:56 Agents and Reinforcement Learning Principles 23:17 PPO and the Introduction of GRPO 48:12 Reward Model vs. Reward Function 51:22 The Math Behind the Reinforce Algorithm 01:08:50 PPO Formula Breakdown 01:16:29 GRPO Deep Dive 02:00:20 Practical Implementation and Demo with Unsloth 02:33:07 Quantization and the Future of GPUs 02:41:59 Conclusion and Call to Action