Diffusion Language Models: The Next Big Shift in GenAI

Jia-Bin Huang · Beginner ·🧠 Large Language Models ·8mo ago
Most Large Language Models (LLMs) today are based on Autoregressive models (i.e., they predict texts in a left-to-right order). But diffusion models offer iterative refinement, flexible control, and faster sampling. In this video, we explore several ideas for applying diffusion models to language modeling. 00:00 Autoregressive LLMs 00:13 Limitations of Autoregressive models 00:56 How diffusion models work for images 01:26 DiffusionLM: Apply diffusion to word embeddings 02:46 Latent diffusion models: Apply diffusion to paragraph embeddings 03:37 Masked diffusion models 07:41 Scaling laws of…
Watch on YouTube ↗ (saves to browser)

Chapters (8)

Autoregressive LLMs
0:13 Limitations of Autoregressive models
0:56 How diffusion models work for images
1:26 DiffusionLM: Apply diffusion to word embeddings
2:46 Latent diffusion models: Apply diffusion to paragraph embeddings
3:37 Masked diffusion models
7:41 Scaling laws of diffusion models
8:53 Comparing AR and diffusion models in data-constrained settings.
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Next Up
5 Levels of AI Agents - From Simple LLM Calls to Multi-Agent Systems
Dave Ebbelaar (LLM Eng)