The Physics of Diffusion Models

Key Takeaways

The bridge between the physics of motion and generative AI are diffusion models. In physics, time-dependent fields are everywhere. When dropping ink in water, concentration changes over time and space. When heat travels through the environment, temperature evolves in time and space. What connects these real-world examples is diffusion, the tendency of particles to move from high to low concentration, which changes their positional distribution over time. Here's how diffusion fits in the lifetime of a model. We'll consider three stages: training data generation, model training, and model inference, also known as sampling for generative tasks. Diffusion comes in before and after model training. The forward direction from structured to chaos produces training data, and the reverse direction from chaos back to structure is what enables us to find new data.