Video Generation with Diffusion Transformers | Generative AI

In this video, we dive deep into Latte, a latent diffusion transformer for video generation. This generative video diffusion model combines diffusion techniques with transformer architecture and is trained on latent frames of videos. We start with a quick recap of diffusion transformers, as the core building block of this latent transformer for video generation is similar to the adaptive layer norm block variant from the DiT (Diffusion Transformer) paper. Next, we explore specific features of the Latte model, including video patch embedding for processing latent frames, spatial and temporal…