Variational Autoencoder [VAE] from scratch | Intuition + Coding

Autoencoders and Variational Autoencoders often look almost identical in diagrams, an encoder, a latent space, and a decoder, but the difference between them completely changes what these models can and cannot do. A standard autoencoder learns a direct, deterministic mapping from image space to latent space and back. For a given input image, the encoder always produces the same latent vector, and the decoder always produces the same reconstruction. This makes autoencoders very good at learning compact representations, removing noise, compressing images, and detecting anomalies, because the latent space is optimized purely for reconstruction fidelity. The model is rewarded for being precise, not for being creative, and the latent space ends up reflecting that objective. The limitation appears the moment we treat the latent space as something we can sample from. An autoencoder does not learn how to organize its latent space in a smooth or continuous way. Two nearby latent points do not necessarily correspond to similar images, and random sampling usually produces meaningless outputs. This is not a failure of training, it is simply not what the model was designed to do. Variational Autoencoders change exactly this assumption. Instead of mapping an image to a single point in latent space, the encoder maps it to a distribution defined by a mean and a variance. The latent vector is then sampled from this distribution, which introduces controlled stochasticity into the model. During training, the latent space is explicitly regularized to follow a known prior distribution, which forces it to become smooth, continuous, and sample friendly. This single change has deep consequences. VAEs sacrifice some reconstruction sharpness in exchange for a latent space that can be meaningfully explored and sampled. Interpolations between points become meaningful. Random samples decode into plausible images. The model is no longer just compressing data; it is learning a structured gene