Batch Norm vs Layer Norm - Explained

Batch normalization made deep networks trainable back in 2015 — yet every transformer throws it out and uses layer normalization instead. This video shows why. Both methods do the exact same thing to each activation: subtract a mean, divide by a standard deviation, then apply a learnable scale and shift. The only thing that changes is which axis of the batch×feature grid you average over. Batch norm goes down a column (one feature, across every example in the batch); layer norm goes across a row (one example, across all of its own features). We start from a concrete grid of activations to make that axis visible, read both formulas straight off the highlighted direction, and then look at why the batch axis breaks down for sequence models: variable-length sequences, tiny batches, and generating one token at a time at inference all wreck the batch statistics, while the per-example estimate is untouched. Batch norm couples every token to its batch-mates; layer norm lets each token stand on its own — which is exactly what a transformer needs. *Related Videos* ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ Normalization vs Standardization - Explained: https://youtu.be/87C5hkTY8RI Activation Functions in Neural Networks - Explained: https://youtu.be/slp222E_0d4 Stochastic Gradient Descent - Explained: https://youtu.be/DuE_XZovR5o Recurrent Neural Networks (RNNs) - Explained: https://youtu.be/8G1fImBCMcQ Softmax function - Explained: https://youtu.be/oJU6-qW6xZU Convolutional Neural Networks (CNNs) - Explained: https://youtu.be/YGILT182T6w An Introduction to Graph Neural Networks: https://youtu.be/aFnHYEv71U4 *Contents* ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 00:00 - Intro 00:48 - The activation grid 01:36 - Batch Norm: down the columns 02:19 - Layer Norm: across the rows 03:00 - Why the batch axis breaks 03:58 - Each token on its own *Follow Me* ▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬▬ 🐦 X: @datamlistic https://x.com/datamlistic 📸 Instagram: @datamlistic https://www.instagram.com/datamlistic 📱 TikTok: @datamlistic htt