Linear Transformation in Self Attention | Transformers in Deep Learning | Part 3

In this third video of our Transformer series, we’re diving deep into the concept of Linear Transformations in Self Attention. Linear Transformation is fundamental in Self Attention Mechanism, shaping how inputs are mapped to key, query, and value vectors. In this lesson, we’ll explore the role of linear transformation, breaking down the math behind them to see why they’re essential for capturing dependencies in Self Attention. We’ll go through detailed mathematical proofs to show how Linear Transformation work and why it is crucial for capturing relevant similarities and generate an appropri…