I Computed Self-Attention by Hand. Every Single Number.

1. 1Compute the query, key, and value vectors by hand
2. 2Apply the self-attention mechanism using these vectors
3. 3Calculate the attention weights and scores manually
4. 4Normalize the attention weights using softmax
5. 5Compute the weighted sum of the value vectors
6. 6Visualize the self-attention process to understand its effects

ML engineers and researchers can benefit from this exercise to deepen their understanding of transformer architecture

💡 Self-attention is a core component of transformer architecture, and computing it by hand can provide valuable insights

💡 Compute self-attention by hand to grasp transformer internals