Spiral RoPE: Vision Transformers Finally Learn to See Diagonals

In the fourth part of my RoPE series, we leave language behind and move into vision. When rotary position embeddings get adapted for image… Continue reading on Medium »