What I Learned Trying (and Mostly Failing) to Understand Attention Heads

Over the last few years, “attention” has become one of the most overloaded words in machine learning....