Evolution of Self-Attention in Vision

In the second installment of this series, the video delves into the advanced world of self-attention in the visual domain. It focuses on three pivotal papers that leverage self-attention mechanisms for image processing. The video builds on the foundation laid by the non-local module discussed in the first video. The spotlight is on attention-augmented convolution and two seminal papers on stand-alone self-attention, each proposing innovative modifications to the ResNet architecture. This exploration not only elucidates the evolution of attention mechanisms in vision but also sets the groundw…