Sliding Window Attention Explained: The Core Concept and the Math, without any fluff :)

This article is based on Longformer: The Long-Document Transformer (Beltagy, Peters & Cohan, 2020, arXiv:2004.05150). The paper introduces… Continue reading on Medium »