Applied Deep Learning 2023 - Lecture 8 - Transformers

About this lesson

Complete Playlist: https://www.youtube.com/playlist?list=PLNsFwZQ_pkE87JO3T_mvedVTlw0sjUzKh == Literature == 1. Halthor, Transformer Neural Networks Explained, 2020 2. Vaswani et al., Attention Is All You Need, 2017 3. Carion et al. End-to-End Object Detection with Transformers, 2020 4. Kilcher, End-to-End Object Detection with Transformers (Paper explanation), 2020 5. Pacha et al., A Baseline for General Music Object Detection with Deep Learning, 2018 6. Parmar et al. Image Transformer, 2018 7. Kilcher, Attention Is All You Need (Explained), 2017 8. Phi, Illustrated Guide to Transformers: Step by Step Explanation, 2020 9. Olah et al. Attention and Augmented Recurrent Neural Networks, 2016 10. Peters et al. Deep contextualized word representations, 2018 11. Howard et al. Universal Language Model Fine-tuning for Text Classification, 2018 12. Devlin et al. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, 2018 13. Kitaev et al. Reformer: The Efficient Transformer, 2020 14. Wang et al. Linformer: Self-Attention with Linear Complexity, 2020 15. Wu et al. Pay less attention with Lightweight and Dynamic Convolutions, 2019 16. Dosovitskiy et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale, 2021 17. Liu et al. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows, 2021 18. Liu et al. Swin Transformer V2: Scaling Up Capacity and Resolution, 2022 19. Sun et al. Rethinking Transformer-based Set Prediction for Object Detection, 2021 20. Gildenblat, Exploring Explainability for Vision Transformers, 2021

Original Description