Vision Transformer(ViT) - Image is worth 16x16 words | Paper Explained

In this video, we discuss the paper “An image is worth 16x16 words: Transformers for image recognition at scale” which introduced the Vision Transformer(ViT) architecture. We start with the motivation of this remarkable architecture, continue with early transformers for computer vision works, dive deep into the intricate details of ViT architecture, unpack ViT training and finetuning methodologies, and highlight significant developments from recent follow-up papers. Enjoy the video? Show your support with a Like, and don't forget to Subscribe for more insightful discussions. Any feedback, questions, or innovative ideas are always welcome in the comment section below! Slides: https://docs.google.com/presentation/d/1IcXGiKPoEHDVgC7tlNLFlBIkUunq_H46Arv3kUYHh6g/edit?usp=sharing Personal links: - Twitter: https://twitter.com/Jeande_d - LinkedIn: https://www.linkedin.com/in/nyandwi/ - GitHub: https://github.com/Nyandwi - Deep Learning Revision Newsletter: https://deeprevision.substack.com - Personal website: https://nyandwi.com - Complete Machine Learning Package: https://nyandwi.com/machine_learning_complete/ Some links from the video: - ViT paper: https://arxiv.org/abs/2010.11929 - Big vision repo: https://github.com/google-research/big_vision - ViT Pytorch: https://github.com/lucidrains/vit-pytorch - Yann LeCun Tweet on ViT vs CNNs: https://twitter.com/ylecun/status/1481198016266739715 #deeplearning #ai #computervision #transformers