Vokenization Explained!

This video explains a new approach to Visually supervise Language models that achieves performance gains on Language-Only tasks like the GLUE benchmark and SQuAD question answering. This is done by constructing a token-image matching (vokens) and classifying corresponding tokens with a a weakly supervised loss function. Thanks for watching! Please Subscribe! Paper Links: Vokenization: https://arxiv.org/pdf/2010.06775.pdf ImageBERT: https://arxiv.org/pdf/2001.07966.pdf VilBERT: https://arxiv.org/pdf/1908.02265.pdf LXMERT: https://arxiv.org/pdf/1908.07490.pdf UNITER: https://arxiv.org/pdf/1909.11740.pdf Visual Genome: https://visualgenome.org/ 12-in-1: Multi-task Vision and Language Representation Learning: https://arxiv.org/pdf/1912.02315.pdf How Context Affects Language Models' Factual Predictions: https://arxiv.org/pdf/2005.04611.pdf Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines: https://www.nature.com/articles/s41746-020-00341-z ConVIRT: https://arxiv.org/pdf/2010.00747.pdf Climbing towards NLU: https://arxiv.org/pdf/2010.00747.pdf Weak Supervision: A New Programming Paradigm for Machine Learning: http://ai.stanford.edu/blog/weak-supervision/ Thanks for watching! Chapters 0:00 Introduction 1:16 Idea of Vision-Language Models 2:40 Overview of Vokenization 3:38 Voken Examples 4:45 Weak Supervision 6:00 Image Retrieval for Supervision 7:47 What is Grounded Language? 8:25 Issues with Existing Datasets 10:28 Exciting Results for Vision-Language! 13:07 Multi-Modal Learning 14:45 On Meaing, Form, and Understanding 16:04 Information Retrieval in NLP