LLaVA paper - Comprehensive dissection

In this video, I dissect the LLaVA paper end to end in a single long-form session, going through the paper the way one would read it seriously for research, instead of treating it as just another multimodal demo. LLaVA is an important work because it shows how strong visual instruction following can emerge not from exotic architectures, but from very careful data curation and alignment choices built on top of existing vision and language models. This video is part of the Reading Research Papers series, where the focus is on building the habit of slow, structured paper reading, understanding the motivation behind each design decision, and connecting the dots between architecture, data, and training objectives. We look at how LLaVA combines a pretrained vision encoder with a large language model, how the projection layer is used to bridge modalities, and why the instruction tuning stage is far more critical than it may initially appear. A major part of the discussion is around data, especially the role of GPT-generated multimodal instruction data, and why this shift from pure architectural novelty to data-centric alignment is such an important moment in vision language modeling. We discuss what LLaVA is actually learning during instruction tuning, how visual grounding emerges, what limitations still remain, and why LLaVA behaves very differently from earlier captioning or VQA-style models. Rather than presenting LLaVA as a finished solution, this session treats it as a research stepping stone, helping you understand what problems it solves well, what problems it only partially addresses, and how it influenced the next wave of vision language instruction models. If you want to build the skill of reading modern multimodal papers deeply, and understand how ideas like instruction tuning, alignment, and dataset design shape model behavior, this video will be valuable.