GPT vs BERT Explained Transformer Based Models Decoder vs Encoder Simple

In this video, we understand how modern language models evolved after Transformers and learn the difference between GPT and BERT in a simple and intuitive way. Here is the GitHub repo link https://github.com/switch2ai You can download all the code, scripts, and documents from the above GitHub repository. We start from the evolution timeline. 2017 Transformer architecture introduced 2018 ULMFiT introduced transfer learning in NLP Language Model A language model estimates the probability of a token occurring in a sequence Example apple is a great company The model learns patterns, grammar, and structure of language Goal Model learns how language works This leads to powerful systems like Generative AI and eventually AGI GPT Model Introduced by OpenAI in 2018 Full form Generative Pretrained Transformer Architecture Decoder based Transformer How it works Understands text in one direction only Reason Uses masked attention so it cannot see future words Example apple is a great company The model predicts next word step by step Use cases Text generation Chatbots Code generation BERT Model Introduced by Google in 2018 Full form Bidirectional Encoder Representations from Transformers Architecture Encoder only Transformer How it works Understands text in both directions Example apple Can understand whether it refers to fruit or company based on full context Training Tasks in BERT Masked Language Model MLM Model predicts missing word Example we are learning transformer we are blank transformer Model learns to predict the missing word Process Input goes through encoder Each word becomes contextual vector Masked position is predicted Next Sentence Prediction NSP Model checks if second sentence logically follows first sentence Example Sentence 1 BERT is a language model Sentence 2 It learns text representations This is correct sequence Incorrect example Sentence 1 BERT is a language model Sentence 2 This is GenAI class Training