Learn how ChatGPT and DeepSeek models work: How Transformer LLMs Work [Free Course]

Key Takeaways

I'm delighted to introduce how Transformer alums work built with Jay Alma and Martin honas the authors of the beautifully illustrated book Hands-On lunch language models Jay is director of engineering fellow at coher and Martin is senior clinical data scientist at the Netherlands Comprehensive Cancer Institute in this course you learn at a deep technical level about the inner workings of how the Transformer Network architecture the P alms works this is the architecture that revolutionize generative AI in fact the GPT in chat GPT stands for generative pre-train Transformer so you build an intuition on how L's process text and you also work with code examples to illustrate the key components of the transform architecture so what you learn is things like what's an attention mechanism and different flavors like self attention and what is a KV cache and so on and if these terms don't yet make sense to you they will after this School the original Transformer was introduced in the 2017 paper attention is all you need by Ashish vaswani and others as a highly scalable model for machine translation TOS variants of this architecture Now power most of today's OMS from open AI anthropic Google coher and meta in 2018 Jay pioneered the efforts of explaining the Transformer architecture in a well-known article The Illustrated Transformer Jay created these wonderful visualizations of the Transformer that help many people understand how it works he also Illustrated other models like gpd2 Bert and staple diffusion thank you Andrew it's great to be here Martin and I think that illustrating complex Concepts such as Transformers creates a fun and easy learning process in our book we worked on an updated version of The Illustrated Transformer as well as describing how to prompt use and train with Hands-On coding examples and we're excited to share some of those ideas with you thank you Jay and Andrew in this course you will see an overview of how language models evolved into the Transformer architecture focusing on language representation you will see early representations where large sparse vectors simply Mark the presence of a word to the smaller dense contextual embeddings that represent the meaning of a word in the context of the sentences they are in you will also learn the meaning of this mysterious and much overused word embedding you will then explore how llm inputs are broken down into tokens which represent words word pieces before they are sent to the language model there are several popular tokenizers and you will see how they differ you will also learn how llms map each token to an embedding Vector you'll then take a closer look at the components of the llm architecture and learn how decoder only llms generate outputs you will learn the details of the Transformer block and how it has evolved in the years since the original paper was released you'll explore an implementation of recent models in the huging face Transformers library and after finishing this course you understand how LMS work in great depth and you have intuitions to help how you approach building applications with LMS I hope you enjoy the course [Music]