How LLM (ChatGPT) works - Animated

Have you ever wondered what happens when you ask something to an AI app like ChatGPT? It feels almost magical, right? You type a question and instantly you get a smart answer. Behind the scenes, there's no magic, just some incredibly clever systems working together. In this video, we're going to see how a large language model or LLM actually works. All right, time to break the suspense. At its core, a large language model is just a next word predictor. That's it. It doesn't think like a human. It doesn't understand in the way we do. Instead, it looks at a sentence and tries to guess what word should come next. For example, if I say, "I went to the bakery to buy some." What would you guess? Probably bread, right? Not shoes, not clouds, but bread. That's because your brain has seen similar sentences before and that's exactly what the model does, too. It has read massive amounts of text and learned patterns like bakery is often followed by bread, cake, or pastries. So, when it generates text, it's constantly asking, "What's the most likely next word given everything so far?" First, the model needs training data to learn from. So, we gather huge amounts of text from books, websites, and conversations across the internet. The model doesn't look at this entire training data at once. It only sees a small chunk of text at a time. This chunk is called the context window. Now, we turn this training data into input-output pairs. For example, if the input is, "The cat sat on the", the correct output is mat. Then we slide the window forward one word at a time. In this way, thousands, even millions of such examples are created for model training. It's just like a game where you guess the next word in a sentence. The more you play, the better you get. By learning from all these examples, the model becomes really good at predicting what comes next. And that's the secret behind how language models work. Let's start with a simple example. The cat sat on the mat. Humans read this as words, but large language models don't see words the same way. Instead, they break text into smaller pieces called tokens. In this case, each word becomes a token. The cat sat on the mat. Sometimes tokens are whole words, sometimes parts of words, or even punctuation. This process is called tokenization. It's the first step that converts human language into something a model can process. Now that we have tokens, the model still can't understand them directly. Computers work with numbers, not words. So, each token is converted into a vector, a list of numbers. Think of this like giving each word a coordinate in a high-dimensional space. These vectors capture meaning. Words with similar meanings, such as cat, dog, and rat, end up closer together. This step is called vectorization or embedding. This is where things get interesting. Each word doesn't exist in isolation. It depends on other words in the sentence. Self-attention allows each token to look at every other token and decide what matters. For example, sat relates strongly to cat, but not as much to the. This is how model understands that it was the cat who sat. Stronger connections mean higher importance. This is how the model understands context by dynamically focusing on relevant words. Think of another example. The animal did not cross the road as it was tired. Self-attention helps the model figure out what it refers to. It looks at all the words and decides that it is most related to the animal, not the street, because that makes the most sense. So, in a simple way, self-attention lets each word pay attention to the most relevant other words in the sentence to understand meaning. After understanding the sentence, the model predicts the next word. It doesn't know the answer. It calculates probabilities. For example, after the cat sat on the, possible next words might be mat, chair, or floor. Each gets a probability. The model usually picks the word with the highest probability, or sometimes samples for variety. This is how text is generated, one word at a time. To summarize, instead of generating a whole sentence at once, an LLM predicts the most likely next word based on everything that came before it. This rapid-fire chain of logic continues until a complete, coherent thought is formed. It's a seamless blend of math and language, turning a string of predictions into the helpful answers you see on your screen. Thanks for watching the video. Hope it was useful. Subscribe and watch out for more animations videos on how reasoning models work, how image models work, animations on rag and agentic AI, and more.