LLMs in a Nutshell

Every chatbot reply you see is generated by repeatedly guessing the next token. A token is roughly a chunk of a word. Imagine you type what is the capital of France. The model scores every possible next token, picks one, appends it. The scores again capital again is Paris. Tken by token the response materializes from pure prediction. This loop is the entire mechanism behind every large language model. That's it. Yet from this simple process, something that feels like understanding emerges. How does guessing the next word produce coherent reasoning? Let's break it down. The model doesn't output one token. It produces a score called a loget for every token in its vocabulary. Typically tens of thousands of options. These logets pass through softmax, converting them into a probability distribution. One token might get 90%, another 2%, another a tiny fraction. The model samples from this distribution. Two user tunable parameters shape the randomness. Temperature flattens or sharpens probabilities. Top P truncates to the most likely candidates. Higher temperature means creative output. Lower means predictable. This is why identical prompts yield different answers. The model is deterministic. The sampling introduces controlled randomness. That's what makes it feel creative. Where does predictionability come from? Training on text broken into batches of tokens. The scale is staggering. Hundreds of billions of tokens, equivalent to millions of books. But volume alone isn't enough. Data quality and dduplication matter just as much. The model itself is parameters, numbers that determine how input maps to output. Think of each as a tiny dial in a massive matrix of weights. Frontier models range from hundreds of billions to low trillions of parameters. No human sets these values. They start random, producing gibberish, then get tuned through training. The dials adjust until patterns emerge. Training follows a loop. Take text from the data set. Mask the final token. Ask the model to predict it. Compare the prediction to ground truth using cross entropy loss, a measure of how surprised the model was. Then back propagation calculates how to nudge each parameter. Updates happen over batches of tokens to stabilize learning. Predict, compare, nudge weights. Repeat trillions of times. Here's why it generalizes. The objective is so broad that the model must learn grammar, facts, reasoning, humor. None is programmed. It emerges because broad prediction requires broad understanding. The loss teaches patterns. Knowledge is a side effect. Training at scale requires immense computation, months on clusters of thousands of specialized chips. Costs scale with data size, model size, and training steps. But hardware alone wasn't the bottleneck. Before the transformer era, language models processed tokens sequentially. Recurrent networks handled one token at a time. To process token 100, you waited for 99. Think of it as a single lane conveyor. Parallel hardware was bottlenecked by sequential dependence. There's another constraint. Longer context windows mean quadratically more computation. More length equals more cost. Researchers needed architectures designed from the ground up for parallelism. The transformer changed everything. Its key insight. Process all tokens simultaneously. First, each token becomes an embedding vector, a learned list of numbers encoding meaning. Similar concepts cluster nearby in this highdimensional space. But order matters. So transformers add positional encodings, signals overlaid on embeddings, either syosoidal patterns or learned values that indicate sequence position. Think of embeddings as points with positional waves overlaid. Now the model sees everything at once. No waiting. A wide parallel highway replacing that single lane conveyor. This architectural shift unlocked training at massive scale. But embeddings alone don't capture context. That requires attention. Attention lets each token ask which other tokens matter for understanding me. The model computes relevant scores between all token pairs in parallel. When processing the bank was covered in grass, bank attends to grass and covered, shifting its representation to riverbank. Multiple attention heads run simultaneously, each learning different relationships. syntax, semantics, co-reference. This parallel scoring finally keeps hardware busy, but attention scales quadratically. A thousand tokens means a million scores. This n squared cost drives research into sparse and linear attention variants. Between attention layers sit feed forward networks storing learned patterns. Stack dozens of these blocks and embeddings refine into rich contextual representations. Everything so far is pre-training predicting internet text. But autocomplete isn't an assistant. Ask a pre-trained model to summarize and it might continue instead. So training has two more phases. Supervised fine-tuning. Humans write ideal responses. The model mimics them. Then reinforcement learning from human feedback. Raiders compare outputs. Explain gravity might yield two responses. The clear, accurate one gets preferred over the condescending one. These preferences nudge parameters toward helpfulness. The result? A system that doesn't possess human understanding yet produces useful reasoned behavior. As architectures evolve, the core recipe remains. Predict tokens at scale, then guide with feedback. So, here's the question nobody can answer yet. We built a system to predict the next token. We scaled it up. Emergent capabilities appeared. Reasoning, coding, translation, tasks never explicitly trained. Each generation surprises us with abilities we didn't expect. Some believe this is the path to artificial general intelligence, that scaling prediction further will yield systems that match or exceed human cognition. Others argue that prediction alone can never produce true understanding, that something fundamental is missing. We don't know which view is correct. But we do know this. The most capable AI systems ever built are at their core just trying to