PagedAttention: Behind vLLLM's Insane Speed

PagedAttention is the “virtual memory” idea applied to LLM inference: instead of storing each request’s KV cache in one big contiguous chunk, vLLM breaks it into fixed-size blocks and maps logical tokens to physical GPU memory with a block table. The result is far less fragmentation, smarter reuse, and higher throughput under real, mixed-length traffic. In this video, we visualize the problem with a Tetris-style fragmentation demo, then build up the exact data structures (KV blocks + block tables), how prefill vs decode works, how sharing helps for parallel sampling/beam search, and what happ…