Expressivity: AI Mental Model #6

========================================================= 📓 Free visual lecture notes for this episode: https://vizuaraai.github.io/great-mental-models-of-ai/lecture-06-expressivity.html ========================================================= If you cannot solve it, lift it. When a problem refuses to yield, maybe it is not impossible. Maybe you are asking it in too few dimensions. This is Lecture 6 of The Great Mental Models of Artificial Intelligence. In this episode, we explore Expressivity: the idea that richer, wider, higher-dimensional representations can make hidden patterns visible. Compression squeezes things down to find the essence. Expressivity does the opposite: it lifts things up into a bigger space, where the answer has room to appear. In this lecture, we look at: (1) Why expressivity is the opposite of compression (2) How XOR becomes separable when lifted into 3D (3) The kernel trick and higher-dimensional classification (4) Why higher-degree curves can capture richer patterns (5) How model scale creates new capabilities in LLMs (6) Why transformer feed-forward layers project tokens up to wider spaces (7) How CNNs lift images into stacks of feature maps (8) How Fourier features help NeRFs capture sharp detail (9) Why too much expressivity can overfit noise #ArtificialIntelligence #MachineLearning #DeepLearning #Expressivity #NeuralNetworks #LLM #Transformers #ComputerVision #Vizuara