Convolutional Neural Networks (CNNs) - Explained

Key Takeaways

Consider a simple neural network, a few layers of neurons, each connected to the next. You feed in some numbers, it processes them, and out comes a prediction. Works beautifully for tabular data, structured inputs, things like that. But what happens when you want to feed it an image? Let's say we have this little smiley face, an 8 by 8 grid of pixels. Each pixel is just a number, zero for white, and values between 127 and 255 for the darker parts that form the face. So really, an image is just a matrix of numbers. Now, the obvious thing to do is flatten it, take that two-dimensional grid and stretch it into a single long vector, row by row, pixel by pixel, one after the other. And sure, you could feed that into a neural network. But here's the problem. Take these two pixels, they're right next to each other in the image, vertical neighbors. They clearly share important local information. But once you flatten the image, suddenly they're eight positions apart in the vector. The network has no idea they were ever neighbors. That spatial structure, that local context, it's gone. And that's a real issue because so much of what matters in an image is about which pixels are near each other. So we need something smarter. This is where convolution comes in. Instead of looking at every pixel in relation to every other pixel, we take a small grid called a kernel and slide it across the image. Here we have a 3 by 3 kernel, and our input is a 6 by 6 image. Let's see how it works. The kernel sits on top of a 3 by 3 patch of the image. First, we multiply each kernel value by the corresponding image value. That gives us nine product. Then, we add all those products together to get a single output number. So, the whole operation is just a multiplication followed by an addition. Now, the kernel slides to the next position and does the same thing, multiply then add. Each position produces one value, filling up the output grid one cell at a time. And the output size, if your input is H by W and your kernel is K by K, the output is H minus K plus one by W minus K plus one. So, our 6 by 6 input with a 3 by 3 kernel gives a 4 by 4 output. Now, one kernel detects one type of pattern, maybe vertical edges or horizontal gradients. But we want to detect many different features. So, what do we do? We use multiple kernels. Each one slides across the same input image independently, and each one produces its own 2D output. Then, we simply stack all these outputs together. If we have three kernels, we get three output maps stacked into a volume, H prime by W prime by C prime, where C prime is the number of kernels. And that's the key insight and I seen the number of output channels is simply the number of kernels we choose to use. More kernels means more features detected. So far, we've been working with a single channel image, just one layer of pixel values, like a grayscale picture. But real images have depth. Think of an RGB image. It has three channels, one for red, one for green, one for blue. So, the actual shape of the input isn't just H by W, it's H by W by C, where C is the number of channels. Now, our kernel needs to match that depth. Instead of a flat K by K grid, the kernel becomes a K by K by C volume. It extends through all the channels. The kernel still slides across the spatial dimensions, but at each position, it processes all channels at once, producing a single number. So, even with a multi-channel input, each kernel still produces a single 2D output of size H prime by W prime. And if we use multiple kernels, say C prime of them, we stack all those outputs to get H prime by W prime by C prime, the same pattern as before. In a real convolutional neural network, we don't just do this once. We chain multiple convolutional layers together, one after the other. The output of one layer becomes the input to the next. Notice how the spatial dimensions gradually get smaller, while the depth, the number of channels, grows. The early layers work with large spatial maps and few channels, while the deeper layers work with small spatial maps packed with many channels. At the very end, we take that final compact volume and flatten it into a one-dimensional vector. This vector then feeds into a regular fully connected neural network for classification. But what exactly is that flattened vector? It's a learned feature vector, a representation that the network has learned to extract from the raw pixels. You can think of it as a distillation of the image's most important information, compressed into a format that's useful for making predictions. If that terminology feels a bit abstract, don't worry about it. Just think of it as the network's summary of what it sees in the image. The essential patterns distilled down to a compact form. Now, let's talk about how we actually control the size of each layer's output. We mentioned that the number of kernels determines C prime, the channel dimension, and the kernel size affects H prime and W prime through that simple formula. But in practice, there's another important operation, pooling. Specifically, max pooling. Here's how it works. Take this 4 by 4 grid. We slide a 2 by 2 window across it, no overlap, and from each window, we simply take the maximum value. The top-left window has values 1, 3, 4, and 6. The max is 6. Top right, the max is 8. Bottom left, 7. Bottom right, 4. So, a 4 by 4 map becomes a 2 by 2 map. We've cut the spatial dimensions in half, just like that. And here's the typical pattern you see in practice. We start with a large spatial input, say 32 by 32, with just three channels for RGB. After each convolutional layer, we apply pooling to shrink the spatial dimensions. But at the same time, we increased the number of kernels in each layer, so the channel dimension grows. The spatial dims shrink while the channels grow. The network progressively trades spatial resolution for richer feature representations. Now, all these design choices, local kernels, shared weights, pooling, they might seem arbitrary at first, but they all fall beautifully into place when you think about the inductive biases they introduce. These are the assumptions baked into the architecture that makes CNNs particularly well suited for visual data. The first one is local connectivity. Each output neuron only looks at a small local patch of the input, just the region covered by the kernel. It doesn't see the entire image. And that makes sense because in most images, nearby pixels are far more related to each other than distant ones. A pixel in the top left corner rarely has anything to do with a pixel in the bottom right. Second is translation equivariance. If you shift a feature in the input, say you move an edge two pixels to the right, the output shifts by the same amount. The convolution operation naturally preserves spatial relationships. It doesn't matter where a feature appears, the response will follow it. Third parameter sharing. The same kernel weights are used at every position in the image. This is incredibly efficient. Instead of learning separate weights for every single location, you learn one set of weights that works everywhere. This dramatically reduces the number of parameters and makes the network much easier to train. Fourth is translation invariance. Thanks to pooling, the final classification doesn't depend on exactly where an object appears in the image. A cat in the top left corner gets the same label as a cat in the bottom right. The pooling layers progressively discard precise positional information, keeping only whether a feature was detected somewhere in a region. And finally, hierarchical feature composition. Early layers detect simple features like edges and gradients. The next layers combine those edges into textures and simple shapes. Deeper layers compose those into recognizable parts, an eye, a wheel, a leaf. And the deepest layers combine parts into whole objects. Each layer builds on what the previous one learned, creating an increasingly abstract and powerful representation. And that's basically it. Convolutional Neural Networks, a beautifully elegant architecture that respects the structure of visual data. Thanks for watching. Like the video. See you next time. Bye-bye.