Python Tutorial : Deeper networks

Key Takeaways

the difference between modern deep learning and a historical neural networks that didn't deliver these amazing results is the use of models with not just one hidden layer but with many successive hidden layers we forward propagate through these successive layers in a similar way to what you saw for a single hidden layer here is a network with two Vidhan layers we first fill in the values for hidden layer one as a function of the inputs then apply the activation function to fill in the values in these nodes then use values from the first hidden layer to fill in the second and layer then we make a prediction based on the outputs of hidden layer two in practice is becoming common to have neural networks that have many many layers five layers ten layers a few years ago 15 layers with state of the art but this can scale quite naturally to even a thousand layers you use the same forward propagation process but you apply that iterative process more times let's walk through the first steps of that assume all layers here use the rel you activation function we start by filling in the top node of the first hidden layer that will use these two weights the top weight contributes three times to or6 the bottom weight contributes twenty the rail you activation function on a positive number just returns that number so we get 26 now let's do the bottom nodes of that first hidden layer we use these two nodes using the same process we get four times three or twelve from this weight and minus twenty-five from the bottom weight so the input to this node is 12 minus 25 recall that when we apply value to a negative number we get zero so this node is zero we've shown the values for the subsequent layers here pause this video and verify that you can calculate the same values at each node at this point you understand the mechanics for how neural networks make predictions let's close this chapter with an interesting and important fact about these deep networks that is they internally build up representations of the patterns in the data that are useful for making predictions and they find increasingly complex patterns as we go through successive hidden layers of the network in this way neural networks partially replace the need for feature engineering or manually creating better predictive features deep learning is also sometimes called represented representation learning because subsequent layers build increasingly sophisticated representations of the raw data until we get to a stage where we can make predictions this is easiest to understand from an application to images which you will see later in this course even if you haven't worked with images you may find it useful to think through this example heuristic when a neural network tries to classify an image the first hidden layers build up patterns or interactions that are conceptually simple a simple interaction would look at groups of nearby pixels and find patterns like diagonal lines horizontal lines vertical lines valyrian areas etc once the network has identified where there are diagonal lines and horizontal lines and vertical lines subsequent layers combine that information to find larger patterns like big squares a later layer might put together location of squares and other geometric shapes to identify a checkerboard pattern a face a car or whatever is in the image the cool thing about deep learning is that the modeler doesn't need to specify those interactions we never tell the model to look for diagonal lines instead when you train the model which you learn to do in the next chapter the network gets weights that find the relevant patterns to make better predictions working with images may still seem abstract but this idea of finding increasingly complex or abstract patterns is a recurring theme when people talk about deep learning and it will feel more concrete as you work with these networks more