Python Tutorial : Deeper networks

Want to learn more? Take the full course at https://learn.datacamp.com/courses/introduction-to-deep-learning-in-python at your own pace. More than a video, you'll learn hands-on coding & quickly apply skills to your daily work. --- The difference between modern deep learning and the historical neural networks that didn’t deliver these amazing results, is the use of models with not just one hidden layer, but with many successive hidden layers. We forward propagate through these successive layers in a similar way to what you saw for a single hidden layer. Here is a network with two hidden layers. We first fill in the values for hidden layer one as a function of the inputs. Then apply the activation function to fill in the values in these nodes. Then use values from the first hidden layer to fill in the second hidden layer. Then we make a prediction based on the outputs of hidden layer two. In practice, it's becoming common to have neural networks that have many, many layers; five layers, ten layers. A few years ago 15 layers was state of the art but this can scale quite naturally to even a thousand layers. You use the same forward propagation process, but you apply that iterative process more times. Let's walk through the first steps of that. Assume all layers here use the ReLU activation function. We'll start by filling in the top node of the first hidden layer. That will use these two weights. The top weights contribute 3 times 2, or 6. The bottom weight contributes 20. The ReLU activation function on a positive number just returns that number. So we get 26. Now let's do the bottom node of that first hidden layer. We use these two nodes. Using the same process, we get 4 times 3, or 12 from this weight. And -25 from the bottom weight. So the input to this node is 12 minus 25. Recall that, when we apply ReLU to a negative number, we get 0. So this node is 0.We've shown the values for the subsequent layers here. Pause this video, and verify you can calculate