Vectorizing Across Multiple Examples (C1W3L04)

Key Takeaways

in the last video you saw how to compute the prediction on a new network given a single training example in this video you see how to vectorize across multiple training examples and the outcome will be quite similar to what you saw for logistic regression where by stacking up different training examples in different columns of the matrix you'll be able to take the equations you have from the previous video and with very little modification change them to make the neural network compute the outputs on all the examples on pretty much all at the same time so let's see the details of how to do that these were the four equations we have from the previous video of how you compute Z 1 a 1 Z 2 and a 2 and they tell you how given an input feature vector X you can use them to generate a 2 equals y hat for single training example now if you have M training examples you need to repeat this process for say the first training example X superscript round records one to compute Y hat one Wester prediction on your first training example then X to use that to generate prediction y hat two and so on down to XM to generate a prediction y hat M and so in order to write this the activation function notation as well I'm going to write this as a two square bracket round bracket 1 this is a 2 2 and a 2 m so this notation a square bracket 2 round bracket I the round bracket I refers to training example I and the square bracket 2 refers to layer 2 ok so that's how the square bracket and the round bracket industries work and so the suggest that if you have an unvectorized implementation and want to compute the prediction for all your training examples you need to do for I equals 1 to em on there basically intimate these four equations where you need I guess z1 I equals 31 X I plus B 1 on a 1 I equals sigmoid z 1i z 2 I equals W 2 a 1 I plus V 2 and a 2 I equals sigmoid of z2 I right so it's basically you know these four equations on top and adding the superscript round bracket I to all the variables that depend on the training example so adding those superstream round bracket I to X Z and a if you want to compute all the outputs on your M training examples what we like to do is vectorize this whole computation so it's to get rid of this volume and by the way in case it seems like I'm getting a little more of nitty-gritty linear algebra it turns out that being able to implement this correctly is important in the deep learning error and we actually chose the notation very carefully for this class to make these vectorization as as easy as possible so I hope that great through this nitty-gritty will actually help you to more quickly get your correct implementations of these advents working all right so let me just copy this whole block of code to the next slide and then we'll see how to vectorize this so here's we had from the previous line with a four group going over all M training examples so recall that we define the matrix X to be equal to our training examples stacked up on these columns like so so take the training examples stack them in columns so this becomes a n or maybe NX by dimensional matrix I'm just going to give away the punchline and tell you what you need to implement in order to vectorize implementation of this for loop turns out what you need to do is compute capital Z 1 equals W 1 X plus B 1 capital a1 equals sigmoid of z1 then tap code Z 2 equals W 2 times a 1 plus B 2 and then a2 equals sigmoid of Z 2 so if you want the analogy is that we went from lowercase vector X s to this capital case X matrix by stacking up the lower case X's in different columns if you do the same thing for the Z's so for example if you take Z 1 1 z 1 2 and so on these are all column vectors up to Z 1 m right so that's this first quantity but all M of them and stack them in columns then this gives you the matrix Z 1 and similarly if you look at say this quantity you take a 1 1 a 1 2 and so on in a 1m and stack them up in columns then this just as we went from lower case X to capital case X and lo que si to Catholic 8z this goes from the lower case a which are vectors to do some capital A 1 that's over there and similarly for Z 2 and a 2 right there also attained by taking these vectors and stacking them horizontally and taking these vectors and stacking them horizontally in order to get Z Capital Z 2 and capital e 2 one of the property of this notation that might help you to think about it is that these matrices say Z and a horizontally we're going to index across training examples so that's why the horizontal index you know corresponds to different training examples is sweep from left to right you're scanning through the training set and vertically this vertical index corresponds to different notes in the neural network so for example this note this value at the topmost topmost corner of the matrix corresponds to the activation of the first hidden unit on the first training example on one value down corresponds to the activation in the second hidden unit on the first training example then the third heading unit on the first training example and so on so as you scan down this is new indexing into the hidden units number where as you do with horizontally then you're going from the first hidden unit in the first training example to you now the first in the human second training example the third turn example and so on until this note here corresponds to the activation of the first hidden unit in the final training example in the M training example ok so the horizontal the the matrix a goes over a different training examples and vertically the different indices in the matrix a corresponds to different hidden units and a similar intuition holds true for the matrix Z as well as well as for X where horizontally it corresponds to different training examples and vertically it corresponds to different features different input features which are really different notes in phileo of the neural network so with these equations you now know how to implement a neural network with vectorization that is vectorization across multiple examples in the next video I want to show you a bit more justification about why this is a correct implementation of this type of vectorization it turns out the justification will be similar to whether you have seen for logistic regression let's go on to the next video