1D convolution for neural networks, part 3: Sliding dot product equations longhand

Key Takeaways

when we go to write out the values for our convolution result we can do this pairwise multiplication and summation explicitly we can write it out the long way so let's start by jumping kind of into the middle we'll look at Y of P so our result at the position P this means that our kernel fully overlaps the signal X so it goes from minus P 2 plus P and each one of those aligns with one element of the signal and in addition we know that the very tail end after we've flipped it the very left-hand side of that flipped kernel is W sub P we know that lines up exactly with the first value of our signal X sub 0 and we know that because we're calculating Y sub P the convolution at position P so by counting down from P to 0 we're counting our weights from 0 up to P and so we know that our first element is X sub 0 Y sub P and then we just step counting up on our X's and down on our w's because our W has been reversed our kernel has been reversed so we have X sub 1 times W of P minus 1 and we keep going until we get to the center of our kernel which is X sub P times what W sub 0 and then we keep going until we get to the far end of our kernel which is X sub 2 times P W sub minus P you can see there's a little typo here where I repeat the X sub P W sub 0 that shouldn't be there I'll change that but this shows going all the way from counting from WP to W minus B and from X sub 0 to X sub 2 P so this is a full fully overlapped a fully valid convolution as we step backward from here toward Y sub P minus 1 Y sub P minus 2 all the way back to Y sub 2 y sub 1 Y sub Z we get to a condition where our kernel does not completely overlap our signal some of its dangling off to the side the way we handle that mathematically is we pretend that our signal extends out further but that all of those values are zero but what that means is that we can safely ignore them in our equations because zero times each of the weights in our kernel is going to be zero and so we can always start with for y Sub Zero we can just start right in the middle of our kernel and say it's X sub zero times W sub zero and we can count up on our X index down on our W index because there are kernels been flipped X sub one times W minus one X sub 2 times W sub minus two all the way out to X sub P times W sub minus P then when we go to our next position Y sub one our next output value we just shift all of our weights by one but now we can include the W sub one time X sub zero and then move to X sub one times W sub zero and count all the way up to X sub P plus 1 and W sub minus P and we step through until we're completely overlapped at Y sub P and then we keep going we just increment our X index by one every time we increment our Y index by one and run that sliding dot product step by step all the way across and when we get to the far end we do the same thing than we did at the beginning any portion of the kernel that extends past the end of the signal just gets treated like it's lined up with zeros so each of those kernel elements get multiplied by zero so we can ignore them and then we keep going until that Center element on the kernel is lined up with the last element in our Y in our output array Y sub M and then that's our last value