1D convolution for neural networks, part 2: Convolution copies the kernel

you can see here where our signal has just sparsely nonzero elements most of them are zero except for a few that each of these nonzero elements results in a copy of the original kernel depending on the sign and on the magnitude of the nonzero element we can change both the size and the direction the size the magnitude and the sign of the kernel in the result but each of the points just takes a copy of the kernel adds it to the convolution and then scales it accordingly this becomes harder to see when the signal becomes dense has nonzero values altogether but you can imagine it as each element of the signal just taking a copy of the kernel and adding it in to the result we can see here even with a different kernel we get the same result by taking it flipping it and doing the sliding dot product with a signal a sparse signal we'll take that kernel and make scaled copies of it you can see the first one is just like the original kernel but smaller in magnitude the second copy is just like the original kernel and similar in magnitude and the third copy is just like the original kernel but flipped in sign we can express this same thing in math let's call our signal X our kernel W and our convolution result Y in this case we will make sure that our result Y is the same length as our signal X by trimming off the ends we'll come back to this and relax this assumption later and then our kernel W is going to have n elements our input and output Rx and ry are going to have m elements and this is what it would look like in a neural network your input is your signal X your result is your output Y and your Colonel W are your internal weights within the layer that you'll learn during the training phase well change the notation up a little bit instead of having R instead of indexing the values of our kernel from 0 to n minus 1 it makes it a little easier to handle if we index it between minus P and P this assumes that we have an odd number of elements in our kernel which is convenient when we go to slide it along it means that we can line up the value of the result with the value of the signal and they don't get sandwiched halfway in between so odd-numbered kernel is helpful and by assuming an odd-numbered kernel we can index it from minus P to P and this just makes some of the notation a little more convenient later