1D convolution for neural networks, part 9: Stride

Key Takeaways

there's nothing that says that you have to pad there's nothing that says that your output has to be the same size as your input another option is just to start calculating convolutions once your signal completely is overlapped by your kernel and those convolutions are all valid they don't rely on any data that's outside the range of your signal ever that's a nice very safe way to make sure you get valid convolutions all throughout you don't have to make any guesses about what data to invent and the only drawback then is your output is a little shorter than your input so it in effect reduces the volume of data that you have if you're looking at a time series signal it'll chop off the beginning and end portion if you're looking at an image it'll chop off a border around the image but you're guaranteed that what you're left with is all signal and no edge artifacts another element that you can play with when doing convolution is the stride so by default convolution has a stride of 1 which means each time you shift your kernel over by just one position before calculating the next convolution value this is safe it's complete it's dense sometimes this computation is a bottleneck in whatever it is you're trying to do and you'd like to speed it up also sometimes the signal that you've sampled you've sampled much more densely than you need to and it's okay to just revisit it periodically you don't have to check it every single millisecond in that case you can skip so instead of sliding your kernel over one position you can slide it over two or three or four positions each time or even more and just calculate the convolution periodically this skip is called a stride so if you're visiting every single position that's a stride of 1 if you're skipping a position needs time that's a stride of 2 etc and the stride that you choose is based on what you know about your data how densely you expect that information that you care about to be represented and also how you plan to use the result in the case of convolutional neural networks often the input signal is much more dense than can be usefully handled by the network downstream so some type of resolution reduction happens usually this happens using pooling which we'll talk about a little bit later but another way that you can do this is by introducing a stride so that you're only calculating in the two-dimensional case you're only calculating 1/4 of the total number of convolutions by reducing your resolution by a factor of 2 in both directions and you get a reduction in resolution and a speed-up in your computation time