How Does Deep Learning Work? | Two Minute Papers #24

Key Takeaways

dear fellow Scholars this is 2minute papers with car a neural network is a very loose model of the human brain that we can program in a computer or it's perhaps more appropriate to say that it is inspired by our knowledge of the inner workings of a human brain now let's note that artificial neural networks have been studied for decades by experts and the goal here is not to show all aspects but one intuitive graphical aspect that is really cool and easy to understand take a look at these curves on a plane these curves are a collection of points and these points you can imagine as images sounds or any kind of input data that we try to learn the red and blue curves represent two different classes the red can mean images of trains and the blue for instance images of bunnies now after we have trained the network from this limited data which is basically a bunch of images of trains and bunnies we will get new points on this plane new images and we would like to know whether this new image looks like a train or a bunny this is what the algorithm has to find out and this we call a classification problem to which a simple and bad solution would be simply cutting the plane in half with a line images belonging to the red regions will be classified as the red class and the blue regions as the blue class now as you can see the red region cuts into the blue curve which means that some trains would be misclassified as bunnies it seems that if we look at the problem from this angle we cannot really separate the two classes perfectly with a straight line however if we use a simple neural network it will give us this result hey but that's cheating we were talking about straight lines right this is anything but a straight line a key concept of neural networks is that they create an inner representation of the data model and try to solve the problem in that space what this intuitively means is that the algorithm will start transforming and warping these curves where their shapes start changing and it finds that if we do well with this warping step we can actually draw a line to separate these two classes after we undo this warping and transform the line back to the original problem it will look like a curve really cool isn't it so these are actually lines only in a different representation of the problem who said that the original representation is the best way to solve a problem take a look at this example with the entangled spirals can we separate these with a line not a chance but the answer is not a chance with this representation but if one starts warping them correctly there will be states where they can easily be separated however there are rules in this game for instance one cannot just rip out one of the Spyros here and put it somewhere else these Transformations have to be homeomorphisms which is a term that mathematicians like to use it intuitively means that the warpings are not too crazy meaning that we don't tear apart important structures and as they remain intact the Warped solution is still meaningful with respect to the original problem problem now comes the Deep learning part deep learning means that the neural network has multiple of these hidden layers and can therefore create much more effective inner representations of the data from an earlier episode we've seen in an image recognition test that as we go further and further into the layers first we'll see an edge detector and as a combination of edges object Parts emerge and in the later layers a combination of object Parts create object models let's take take a look at this example we have a bullseye here if you will and you can see that the network is trying to warp this to separate it with a line but in vain however if we have a deep neural network we have more degrees of freedom more directions and possibilities to warp this data and if you think intuitively if this were a piece of paper you could put your finger behind the Red Zone and push it in making it possible to separate the two regions with a line let's take a look at one-dimensional example to better see what's going on this line is the 1D equivalent of the original problem and you can see that the problem becomes quite trivial if we have the freedom to do this kind of transformation we can easily encounter cases where the data is very severely Tangled and we don't know how good the best solution can be there is a very heavily academic subfield of mathematics called not Theory which is the study of tangling and untangling objects it is is subject to a lot of snarky comments for not being well too exciting or useful what is really mind-blowing is that not Theory can actually help us study these kinds of problems and it may ultimately end up being useful for recognizing traffic signs and designing self-driving cars now it's time to get our hands dirty let's run a neural network on this data set and see what happens if we use a low number of neurons and one layer you can see that it is trying ferociously but we know that it is going to be a fruitless Endeavor upon increasing the number of neurons magic happens and we know exactly why yeah thanks so much for watching and for your generous support I feel really privileged to have supporters like you fellow Scholars thank you and I'll see you next time