[MINI] Primer on Deep Learning

Key Takeaways

[Music] data skeptic is the official podcast of datas skeptic.com bringing you stories interviews and manyi episodes on topics in data science machine learning statistics and artificial intelligence today's topic is deep learning so Linda what does this phrase mean to you you must have heard about deep learning in the news and things huh nope do you know what a neural network is no have you heard that term before yeah you say it pretty often the last time we mentioned it reminds us of the brain that's right and do you know much about how the brain works I don't well you've heard of neurons I'm sure right mhm so a neuron is just a cell that takes in a bunch of signals from other neurons electrical signals and then it decides if it wants to transmit its own signal or not just decides yeah based on the inputs I'm not a neurologist and I've probably done a horrible job describing that biologically because neural networks in the computer anyway they're sort of loosely inspired by the brain uh that's kind of where the basic idea came from and they're vaguely like the brain but they're just these mathematical ideas actually in a neural network in a computer think of it as a node just this idea that has a bunch of inputs and it sums them all up or usually it sums them all up and then it applies some function to them and then it transmits that as an output so this is similar to logistic regression we talked about last week well let me show you a little game I put together to help teach this concept all right so Kyle Drew four columns and then in the left column it says 1 and four and on the right column it says 10 and 8 going vertically so the two inputs are the one and the four and then you want to get this output that the answer is 10 and8 okay Kyle have four little cards mhm and you're supposed to Overlay it in the two remaining Columns of the game you take the farle number and you input it and on the cards it tells you whether it's multiply you sum it then divide it and then it will output final answer and so you're supposed to put the cards in the correct order to get the output on the other side of the card mhm so you're basically supposed to fill in the blanks and there's going to be a picture of all this on the in the show notes actually go over to datas skeptic.com and find the pictures there so anyone who wants to play along or just see this game can see more of it but basically I've given you what amounts to an algebra problem right I told you the inputs and I told you the outputs and I'm saying figure out what goes in the middle mhm let's do a simple one having nothing to do with the game so I'm going to tell you examples of inputs and outputs and they're all going through the same function but I'm not telling you what the function is you have to guess all right okay so input is one mhm output is two mhm input is three output is six input is four output is eight yeah you're just multiplying it by two there you go you solved that puzzle it was not difficult now what if I gave you a problem that instead of just multiplying by two the actual answer was multiply by two then Square it and then divide by 16 and then take the tangent of that and then uh raise it to the power of six and that's your answer it may not be difficult but I would probably need a calculator in some time as it sounds tedious yeah there are simple math functions like doubling which is the one I just gave you and there are more complicated math functions like look at all the pixels in an image and determine if it's has a cat in it or not now image recognition is just a lot of algebra on scale or a lot of arithmetic actually once you have the problem solved there's nothing fancy about it it's just heavy heavy duty computation what does it compute for neural networks are collections of nodes and they all take different inputs so if we think of them as being in layers you had a like a zero well I guess a one layer problem I gave you initially you just you know the weight which is two so you take the input times its weight you get the output but now the problem can have a lot more inputs all at the same time and the nodes can each then have many many more inputs so I've given you a more complicated version in the game where there's two nodes so each node takes both the inputs sums multiplies them by something those are the weights sums them up and then puts them through a function that's what every neuron does basically sum them up put them through a function and for Simplicity sake our function is just divide by two so now you know that there's a process from input to Output where you start at the input you multiply by some weight to both of them you sum it up and you you get the value of two other neurons then you do that again through another layer and then you can get the output so how much harder of a problem is this to you well seems at every node there's two things going on then you push it out and then you have two more things going on so there's at least five steps for every number and I have given you how many possible cards you have given me four Linda knows the input and the output layers there are two hidden layers of neurons in the middle but I've given her four cards so can they be switched in order no no so the problem is easier than that so the first slot the first layer the first hidden layer it has one of two possible values and the second hidden layer also has one of two possible values so you basically got four combinations here that you have to solve you could it four yeah two and two okay so now if you wanted to just Brute Force this you could do it right you could swap them out into all four combinations and run through a little bit of arithmetic and figure out which two cards are correct right mhm so actually maybe see how easy that is go go through that exercise let's see if you can get it right well so I just figured it out and with these two combos they did not turn out the way I thought they would I just completed them so what I will do next for brute forth first first describe your strategy how did you try and how did you guess and how did what did you do wrong I did not guess I simply just put the cards down and then I did the arithmetic how many steps did you have to take like I said five for every one so 10 okay so you took 10 steps how however you're counting steps for one configuration so worst case scenario you're going to do we have four combinations you're going to do four time 10 40 steps here right yeah but I probably only need to do two and then from there I could figure out the rest by looking at the sum all right let's see let's see you work through it can I put that in a loop and make it yeah hopefully it's on key well I'll autotune it for you so it sounds good I don't need autotune I've been taking vcal well we'll see what the autotuner thinks I did it you solved it yep lucky you so it only took me I guess it took me 20 steps now I'm going to give you another problem this time it has 10 inputs 10 outputs and six layers what yeah let me see what all right so I'm your handwriting's awful so now you have 10 input nodes 10 output noes and six hidden layers I'm going to go out for lunch and you try and solve this while I'm gone okay I don't know I wanted a nap first all right well the second one is just for for pretend uh the point I'm trying to make is when you have very small neural networks like I just gave you you can kind of solve these things by hand you can the goal of it always is to try and find the optimal weights now actually I made this pretty easy for you cuz I gave you combinations of what the weights were what are the weights what what are weights to you good question each of those places where I said the output of this node you multiply by some number that's a weight now I made this a little bit easy for you by giving you cards that already had picked the weights right and one of my cards was correct in both cases What if I hadn't told you the weights and you the options of what the weights could be well I wouldn't know how many steps I would just make it up I'd be like oh one times 10 so there I'm done well oh that's interesting um how do I know if I'm right or wrong well so then that becomes that's a really good okay so that's a good point this is because I gave you one input and one output but in general you would be learning multiple inputs and multiple outputs that all have the same function applied to them so now in this example I like I was saying I gave you the weight yeah now how much easier was this for you than if I'd ask you to find the weights on your own oh a lot easier how long would you have been here if I didn't give you those weights but I have everything else yep well you have the inputs and the outputs yeah well probably wouldn't here at least 30 more minutes at least yeah that's the tricky and interesting thing once you have the weights it's pretty easy to compute how the model works but coming up with the magic set of Weights like what are the perfect weights how do we do it that's the challenge that algorithms in deep learning and different techniques in deep learning need to be invented for so that we can find them in Practical amounts of time on the available data and solve the problem pretty efficiently and that's what the whole field of deep learning research is all about weights mhm what yeah just find the weights that's so just not magical well I mean pointed if you want me to put some magic in I'll talk about neural architecture and how they construct these things um saying all deep learning is is finding weights is true in the same way that saying all painting is is putting chemicals on canvas so would you regard this as a difficult problem um I can make something up well then you ought to apply for a deep learning job I'll would make something up that simple I wouldn't overthink it I think your little strategy over overthinks it so I would make it very simple and go 2+ 2 is 2 minus three then you get the number well are you saying there's no other way to get to these number inputs and outputs unless it's your problem because that's not true I don't think we're going to be talking about the ways in which you might do this in the coming weeks we're going to talk about rnns and convolutional neural networks and back propagation and the vanishing gradients and all these interesting neural network topics but today what I want to do is more establish what neural networks are and deep learning is just basically neural networks that have many many layers in the middle the problem I gave you had two layers but these can have pretty much any amount of layers you want now a lot of deep learning is about how you set these up like I was saying neural architecture maybe you could speculate if you were going to build up one of these regardless of what would motivate you to do so what are some of the choices you would have to make well you had to decide that you were going to sum it and then you had to decide which steps and what order which nodes connected to which and then the weights and how many layers yeah and even a few more things potentially so I made a lot of choices in constructing my neural network and that's actually what a deep learning person does a lot of they have to decide how to structure their neural network and hope it gets a good result so that's why also there's a variety of different approaches in deep learning and different things like gating and and uh lstms and stuff we'll get into in the future but all these little tricks and stuff are all basically there to make it possible for you to find those elusive weights that solve your problem for you and uh there's different ways of configuring these things that all have different advantages and disadvantages and that's largely the study of deep learning well my final thoughts are this is kind of like algebra but with more layers so in my opinion algebra is all you need to know yeah linear algebra there's you're exactly right your instincts are good there's a ton we represent these algebraically absolutely and then we take advantage of interesting things we know about algebra and calculus to make it possible to solve these things quickly but that's a little bit more advanced we'll get into that probably in our interviews and stuff like that that's how you end all minis for reasons that you won't get into unclear if you're an expert or not well they have to keep tuning in to find out I guess next time I'm a skeptic so before we go I want to share a quick word from our sponsor this week which is again the data science Association you might recall uh two weeks ago when I talked to Serene who's back with us today hi Serene hello Kyle you guys have been working hard on getting ready for the upcoming conference tell us when and where that is so the Dallas data science conference 2017 will be held at the University of Texas at Dallas on February 18th and this time we also invited Kyle the host to be our one of our speakers we are really glad that and honor to have you joining us oh thank you so much yeah I'm really excited it could fit my schedule so I'll be there as well so my talk is going to be titled our API Services taking over all the interesting data science problems Serene can you tell us who else people can get to hear besides having Kyo to be at our conference we also have speakers from IBM talking about the cases on real life application of analytics we also have Clarity solution group her name is mton Balon and she will give a talk on image processing as a part of the Big Data initiative another speaker from EMC Del and the topics will be helping Business Leaders get over their learning curve in advanced analytics this conference will feature a wide variety of data science experts from industry leading companies such as EMC Dell IBM and Amazon I'm looking forward to hearing a lot of those talks myself seren are there any tickets left we only have 30 tickets left so if you want to join purchase the tickets now yep good advice where can they go to do that you may register at Dallas data science. eventbrite.com Serena what's this going to cost people for student tickets it will be $40 and for general admission it will be $6 once again that's Dallas datas science. eventbrite.com to get your tickets I hope I get a chance to meet some of you Saturday February 18th at this conference so come up and say hello