Let's code a neural network in plain JavaScript Part 1

Key Takeaways

good Monday morning today we're going to code a neural network from scratch in JavaScript no libraries no assumptions about knowing math because I don't know any math I don't know statistics I don't know AI I only have a vague very vague idea about what our neural network is from some YouTube videos I watched I have been procrastinating on learning machine learning for so long let's just do it I am MC J and you are watching fun fun function [Music] today's show is sponsored by pusher who sha makes it easy to add real-time communication and collaboration features to your app chats pub/sub mobile push and Pheebs check them out at push r dot fun fun dot-com that link is also in the episode description Sweden is very warm so my energy levels might be steadily dropping until I die in this episode alright so last week I watched a video by Daniel Schiffman on the basics of implementing a neural network I like Schiffman I wanted to get started with with machine learning and I just figured that hey neural networks that sounds cool it's start with that I streamed it on Twitch I sometimes stream on Twitch so if you don't follow me on Twitch you should at twitch.tv slash fun fun function twitch is still a little bit of a hobby but I really like doing it it was a great deal of fun didn't turn out too well though it's in terms of results I didn't really complete the neural network but I'm not sleeping around with it a little bit more and I think I have something to show you from what I've learned so in this machine learning series I'm going to try to piggyback a lot on the fact that I don't know anything I don't know are much Python I don't know much about math I don't know much about statistics I don't know a lot about probability I don't know a lot about machine learning and AI so I'm going to be utilizing that beginner's mind 7 and try to use the space from where I just when I learned something I'm going to try to still remember what it was like before I learned it and I'm going to use that to to make these videos for you so I'm kind of making making these videos for myself but like two days ago hopefully that will turn out into interesting content or it might just be very very messy this is a new and experimental way of doing videos for me so we'll leave your comments down below after you've watched it where you thought about it and what how I can make it better or if I should just throw it away anyway let's get going have a look at this I'm using observable notebooks because they are amazing not getting sponsored or anything they're just these in really handy interactive notebooks it's kind of like jsfiddle but you can actually store data and oh they're so good I won't go into a high-level explanation of what a neural network is honestly because there's already a pretty much the perfect video on that exists already by three blue one brown but what is a neural network I've linked it here just look at that first before viewing this video if you're unsure what a neural network is it's going to give you a good sort of high-level overview of what it is he also goes into a couple of Matthew details that it's they're going to be scary but don't worry about it we're not gonna go into that just try to you know like oh there's math so our goal is to make out like an ultra ultra simple neural network like the silliest little neural network like it's not even a network it's just one brain cell one neuron perceptron whatever like words they use we're not going to use those words where we're trying to we're gonna try to focus here on getting getting the principle right so I'm not gonna spend a lot of time in words I'm not going to spend a lot of time in the the brain analogies wouldn't accidents in New York like blood that's not really important I don't mean to on all the interesting research that has been done on that but I find that getting getting confused and into the the math and the the brain science of things kind of is kind of distracting has been distracting for me when learning these things I just want to get to the essence of it and understand how it solves problems after that well then we can go dive into the interesting aspects of what lies beyond and how it how it connects into everything but for now we're interested in looking at what a neural network is and how it works so we're going to build a little silly neural network and the purpose of the this little silly neural network is to classify on what side of a coordinate system that a point is so you just imagine a coordinate system and this is like line between it and then it's a bunch of points and then the neural network we will teach the neural network to figure out if a point is on this side of the of the coordinate system or on the line going through the coordinate system or on this side it's not a very smart AI that we're building but you know baby steps building Skynet is a it's a later part in this series okay let's begin by we're gonna generate a bunch of random points you know like points with XY coordinates in a coordinate system in machine learning there's a lot of coordinate systems that's a little bit of math and it's a little bit of geometry there which might give you flashbacks from from high school or whatever but like try to breathe the first thing that we're going to do is to pull in rammed up this is one of the nice things about observable is that you can just pull in our NPM modules it's really nice so let's create some random points random points I'm going to use the are range function and the orange function gives us like numbers from 0 to 5 Z like it just gives us you know the numbers one two yeah I'm really not interested in the actual numbers here I just want an array that is 100 long so that I can do math on it and then I can really care what is in the array I just want you you know do I get assigned a random number there and this was like during the stream at one of the stream viewers actually wrote this function for me so it was nice and interactive which way and there's and we want the numbers to range from minus 1 to 1 so you see here like we get like you see like trenches from minus one to two 180th right numbers by the way you might take offense to this syntax here this is observable an observable thing this is not strictly a global variable like like playing JavaScript interpret it this is actually creating an observable so when if I do like random I can do random points dot length here and ask random points there so whenever I change to this range here you want to watch this number here let's change this to 200 that will automatically update so that this is how this is a basic item or thing in observable and how you declare them so it's it's it's at first it seems like a disgusting syntax but it's actually really really nice to deal with anyway this is not actually points it yet it's as you see it's it's just just numbers but we pour at point is an x-coordinate and y-coordinate so that is it's what we won't we won't and x and y-coordinate so come Friday X and we're gonna run away I am trying to be really really happy even though it is so warm who we now have some X&Y coordinates that's nice alright I want to make this as visual and and interactive as possible so that you don't need to work execute a lot of code in your head and keep abstractions in your head so we're gonna in the series we're going to try to as much as possible constantly visualize what the hell is going on so I'm going to draw the coordinate system and we're gonna draw it with SVG what mpj bear with me it's gonna be cool okay it drew something but it draws everything in the top left corner and that is because you know the the random points that I just keep it with between minus 1 and 1 and I don't know why I did that that was just dumb let's not do that instead I want to keep them between like how big the the the coordinate system is so I'm going to like make these variables so I'm going to kill this x max and I'm going to call this that Y max and we're going to why why max 400 and I'm going to also do x max 400 by the way this when I after I finish this episode I'm going to put this a notebook public so you can find it in the app so description it's not gonna have this Earl it's gonna have another early in the after it's polish but you're gonna be fine it and you can play around with it yourself while you're doing this or you can just create it from scratch and for a while anyway let's see look up X max here X Max here and the random points has to be generated from this so X is going to be 0 to X Max and it's going to be C 0 to Y max and what is there and then it didn't work at all it not whatsoever so this look like what this actually look this actual will look fine okay inspect an SVG the SVG it only draws one circle and then there's circle inside the circle and then a circle inside this you can watch inception I didn't want inception oh it's because I'm not closing the tag there we go okay radius 5 is is too big radius 3 seems more sane this shows you a lot of the power with observable let's let me just when I three generate points here you see that it's updating everything is mmm observing than reactive and nice super cool I love observable remember that I said that we wanted to start classifying we're on the what side of the coordinate system the the circles are or L not the circles the points that we are going circles for we have the data and then we have the visualization of the data anyway I'm going to draw a line come on no oh yes ok let's note I want it to be purple so we're going to call this team a team one and we weren't told this team minus 1 so 1 minus 1 and I'm going to write a little function that we call team and it's going to be take a point and if the point a point X is above a point point why it's it's going to be team team 1 otherwise it's going to be t minus y and then I'm actually going to use this function here to give it a different color so that we make sure that our function is correct all right so it turns out that this team is team -1 and this is team 1 now the tricky thing with teaching machine learning is that the examples that you need to use in order to understand what the hell is happening are so simple that you might as well have used code and that is that is definitely the case in in this in this case so if we look at this this we have already solved the problem here by using this little team function that is that with this team function takes a point and then it actually act actually correctly classifies a point so in this problem this is a very very very simple problem so it's it because it has only two inputs so it has an X and a y-coordinate that that's all that determines what how where in what team appoint and Suppan and the logic is also very simple it's just if if X is bigger than then Y then it ends up in one team or where another and in real life we have much more complicated classification problems for instance let's say that you have a bunch of data on house prices hospice is a classical problem for this so every row might be like a sale or a house so instead of these things these random points x and y coordinates it might be just houses in in a long list and they they each each table row of these houses they have like like a like a square like how big the apartment is might have square meters or swear feet or whatever you use in alien land mytab the where it is located what area it is located it might have like a the year that the building was was built the number of rooms and it might have like 20 of these might serve in more complicated problems there might be hundreds of these things in this problem that we're looking at here it's the X&Y the coordinates but in an another it's another problem it might be aspects of a house that effects the price such as size and and and location and in another problem might be flowers it might be petals and petals size like when classifying images of animals it might be like have pointy ears they have like if they have whiskers and what we want to do is that we want to take all of these inputs or traits or whatever we want to call them and we want to gobble them together somehow with some logic and some not really lawyer like if we did it with programming we would use logic we used a lot of if statements and or perhaps if the house prices this and like if it's the location is this then this meant effect and like but it it works in this simple case but in the house pricing case it's it's gonna be extremely complicated and it's just code is not very suitable if we created a housing price predictor we would take all these housing prices and garble them together and somehow produce an output price based on all their inputs so with the housing prices we wouldn't be able to visualize this weather with a 2d coordinate system because there would be way more dimensions to the problem than there are coordinates in real life like it wouldn't like it would be not three-dimensional it would be like 13 dimensions and humans can visualize that that is when machine learning becomes very abstract and very useful but that's not what we're gonna do today or the reason that I explained that is to give you an idea of why we use machine learning and have you like you have to use your imagination a little bit to understand the why of this because but in this case where a problem is very pedagogically simple ins used these two points let's create an AI we've got create a function call guess that does exactly the same thing as team but instead of using logic to determine where where point is instead of using math it's going to use machine learning so let's just start yes it's just bear with me here yes it's a function guess it's a function that takes Waits and it takes inputs and you know inputs it's like oh that's a general term for things oh my god it's gonna call it point because the point is our inputs in this case remember then the inputs in this case is just two points on a on a coordinate system and x and y y but in a neural netting might be like housing prices or like cat's ears or whiskers it might be like a list of thing and that's called the input but in this case the point is it's fine otherwise we might get confused if I try to think too much about the abstraction yeah yeah let's just okay so let's start there we are going to need some weights because otherwise we've got computer I'm gonna what are the weights the weights are kind of the brain state of our AI the the weights is what we're going to be multiplying every everything with and the weights they're gonna start out random the AIS should just have some weights and then it's going to we're going to train the AI we're going to have the AI guess based on its random weights what a given coordinate is what what team a given coordinate is and it's gonna get wrong because the the the weights are all random it's gonna just multiply it by its weights and it's gonna be pretty off but we are then going to give it the correct position because we know that position in in this case we have a function that gives us the correct answer that the team but in in real life we would have like the actual sale prices for houses that we could give and when we give it that you can see how wrong was I okay I was this wrong then I adjust my weight a little bit towards that wrongness if that's the gist of it don't worry if you're confused that it's normal this is not intuitive in any way but let's just bear with me I'm just trying to give you a sense of where we're going before I start coding so just try to follow random bits whoo yes okay we have some random weights it's an X and it's a Y okay this is the weights this is the initial state of our little AI and this is how its its brain looks like inside it's pretty nonsensical and it will be continue to be nonsensical but it's even more sensible now because it's just two random numbers so if weights are is the brain of our AI then then guess is there a scuttle that we're going to put that brain in and that is why it take takes weights as an argument so how does guess use the weights to think I'm going to show you that but first I'm going to switch shirt and probably switch lighting and hair do and maybe complexion as well because I'm going out and meeting some friends and continuing this later one two hello I'm mpj from the not-too-distant future I'm indicating this by this hat I'm sitting around had vivid editing editing this video and I realize it's very long so I've split it into multiple parts so cliffhanger please remember that this episode was kindly sponsored by pusher so please check them out at pusher top front of action column you can subscribe here so that you don't miss the next episode on on Monday or if you are watching this from the future then you can watch the next episode right here and otherwise this is gonna be just a recommended video I think