I'm Sorry Dave, I Can't Do That: Practical Machine Learning for Information Security | SANS@MIC Talk

Key Takeaways

all good evening everybody so my pleasure to meet you my name is Dave Holzer I'm up here in New York right now we're in the middle of a heck of a storm right now so we'll see make sure we don't have any electrical problems at least I hope we don't have any electrical problems during this and I'm gonna talk to you a little bit this evening about some machine learning stuff and before we get started I just want to take a minute and explain a little bit about what our presentation is about tonight what it's motivated by and why I'm choosing to do this thing I mean personally I think it's really awesome that fans has not only reacted to the situation that we're all in right now and particularly those students who were planning to be attending live conferences in the next month or two but also just opening up all of this free stuff for folks and know that I think it's so important I mean we the faculty who are doing these we're not being compensated for this either it's just something we can do to kind of give back and help people take their minds off all the bad things that are going on and I'm sure that some of you have some really serious things happening I know for me we have a really good friend and business partner in the United Kingdom he he himself is sick his wife is second is now in an emergency room unresponsive so it is a terrible time for all of us but let's just take a little while about an hour hour and a half and kind of take our mind off of all of that learn a little bit about machine learning and and something really special that we have that's coming out in the next month or two now as I go into this I want to explain what this talk will do and what it won't do and maybe help some of you to decide that now this this might be a talk you want ask it for example if you are someone who is already engaged in machine learning research and you have a good deal of knowledge about it you probably are not interested in this talk this is gonna be very high-level very fast and it's just enough to introduce all of the major terms so that someone who doesn't have a lot of familiarity can have a conversation an intelligent conversation maybe with a vendor maybe with their boss or someone who's looking to procure something so that you can decide whether or not machine learning is a good fit for you so that's part of what we're doing but there's more now for those of you who are listening to that and saying no I'm not a machine learning researcher I'd like to maybe learn some of this part of what we're hoping are going to get out of this is the fundamentals of what this actually is what AI and what machine learning is because if if you haven't noticed pretty much every vendor out there is telling you that they have a new AI solution and does machine learning and it's just great now of course vendors have made this same promise for a very long time and I myself have a great deal of concern about that in this particular space as do data scientists in general I was having a conversation with a couple of people on reddit earlier today because someone just published a paper about some AI thing they've created that solves kovat 19 it's gonna cure it and in fact it doesn't do anything like that they've they've taken someone's code made a couple little adjustments and it's it's not doing anything meaningful at all but because AI and machine learning are so much the buzzwords of what everyone is trying to sell you and what everyone's telling you you need then if it has machine learning or AI on it people are interested people want to sign up people want to buy it now there's risk they're not just the risk of the thing didn't work I think there's a much bigger risk and the AI community the machine learning community has suffered through this many times over the past decades because for instance even some of the techniques were going to discuss tonight were very well understood in the 80s and 90s we just didn't have computer systems we could do it with but all of the math was done and the evolution of the computers that we can do with this do this with have been primarily bit driven not by machine learning research but instead by graphics processing units and their application and the strong desire in the consumer market for gaming and as those things have advanced the turns out that those same devices are incredibly powerful for doing machine learning and we'll talk a little bit about why that is so one of those big things we're trying to get you to come away with is what do these terms actually mean how does it actually work and that fear I was mentioning is that because of all of the overhyping now you might be too soon for me to say that but as we work through a little bit of this there should be some intuitive kind of aha moments as we go through it and I'm hoping that at least a few of you are going to say oh my goodness is that all this is maybe it's not really where it needs to be yet that doesn't mean it's not useful where we're actually going to continue through and at the end we're going to show you a really practical application of something you can do in real time today using machine learning and information security so I hope you'll find that useful and that's actually something I've kind of stolen out of our 503 class part of what we're gonna wrap up with and it's also something that's talked about in that new AI or machine learning class that we're working on right now and that's not going to be at conferences that's just going to be done as a web series and that'll be available sometime in May or June but anyway our fear is that because of the overhyping we're going to see that another AI winter sets in another drought because all of the funding will dry up when all of a sudden the whole industry realizes everything has been over promised okay so if I've said a lot in there I I do see that my friend Andy is saying that I've got the wrong screen up here so if Andy if you're not looking at I'm sorry Dave I can't do that adventures in machine learning do let me know as far as I know you should be looking at a Jupiter lab page so if that's not the case do someone do let me know you can of course put things over in the chat I've got it up here on my screen so I'm trying to pay attention to that and oh great I see some people saying that that's the right screen thank you very much I took off my bad artwork early yeah I'm just actually thinking about renaming this adventures in machine learning and bad artwork so I'll just throw that up there for a second so that's my freehand Hal 9000 and any event will be back to that a little bit later and have a couple of drawings I'm going to do with you let's get started because I only have about 60 to 90 minutes here and we a lot of ground to cover as we cover that ground I am definitely going to be do some doing some hand wavy stuff and that is most especially around the Python code because all of this stuff um as you can see here I'm using a thing called Jupiter lab which is kind of the newer iteration of Jupiter notebook and and this this talk tonight is not about the Python I'll point out a couple of things about it if you're curious about it or wanna see some of the excerpts of it you know you can get in touch with me I'd be happy to share that with you but I want to talk about the terms and how this works so we understand it and give you a practical application and some really strong intuitions about it with regard to that there will be some math involved you're not going to have to solve any math problems and I have tried very very hard both in this presentation in the 503 class where we have this right now and additionally in the new webcast series that we're doing to make sure that this is very accessible meaning that you do not need to have an understanding of advanced mathematics and what I'm calling advanced mathematics is really from the perspective of someone who is not a mathematician because if you are a mathematician the math that's involved in this is actually not advanced mathematics at all it's it's actually rather fundamental mathematics but as if you're someone who's perhaps may only take in high school mathematics or maybe taking calculus or calculus to this can look like or seem like advanced mathematics anyway let's get started I do see that Vijay is asking if this is going to be available this will be available in some form but I'm not planning on publishing this notebook until we release the class now if you're someone who is taking 5:03 you actually will have some of the scripts that are at the end at the very end they're already available in that class and you can get some of that out of the show me the packets repository I'll just put the name there I'm not going to go look up the whole link show me the packets repository which is out on github not everything is there but many of the pieces necessary to make this have are there alrighty so we're going to get started with this and we're going to introduce this idea of machine learning and by the way we're not using slides I'm sort of on a campaign to outlaw slides as much as I can so we're just gonna use this Jupiter notebook for our machine learning we're going to start with a problem that doesn't sound like machine learning at all we're going to start with a topic called linear regression which already especially for me it's it's 8:30 here so I'm I'm well past my bedtime linear regression does that sound like something that's not a math topic it is but it is not a hard math topic one of the things you're going to find as we go through this and introduce and explain some of these terms is that much of what's happening in these terms is simply trying to assign a precise term to something which can sound really hard if you don't use that term everyday but it's actually a very simple term for example linear regression linear is just the line regression regression is a pretty easy term as well regression is really about the idea repose M about the idea of fitting now I want to illustrate that for just a second so let me just pull over to my my iPad here just give me a second to get the right page up and let's use this one here and this is just a rough drawing of some data that we're going to be looking at in a couple of minutes and just to explain what's happening here in this graph we've got some data and I'm going to say that this comes out of a NetFlow repository within an organization and what it's showing us is over time how many bytes are being transferred across the network this is the kind of thing the raw data that you would be producing out of mrtg reports and other kind of utilization reports and creating charts can graph stuff here we're just looking at it as raw data and showing some histogram data so we're saying that at at this hour over here we ended up with this number of bytes so that's all that's happening in this Kraft now when we look across this graph you can see that over time the the data is not it's not a straight line and that's okay right now we're talking about linear regression which I said regression is about that word fitting and we know linear is about lines so here's what that means I'll give you a very simple definition and for those of you who have more math training than others don't be overly precise please allow me to be a little bit loose about how I define some of these terms right but I'm going to say linear regression will simply be can we look at this data and could you and I perhaps draw a line that best approximates what's happening in that data that is a linear regression well not exactly I mean there's an algorithm involved right now I'm just eyeballing that the algorithm that's involved is we would need to be a bit more precise for instance if we were to pick a point like I'll pick this point right here this little orange point and you can see that my line is rather far away from this point up here which is the point happening at that time slot so so my prediction my line is wrong I made a mistake it's it's an error in fact there's an error function that we could assign to this we could decide I'm going to figure out how far away from the line all of the points are so I'm just going to make up some numbers here let's say that the first point is two below and the second point is two above and the third point is four below and the next point is six above and the next point is eight above and the next point is ten below and we did that for everything I could add all of those up and it would give me the total error although you may already see an intuition that's telling you there's something gonna go a little wrong here because if I add all those up so the negative two and the plus two those kind of counts just ignore those minus four plus six so we're currently at plus two plus eight that's going to be 10 minus 10 so the overall loss is zero which seems to imply this line is perfect it exactly represents the data it fits the data but clearly that's not the case it is very far away from a lot of the data so there's a different way we tend to do this we do a different thing called a mean squared error where we take each error so if my first one was negative two and we square that so that would be for my next error was two that would be for my next error was six that's 36 but by using the square we preserve its magnitude we actually make it bigger but we preserve its magnitude and if we add that all up we now have a much better approximation of how wrong this line is all right so let's let's move away from the math I know you might already be thinking that feels like more math and I'd like to do I'm with you let me just get my notebook back up here so what we're working with though is this idea of the formula of a line which is really the only math you need to have you need to understand or know before coming into this and I feel like that's pretty rudimentary that's maybe ninth or 10th grade in the United States Y is equal to M which we usually define as the slope times the x-coordinate plus B which we define as the y-intercept the formula of the line that formula turns out to be important because what we're going to do is ask the computer to figure this out for us and it's going to use an algorithm that tries to minimize the mean squared error so if it can get that error as small as possible that means that is the very best line that fits that data to get started I've got some Python stuff here that's gonna read in some flow stats I've got which is as I've said to you hours with number of bytes that have gone by now just as an artifact of the data collection process that this comes from I end up with this third column over here that that doesn't really exist the the data is really just the our and the bytes but I didn't want to massage it too much wanted to give it to you just as it is so I just need to get rid of that column now I'm going to tell you something now and I think you will appreciate this much more as we go through the actual machine learning pieces the majority of your time and effort in creating machine learning solutions is spent processing and cleaning your data that's it it turns out that actually building the the neural network that's going to do the the predictions for us is seconds or minutes worth of work especially using the process we're using here I am using a more simple kind of approach toward a really nice interface to it though if you want low level access to do that you can spend a lot more time actually writing custom ways to classify data but still it's the cleaning and processing or pre-processing of the data that takes the time now this is not a lot of pre-processing I need to do I just need to make this column go away so I've got some some Python magic here don't let it distract you what's happening in there this axis and stuff but all that it's really done is it has chopped off that column that's it so I now have 112 rows of two columns column number one is the hour column number two its marked as flows but it's the number of bytes now I'm going to also make another little thing here it's a super common practice that when you're trying to predict something with your data you'd you'd like to hold back some of your data hold out data we can actually even call it that hold out data that we're not going to Train earth our machine on instead we're going to use some of the data for the training and then I can use the other data to see how well a computer does at predicting the rest of the data now I'm gonna just give you a little four gleam here it's not going to do a great job at this prediction but that's okay there's a totally different intuition and we're looking for you to get out of this we're gonna move into some machine learning that tests some really good outputs in just a few minutes but now that I've pulled that data aside and I've created a thing called bytes that's got the first 60 rows and then I've got one called all bytes that has all of the data because we want to do some comparisons and see how this does over time this is just showing that in fact I have loaded that in and it's printing the data out for us in scientific notation it's just the default exponentiation that that python is going to do in a print stuff out and as I said to you we are going to avoid talking about all the Python stuff so I've got a whole big Python function here that we're just gonna wave our hands and say this Python magic is going to create some really handy functions for us that will allow us to plot things on the screen that's it now if you're interested in the details of that I will just leave it on the screen for just a moment you could come back and maybe snapshot that or you can get it out of the notebook when it becomes available but this is just allowing me to graph data out that's it and here's the data and that kind of looks familiar that's something like what my little drawing looked like we can see bytes over time and what we'd like to do then is as the computer to try to to fit that or do an approximation of the data because we said that that regression idea now I just drew a line all I did was approximate it by eye but a regression is actually doing an algorithm it's iteratively looking at a line compared to the data and figuring out what the loss is the loss or the error to see if it can come up with the best fit that's what a linear regression is now to make that work I'm going to just create this error function and this Python here because we're not reading Python so I'll just translate for you that is implementing the thing I described a mean squared error what I actually did you was a sum of the squared error we would just have to divide that by the total number of elements which is exactly what this is doing so it's a mean squared errors which we're coming up with actually I'm just noticing I I forgot to divide it by the number of elements but I'm just gonna leave it as is it's gonna give us a raw error score and that's totally fine next look at that I'm done the machine learning has happened I know it doesn't feel like a lot happened here's what's going on I have called a function that numpy numpy is just a mathematical library for Python provides called polyfit the polyfit allows me to try to fit a curve or a function to data which is exactly what regression is since I'm talking about linear regression that would be a first-order function in other words the highest power found in there would be the power 1 where did that come from if you remember at some point in your high school career you you dealt with things like polynomials where you had x squared plus 2x minus 4 and that would be a second degree equation because or a second-order equation because we had x squared the highest power if we have y equals MX plus B a line the power is one X to the first we don't write it because it's X to the first but it's a first order equation and what I've done here is I've said please do a poly fit for a first order equation trying to find the terms that would best fit that function and when I do that it returns to me two values the values that it is returning to me here are effectively the M value and the B value from y equals MX plus B so in other words this function here is what we end up with it has figured out that that is the best fit so now wait a minute this is very important machine learning I said that machine learning is all about trying to algorithmically or programmatically minimize an error function and I want to be clear that the kind of learning that I'm talking about tonight is very specifically supervised learning supervised learning is an approach toward machine learning where I'm usually trying to classify things or I'm trying to come up with some kind of a value and I'll define the difference between a regression and a classification in a little while because there is an important distinction there but effectively I'm trying to make a prediction about something and I have some known data that I'm trying to work out rules for or trying to work out a function for having have known data and training against it is more than anything what defines something is supervised learning in other words what I'm doing is feeding in data and feeding in outcomes what the answer should be and asking the computer to work out what are the rules to make that happen and that was a fundamental change that happened in machine learning in the 90s and going into the early 2000s because if you remember the 1990s and you were working in information technology you are likely familiar with expert systems and in expert systems you spent you spent not the computer you spent a lot of time writing rules for the computer to help the computer take the input and turn it into the proper output that all changed it all got turned on its head as we moved into this model the most common model that we use today where we feed in predictions outcomes with inputs and ask the computer to work out what the rules are which is exactly what linear regression is I plug in data points I ask it figure out what the coefficients should be for a linear function that approximates that data that best fits that data or minimizes the error if I look at this one it tells me that the error well this error doesn't look so good 1.3 times 10 to the 16th my goodness we are very far off of the error and again I did forget to divide by the number of elements there do I want to bother with that and no I'm not going to bother with that but we could it would not be difficult to do but it makes it look like it's a huge raw error and it is but if we plot it out it turns out that that our graph is actually not very far from what I drew so our our gut intuition was just draw a line there and that's about the best you can do that was a pretty good pretty good guess but could we be better so far what we should know is that machine learning is about fitting the data or creating a function that best fits the data what we're doing is feeding in known data with known outcomes to see if it can work out a formula or some function that minimizes the error and that we can then make predictions with that is effectively what's happening so maybe there's a way that we could get a better approximation of this data and I want to introduce an idea about that because there's another important intuition that we can get out of this if you took calculus now don't worry we're not going to do any but if you took calculus one of the major topics in calculus is sequences and series and unfortunately the way it's taught is really not very intuitive it doesn't help you to understand how it solves problems but in the sequences and series section of your calculus class what I'm about to say might actually explain a lot of what you were trying to be taught in that section that perhaps you didn't understand the idea behind an infinite series in particular I have in mind a thing called the Taylor series which is an infinite series that can be used to approximate the value of sine now sine you might be familiar with that it's a trigonometric function and it's a bit more difficult to create a linear regression on this one because if we were to try to linear regression the answer would be a straight line but you can see that most of the time that line is really wrong this function however is a very difficult thing to try to to kind of quantify now it's easy to calculate it's easy to talk to in terms of relationships of angles but how do you write it as a function sure I can write sine of 30 degrees but that's not what I want I want something like you know x squared divided by 32 plus 15 that's gonna give me the sign and that's where the idea of a Taylor series comes in or an infinite series it turns out that we can create a prediction we can create a function that could very very easily and very accurately predict this one point right up here at 7.5 we could do that and if we added a couple of terms to that we could also predict some of the values a little bit to the left and a little bit to the right and if we added more terms to that we could predict further left and further right and the more terms we add the more to the left or the right we can predict in fact if we were to create an infinite number of terms the Taylor series would produce a curve that looks exactly like sine but just because we say it's an infinite series doesn't mean that we have to go to infinity that's what a Taylor series is designed to do but the way you use that kind of thing is you say to yourself how accurate does this need to be to how many decimal places let's say and then based on that you work out how many terms you actually need to calculate so even though it's an infinite series as long as you are fairly close to the data or the point around which you're doing your prediction you might only have to calculate two three four terms out of that infinite series that's very tractable so let's see if we can take that idea that may be maybe making this a higher order equation right now we did a first-order equation what if we tried to do it as a second-order equation a polynomial or maybe a third or a polynomial with a third power or let's go for broke a fifteenth power and if I just try those I'll have a do the fitting for those it tells me that the first one the first order that was one point three times ten to the 16th that's how far off it is the second order is 1.0 times ten to the 16th that's better the next one is even better and look at that the fifteen daughter is even better it's still far off but it's even better so that makes it seem that more terms must be better higher order equations must be better and if we graph this out note what we get here's our straight line that's our first order equation and then this this curve here is the second order equation and certainly as as we go here from 30 to 40 to 50 to 60 it is definitely a better approximation of our data than what we see happening with our straight line however to the left we do have to admit that at numbers less than 10 at values less than 10 it actually seems to be heading in the wrong direction let's try adding another term to that let's make it a third order equation if 2 is good 3 must be better and when we do that oh wow so it is now low less than 5 it comes up and then it it seems to be tracking better than the second order equation this looks great what we're doing is just manipulating the values to see can I better fit this one of the ways that this applies and it's not directly the same but it's analogous we're going to go through doing some training with two different machine learning strategies we're going to start them very very soon but when we do that one of the things we have to do is figure out how many epochs how many times should we run the training data through more must be better right and similarly more neurons must be better all kinds of things more must be better but is that all waise the case hello just see I can't read this so yeah you're right where I'm saying mark why why not just use a thousand now I'll be honest if I use a thousand pythons gonna start to complain so let me just let me just try that I'll change this guy to a thousand and you can see whoa python gets kind of upset by that it does not want to do that that is not a valid value for it it's just too large you'll in fact see it starting complaining much earlier than that like here it's saying I cannot properly fit that just because the numbers are getting too large something to the 40th power is just too much for her to handle handle let me go back to the 15th power and yeah I see my friend Eric is saying perfect is the enemy of good enough and that's exactly what's happening here let me just recompute these now that I change that and let's just scroll down we have this the second order there's the third order it looks better but here's the fifteenth order equation this doesn't look so good in fact what we've done is we have just done something like overfitting or overtraining overfitting and that could be a difficult concept to get but this illustration to me really really makes clear what's happening you can see that we kind of start off okay but the longer we go the more out of control this gets when I was looking at the second and the third order equation they were tracking with the data in a very real sense you could you could say they were kind of standing back and trying to globalize what was happening they were looking at the overall curve the overall data but when you look at the 15th order equation it it no longer looks like it's really looking at the big picture instead it's as though it's so fixed on trying to make it to this point and make it to this point and make it to this point and make it to this point it's so fixated on that that it can no longer see the whole curve in other words they're missing the big picture because it is so focused on the individual details of what's going on you've heard the expression you can't see the forest for the trees that is effectively what's happening here and there's two ways that that can go wrong way it could be that we are overfitting that could have to do with how we are actually creating our model how many neurons for instance if I am trying to deal with something that deals with I don't know a hundred and twenty bits of stuff and I create a thousand neurons to handle that I am going to weigh over fit on that which means that there's no need for the network to learn anything it can just remember every bit from every example but the other way we can use this idea or this picture here is this idea of overtraining overtraining is my network or my neural my machine learning algorithm becomes so good at spotting the training data like what this is doing its dashing from point to point then it has no hope of doing any useful predictions in the real world and in fact let's just prove that true here I'm now plotting out the all of the data remember I had set aside some of the data so that we were only looking at a small section of it for our initial training but there was a much larger range of data so we could evaluate predictions and it's kind of interesting this data turns out to be cyclical and that's very common for Network data where we have some spikes but then what we dip back down and as odd as it is that lime turns out to be a much better approximation than what's happening in the second order equation and certainly in the third order and absolutely in the fifteenth order which is so far off the charts you can't even see the real data down here anymore because it has totally over trained or over fit on that data so let me just pause for a second linear regression is not the point of this the point here is what we've taken away so far we can say that machine learning is simply using some algorithm and don't worry about how the algorithm works that's not important for us right now but it's using some algorithm that automatically adjusts coefficients in a in a function in order to best fit the data that allows us to make predictions the way it makes those adjustments so how does it decide you know if it's wrong is it's measuring the loss which we said is the error it's the amount off you are that makes total sense we can even imagine ourselves solving a problem that way and trying to see well how far off am I let me make a little adjustment here a little adjustment there and if you think about this in terms of maybe measuring something physical like you're trying to fit something into a space and making a piece of trim to go around your window so you probably have been taught the saying at some point measure twice cut once even then when you make that cut it's always better to make it a little bit too big because you can always make it smaller and adjust and adjust and you know it's too big because it doesn't quite fit fit it kind of overlaps here we're looking at it mathematically oh it's a little too big so I need to make it smaller oh it's a little too small I need to make it bigger so we can look at those numbers look at the error and see which way does it need to be nudged the computer is doing that as well it's looking at the slope of that error it's saying is the error going up or is the error going down if it's going up well then I need to make the number smaller if it's going down then I'm going in the right direction keep going that way so that's all that's happening that loss is the same as the error that term that's all it means but it's all the same stuff it is really saying how far away from reality are we and there's another term you'll see here it's used in some data science is used in statistics and machine learning ground truth reality and ground truth are the same thing facts not alternative facts ground truth or facts things you can actually measure alright so then let's turn this into real machine learning because linear regression is machine learning but it's not what most people would think of as artificial intelligence or machine learning in fact though even just take a take a pause and think about that if you polled most of the people in your enterprise particularly the people who want to buy or spend money on or even the salespeople who are telling you to buy machine learning and AI solutions is that what they think it is or do they have a totally different mental picture about what AI is we are totally over selling and overhyping what AI does today but that doesn't mean that it doesn't have value so let's let's look at another example of that we're going to take a quick stroll through kind of the hello world of machine learning and the problem we're working with is a thing called binary classification you'll also find this called sentiment analysis it's the same thing binary classification means I'm trying to see if it's one thing or the other it's two different fameless' this is being called sentiment classification as well because that's what the original problem was that was proposed can you look at a movie review and based on that can you create some algorithm some process that will learn to identify the sentiment is it positive or is it negative that's all this is doing now the problem we're going to work with here I know it's not an information security problem but we are going to take some of the fundamentals we learned here and turn it into a much more important problem we're identifying where we are identifying unknown protocols passing on a network in real time and to me that's a valuable problem so we'll just introduce it with this here the idea here is we've got movie reviews that IMDB generously donated to the to the world and in these in these movie reviews there are 25,000 movie reviews that are designed to be used for training and 25,000 that can be used for validation to see how well you did this problem as we work through it is is probably going to seem pretty easy but know that less than 10 years ago this was a cutting-edge problem and the the solution that we work out with here would have been a prize-winning solution just ten years ago for this problem but machine learning has definitely made tremendous advances and there are far better things that we can do than what I'm going to demonstrate here why am i demonstrating this then why am i using something not optimal because there's some important takeaways I have for you that are related to what machine learning and AI is actually capable of before we turn that into a practical information security problem now I want to explain the structure of this data a little bit because it will explain some of what's happening inside of this notebook we've we've got the movie reviews and we've got the labels and fortunately someone has already done a great deal of work massaging this data because these are words but we've already seen that really this is adjusting coefficients and functions and words don't really fit well into functions so how can we make those two things work together well but someone has already done has taken all of the reviews and they have encoded them the way they've encoded them is they have taken all of the words that are used in all of the reviews and they perform some frequency analysis on that the word that appears the most frequently is listed as the first word in the data set the word that occurs least frequently is the very last word in the data set why they've done that I'll explain to you in a few minutes it'll become obvious why that was a good choice but then because they've created these Indus indices these numbers that associate with the words with this with this table with this dictionary it means that if we had a sentence like the quick brown fox jumps over the lazy dog and the way the index worked was though was most common fox was second most common Brown is third is his fourth quickest fifth well then this review would be encoded as 1 5 3 2 7 1 1 1 6 8 the quick brown fox so the numbers represent which word we've used that still might seem a little bit strange to you there's a really good reason we've done it and it's in this part right here I'm about to load this training set in and I don't know if you can hear fan noise where I am I've got a big beefy machine that we use here that's got a couple of GPUs and in poodle's of memory and disk space where we do all of our machine learning research and that kind of stuff because you need a lot of memory and GPU is really make the training much much faster and in loading this in I've told it to use the most frequent 10,000 words that appear in that dataset 10,000 words now there are far more words I seem to remember that there are 83 thousand words in that total dictionary but that would mean that I would need to keep all 83 thousand words in memory at the same time which doesn't sound bad but let's make that more difficult I need to keep in memory 25,000 rows of data that are 83 thousand columns wide that is a lot of memory so to make that simpler I'm just going to use 10,000 words and it turns out you don't need all 83,000 words to do a pretty good approximation of what's happening so here we have a little bit of Python magic it loads in that IMDB data and here it's just showing me what that data looks like so I've got the very first review there it is 114 22 16 and as we know that represents the words inside of there I've got a training label of one which I seem to remember one is a positive review and zero is a negative review and then I've just got some examples of some some words so you can see Fon is worth 34 701 sukhino is 50 206 just a couple of random words there this is nice for the computer but we'd like it to be readable for us so I've got a little function here that is going to turn that into something that you and I can read and I've also written a little function here that's going to tell us whether or not it's positive or negative review again I don't want to read the Python with you but you can see here we have taken that review and turned it into text now I just want to explain that in this text you'll see there are question marks the question marks indicate that the word that was in that place is not in the index that I have loaded remember that's an 83,000 or a dictionary but I've only loaded the first 10,000 words so there are some words that it doesn't populate even so let's see it says this is positive that's how it's been pre classified that's the training label it has it says this film was just brilliant casting location scenery story direction everyone's really suited to the part this does sound positive all the punctuation has gone that's just an artifact of how the date has been manipulated but we can reconstitute these they're sort of freeze-dried but we can pull them back together let's just take a second what we need to do though is turn this into something that I can push into math push into math and to do that I need to make it something that's easy to manipulate so I'm gonna for a second here I'm gonna pull up my iPad again I want to take a stab at explaining something and I'm gonna try to make this really simple there's a field of mathematics that we primarily use when it comes to machine learning there are a whole bunch of things that are used I mean we are using calculus and partial differentials there's a lot of that happening but the majority of what's happening in machine learning is actually making use of linear algebra and linear algebra sounds a lot like it has to do with lines like the algebra of lines which is not really true it's really at its most fundamental it is really about transforming things from one space to another and the way we represent things for instance if I gave you a coordinate so if I gave you a coordinate like 1 comma 2 comma 3 put that in parentheses that coordinate there's another way I could write that I could also write it in row-major form as 1 2 3 don't worry about row-major or as column-major form again don't worry about that 1 2 3 which for some of you with maybe a physics background or other kinds of education you'd look at these and say oh that's not a coordinate those are vectors and I know that many have been taught especially in in physics that a vector is something that has both magnitude and direction and that is true in physics but the mathematics definition is not that restrictive in fact if you look at the definition even in a physics book what you'll see is that a vector is something that has and it will say something along these lines a vector is something that has both magnitude and direction comma for example because there are many other things that make up a vector for instance this right here these three coordinates that is a vector but written as coordinates here it is written as a vector as a row here it is as a vector as a column I have over here you might be kind of looking what's going on back here you can see I've got this this attempt at a 3d cube sitting out there on those 3d axes if you wanted to draw that on a piece of paper though or you wanted to draw it on a computer screen you would have to do some kind of a translation because a cube like a rubik's cube that is a 3d thing but your computer screen is flat so you need to translate it from that 3-dimensional space down to a two-dimensional space and it turns out that we can use the linear algebra which is forget lines linear algebra is the algebra of vectors it is the manipulation of vectors or let me make a more complex vector or matrices which are vectors with multiple rows and columns which you'll also find the word tensor around I'm not going to go into the depths of what tensor really means but we can for ourselves we can think of it as a matrix or even a matrix of matrices groups of vectors that's what a tensor is and the main product at least the one that I work with most often and there are others but it explains the naming of Google Google's product tensorflow it's really about the mathematics of vectors and how we can manipulate them and it allows me to take that 3d cube and to represent it in 2d space which turns out to be a super simple thing if you want to make it absolutely the simplest projection proble all you do is you get rid of the Z coordinate or however you choose to mark them I almost always think of these as XY and Z and I know that I am NOT in the majority in that one and a lot of people will label the up and down as Z it's just kind of where I started that's kind of how I ended up there anyway so linear algebra is about translation what does that have to do with machine learning why does it matter well in order to use linear algebra why do I want to use it at all because it turns out that your GPUs are phenomenal linear algebra systems they are phenomenally good at doing parallel calculations on matrices all that we do in computer graphics is based on linear algebra so those devices are exceptionally fast that they're optimized for it so we're going to leverage linear algebra that's what the algorithms do and there are some very specific rules the only one that we really care about is that in order to leverage linear algebra to solve our problem all of our vectors have to be the same shape the same shape pains same number of rows same number of columns now in our case E Rowe is going to be a a review but a review different reviews have different numbers of words some reviews will have ten words some refusal had eighty words how do I make them all the same shape and if I were arbitrarily say let's make it a hundred and we'll just pad out the ones are they're too small it doesn't matter what number you pick you're either wasting a lot of memory or you will truncate reviews so we're going to take a slightly different approach what we're going to do is vectorize the data Victor eyes well we know what a vector is it's an ordered group of numbers an ordered list of numbers I'm going to take the review so here is the review we currently have and I'm going to translate that into an ordered list of numbers into a vector of ten thousand columns and that ten thousand might sound familiar because we loaded in ten thousand words to remember that so there's ten thousand words each one of those words is going to represent one column in our data in other words if word number one is present there will be a one there if word number two is present there will be a one there if we're number three is not present there will be a zero in fact if you look here I've printed out the review as numbers here but down here I have printed out the vectorized form of that word number one does not appear in that review where number two does where number three does word number 10,000 does not now fortunately Python here is saying there's no way I'm printing out all 10,000 of those so just take my word for it it is now just marking every word that appears it's not counting how many times that word appears it is simply indicating that word is present and there's a name for this approach this is called the bag of words approach so you can imagine you have a bag and you say oh I've got a vow here so I'm gonna put a sign back and you'll only put one copy of each word in the bag but now all of our rows will be 10,000 columns wide that means our vectors are the same size with that done man I am ready to do some training and and down here I know it says a bit more data manipulation I'm just doing a little bit of a bit of manipulation here to pull out some validation data don't worry about that at all but this part this part here is everything required to build this neural network so now linear regression anymore neural network this is a deep learning neural network that's it and if I simply run that so we have to give it a second year off it goes look at that it's telling us about the training the magic has happened in its first pass it had an accuracy of 77% while training after it finished that pass it took the validation data and checked how well things were going and it was up to 87% it then started so this was the average over the whole epoch or the whole iteration the whole set of data and then here we now end up with an average of 90% wow that's much much better so this seventy seven point eight was average this probably started at around 89 or 90 percent but the validation accuracy is lower than that it did go up it was 87 it's now 88 so that means that it is still learning but why is the validation accuracy different well remember it's using the training data to actually adjust all of the coefficients to figure out how to best fit how to minimize the error the mapping the translation and actually I'm going to give you a big idea here I showed you a 3d cube and I mapped that to a TD 2d space so I want you to think about this a totally different way what we're doing is creating a 10,000 dimensional vector that describes a movie review and we are translating it through a function that maps it into a space of either positive or negative review so we're translating the 10,000 coordinate system to a two-dimensional system 10,000 dimensions to two and of course none of us can think about what 10,000 dimensions look like it's just mathematically abstract but that's what's happening so in the epoch in the training data it is working with that data to make those translations from 10,000 dimensions to just two and then it takes that holdout data that validation data and says how well are we actually doing and based on this 88% it's gonna feedback some information it's not training on this but it can feed that information through and make some more adjustments so our next pass through where it's 92 percent we have improved slightly we're now at 94% on the training data and we've improved slightly we might benefit from some additi