Complete Machine Learning in One Video | Machine Learning Tutorial For Beginners 2026 | Simplilearn

Key Takeaways

Hey everyone, welcome to our machine learning full course by Senon. Have you ever thought about how Google finishes your sentences? How Netflix always know what you want to watch or how chatbots seem to understand what you're saying? That's all made possible through machine learning. And in this course, you will learn all of that. How it works and how to build smart systems of your own. You might be wondering what makes machine learning so special. Well, it's a big part of how today's technology is getting smarter. From recognizing faces in photos to recommending products online, machine learning is everywhere. It help computers learn from data and make decisions without being told exactly what to do. By 2025, machine learning will be a must-h have skill in industries like healthcare, finance, marketing, and many more. Companies are looking for people who can use machine learning to solve real world problems and create intelligent solutions. And here's the exciting part. The career growth is huge. In India, machine learning experts can earn anywhere between 10 lakh 25 lakhs per year. and in the US it can go beyond $120,000 a year. This course is perfect if you're just starting out. We'll begin with the basics of what machine learning is and then move on to important concepts and techniques that are used in real world projects. Step by step, you'll learn how to go from beginner to someone who can confidently work with machine learning. So, let's get started on this journey. >> Welcome to machine learning tutorial part one. This is part one of a machine learning series put on by SimplyLearn. My name is Richard Kersner. I'm with the SimplyLearn team. That's www.simplearn.com. Get certified, get ahead. What's in it for you today? Well, we'll start off with a brief explanation of why machine learning and what is machine learning. And then we'll get into a few of the types of machine learning. machine learning algorithms, linear regression, decision trees, support vector machine, and finally, we'll do a use case where we're going to classify whether a recipe is of a cupcake or a muffin using the SVM or the support vector machine. Sounds like a delicious way to explore machine learning. So, why machine learning? Why do we even care about having these computers come up and be able to do all these new things for us? Well, because machines can now drive your car for you. still very in the infant stage but it's just exploding as we see with uh Google's Whimo and then Uberhead their program which unfortunately crashed. They know that this is huge. This is going to be the huge industry to change our whole transportation infrastructure. Machine learning is now used to detect over 50 eye diseases. Do you know how amazing that is to have a computer that double checks for the doctor for things they might miss? That's just huge in the health industry. pretty soon they actually do already have that within some areas where maybe not for eyes but for other diseases where they're using the camera on your phone to help pre-diagnose before you go in and see the doctor. And because the machine can now unlock your phone with your face, I mean, that's just cool having it being able to identify your face or your voice and be able to turn stuff on and off for you depending on where you're at and what you need. Talking about an ultimate automation our world we live in. And as we dig in deeper, we have a nice example of Facebook. As you can see here, they have the Facebook post with Halloween. Comment yes if you want it. Order here. Nobody likes spam posts on Facebook that annoy them into interacting with likes, shares, comments, and other actions. I remember the original ones were all if you don't click on here, you will have bad luck or some kind of fear factor. Well, this is a huge thing in a social media when people are getting spammed. And so this tactic known as engagement bait takes advantage of Facebook's newsfeed algorithm by choosing engagement in order to get the greater reach. To eliminate engagement bait, the company reviewed and categorized hundreds of thousands of posts to train a machine learning model that detects different types of engagement bait. So in this case, we have we're using Facebook, but this is of course across all the different social media. they have different tools are building and the Facebook scroll gif will be replaced kind of like a virus coming in there it notices that there's a certain setup with Facebook and it's able to replace it and they have like vote baiting react baiting share baiting they have all these different these are kind of general titles but there certainly are a lot of way of baiting you to go in there and click on something so they fed all this this data was fed into the machine and then they have the new post the new post comes up that takes over part of the Facebook setup up and that's what you're looking at. You're looking at this new post that's replaced like a virus has replaced that. So what Facebook did to eliminate this is they start scanning for keywords and phrases like this and checks the click-through rate. So it starts looking for people who are clicking through it without even looking at it or clicking through it and it's not something that normally would be clicked through. Once Facebook has scanned for these keywords and phrases, it is now able to identify the spam coming in and this makes your life easier. So you're not getting spammed. It's not like walking through an airport and in a lot of countries you have like hundreds of people trying to sell you time share. Come join us. Sign up for this. Eliminates that annoyingness. So now you can just enjoy your Facebook and your cat pictures. Or maybe it's your family pictures. Mine is family. Certainly people like their cat pictures too. Another good example is Google's Deep Mind project Alph Go. A computer program that plays a board game Go has defeated the world's number one Go player. And I hope I say his name right. Kiji, the ultimate go challenge game of three of three was on May 27th, 2017. So that was just last year that this happened. And what makes this so important is that, you know, Go is just is a game. So it's not like you're driving a car or something in our real world, but they are using games to learn how to get the machine learning program to learn. They want it to learn how to learn. And that is a huge step. A lot of this is still in its infant stage as far as development as we saw what happened with the as I referred to earlier the Uber cars. They lost their whole division because they jumped ahead too fast. So still an infant stage but boy is this like the beginning of just an amazing world that is automated in ways we can't even imagine what tomorrow's going to look like. We've looked at a lot of examples of machine learning. So let's see if we can give a little bit more of a concrete definition. What is machine learning? Machine learning is the science of making computers learn and act like humans by feeding data and information without being explicitly programmed. And we see here we have a nice little diagram where we have our ordinary system, your computer. Nowadays, you can even run a lot of the stuff on a cell phone because cell phones have advanced so much. And then with artificial intelligence and machine learning, it now takes the data and it learns from what happened before and then it predicts what's going to come next. And then really the biggest part right now in machine learning that's going on is it improves on that. How do we find a new solution? So we go from descriptive where it's learning about stuff and understanding how it fits together to predicting what it's going to do to postcripting coming up with a new solution. And when we're working on machine learning, there's a number of different diagrams that people have posted for what steps to go through. A lot of it might be very domain specific. So if you're working on photo identification versus language versus medical or physics, some of these are switched around a little bit or new things are put in. They're very specific to the domain. This is kind of a very general diagram. First, you want to define your objective. Very important to know what it is you're wanting to predict. Then you're going to be collecting the data. So once you've defined an objective, you need to collect the data that matches. You spend a lot of time in data science collecting data. And the next step preparing the data. You got to make sure that your data is clean going in. There's the old saying, bad data in, bad answer out or bad data out. And then once you've gone through and we've cleaned all this stuff coming in, then you're going to select the algorithm. Which algorithm are you going to use? You're going to train that algorithm. In this case, I think we're going to be working with SVM, the support vector machine. Then you have to test the model. Does this model work? Is this a valid model for what we're doing? And then once you've tested it, you want to run your prediction. You want to run your prediction or your choice or whatever output it's going to come up with. And then once everything is set and you've done lots of testing, then you want to go ahead and deploy the model. And remember, I said domain specific. This is very general as far as the scope of doing something. A lot of models you get halfway through and you realize that your data is missing something and you have to go collect new data because you've run a test in here someplace along the line. You're saying, "Hey, I'm not really getting the answers I need." So there's a lot of things that are domain specific that become part of this model. This is a very general model, but it's a very good model to start with. And we do have some basic divisions of what machine learning does that's important to know. For instance, do you want to predict a category? Well, if you're categorizing thing, that's classification. For instance, whether the stock price will increase or decrease. So in other words, I'm looking for a yes no answer. Is it going up or is it going down? And in that case, we'd actually say, is it going up? True. If it's not going up, it's false, meaning it's going down. This way, it's a yes no 01. Do you want to predict a quantity? That's regression. So remember, we just did classification. Now we're looking at regression. These are the two major divisions in what data is doing. For instance, predicting the age of a person based on the height, weight, health, and other factors. So based on these different factors, you might guess how old a person is. And then there are a lot of domain specific things like do you want to detect an anomaly? That's anomaly detection. This is actually very popular right now. For instance, you want to detect money withdrawal anomalies. You want to know when someone's making a withdrawal that might not be their own account. We've actually brought this up because this is really big right now. If you're predicting the stock, whether to buy stock or not, you want to be able to know if what's going on in the stock market is an anomaly, use a different prediction model because something else is going on. You got to pull out new information in there. Or is this just the norm? I'm going to get my normal return on my money invested. So being able to detect anomalies is very big in data science these days. Another question that comes up which is on what we call untrained data is do you want to discover structure in unexplored data and that's called clustering. For instance, finding groups of customers with similar behavior given a large database of customer data containing their demographics and past buying records. And in this case, we might notice that anybody who's wearing certain set of shoes goes shopping at certain stores or whatever it is, they're going to make certain purchases. By having that information, it helps us to market or group people together. So then we can now explore that group and find out what it is we want to market to them if you're in the marketing world. And that might also work in just about any arena. You might want to group people together whether they're uh based on their different areas and investments and financial background, whether you're going to give them a loan or not. before you even start looking at whether they're a valid customer for the bank, you might want to look at all these different areas and group them together based on unknown data. So, you're not you don't know what the data is going to tell you, but you want to cluster people together that come together. Let's take a quick detour for quiz time. Oh, my favorite. So, we're going to have a couple questions here under quiz time and um we'll be posting the answers in the part two of this tutorial. So, let's go ahead and take a look at these quiz times questions and hopefully you'll get them all right and it'll get you thinking about how to process data and what's going on. Can you tell what's happening in the following cases? Of course, you're sitting there with your cup of coffee and you have your checkbox and your pen trying to figure out what's your next step in your data science analysis. So, the first one is grouping documents into different categories based on the topic and content of each document. Very big these days. you know, you have legal documents, you have uh maybe it's a sports group documents, maybe you're analyzing newspaper postings, but certainly having that automated is a huge thing in today's world. B, identifying handwritten digits in images correctly. So, we want to know whether uh they're writing an A or capital A, B, C, what are they writing out in their hand digit that they're handwriting. C behavior of a website indicating that the site is not working as designed. D predicting salary of an individual based on his or her years of experience with HR hiring uh setup there. So stay tuned for part two. We'll go ahead and answer these questions when we get to the part two of this tutorial or you can just simply write at the bottom and send a note to SimplyLearn and they'll follow up with you on it. Back to our regular content. And these last few bring us into the next topic which is another way of dividing our types of machine learning and that is with supervised unsupervised and reinforcement learning. Supervised learning is a method used to enable machines to classify predict objects, problems or situations based on labeled data fed to the machine. And in here you see we have a jumble of data with circles, triangles and squares. And we label them. We have what's a circle, what's a triangle, what's a square. And we have our model training and it trains it. So we know the answer. Very important when you're doing supervised learning, you already know the answer to a lot of your information coming in. So you have a huge group of data coming in and then you have new data coming in. So we've trained our model. The model now knows the difference between a circle, a square, a triangle. And now that we've trained it, we can send in in this case a square and a circle goes in and it predicts that the top one's a square and the next one's a circle. And you can see that this is uh being able to predict whether someone's going to default on a loan because I was talking about banks earlier. Supervised learning on stock market whether you're going to make money or not. That's always important. And if you are looking to make a fortune in the stock market, keep in mind it is very difficult to get all the data correct on the stock market. It is very uh it fluctuates in ways you really hard to predict. So it's quite a roller coaster ride. If you're running machine learning on the stock market, you start realizing you really have to dig for new data. So we have supervised learning. And if you have supervised, we need unsupervised learning. In unsupervised learning, machine learning model finds the hidden pattern in an unlabeled data. So in this case, instead of telling it what the circle is and what a triangle is and what a square is, it goes in there, looks at them, and says for whatever reason, it groups them together. Maybe it'll group it by the number of corners, and it notices that a number of them all have three corners, a number of them all have four corners, and a number of them all have no corners. And it's able to filter those through and group them together. We talked about that earlier with looking at a group of people who are out shopping. We want to group them together to find out what they have in common. And of course, once you understand what people have in common, maybe you have one of them who's a customer at your store, or you have five of them are customer at your store, and they have a lot in common with five others who are not customers at your store. How do you market to those five who aren't customers at your store yet? They fit the demographs of who's going to shop there, and you'd like them to shop at your store, not the one next door. Of course, this is a simplified version. You can see very easily the difference between a triangle and a circle, which is might not be so easy in marketing. Reinforcement learning. Reinforcement learning is an important type of machine learning where an agent learns how to behave in an environment by performing actions and seeing the result. And we have here where the in this case a baby. It's actually great that they used an infant for this slide because the reinforcement learning is very much in its infant stages. But it's also probably the biggest machine learning demand out there right now or in the future. It's going to be coming up over the next few years is reinforcement learning and how to make that work for us. And you can see here where we have our action. In the action in this one, it goes into the fire. Hopefully, the baby didn't was just a little candle, not a giant fire pit like it looks like here. When the baby comes out and the new state is the baby is sad and crying because they got burned on the fire. And then maybe they take another action. The baby's called the agent because it's the one taking the actions. And in this case, they didn't go into the fire. They went a different direction. And now the baby's happy and laughing and playing. Reinforcement learning is very easy to understand because that's how as humans that's one of the ways we learn. We learn whether it is you burn yourself on the stove, don't do that anymore. Don't touch the stove. In the big picture, being able to have machine learning program or an AI be able to do this is huge because now we're starting to learn how to learn. That's a big jump in the world of computer and machine learning. And we're going to go back and just kind of go back over supervised versus unsupervised learning. Understanding this is huge because this is going to come up in any project you're working on. We have in supervised learning, we have labeled data. We have direct feedback. So someone's already gone in there and said, "Yes, that's a triangle. No, that's not a triangle." And then you predict an outcome. So you have a nice prediction. This is this this new set of data is coming in and we know what it's going to be. And then with unsupervised training, it's not labeled. So we really don't know what it is. There's no feedback. So, we're not telling it whether it's right or wrong. We're not telling it whether it's a triangle or a square. We're not telling it to go left or right. All we do is we're finding hidden structure in the data, grouping the data together to find out what connects to each other. And then you can use these together. So, imagine you have an image and you're not sure what you're looking for. So, you go in and you have the unstructured data. Find all these things that are connected together and then somebody looks at those and labels them. Now you can take that label data and program something to predict what's in the picture. So you can see how they go back and forth and you can start connecting all these different tools together to make a bigger picture. There are many interesting machine learning algorithms. Let's have a look at a few of them. Hopefully this gave you a little flavor of what's out there and these are some of the most important ones that are currently being used. We'll take a look at linear regression, decision tree and the support vector machine. Let's start with a closer look at linear regression. Linear regression is perhaps one of the most well-known and well understood algorithms in statistics and machine learning. Linear regression is a linear model. For example, a model that assumes a linear relationship between the input variables x and the single output variable y. And you'll see this if you remember from your algebra classes. Y = mx + c. Imagine we are predicting distance traveled y from speed x. Our linear regression model representation for this problem would be y = m * x + c or distance= m * speed + c where m is the coefficient and c is the y intercept. And we're going to look at two different variations of this. First, we're going to start with time is constant. And you can see we have a bicyclist. He's got a safety gear on, thank goodness. Speed equals 10 meters/s. And so over a certain amount of time, his distance equals 36 kilometers. We have a second bicyclist who's going twice the speed or 20 m/s. And you can guess if he's going twice the speed and time is a constant, then he's going to go twice the distance. And that's easy to compute. 36 * 2, you get 72 km. And so if you had the question of how fast somebody is going three times that speed or 30 m/s is, you can easily compute the distance in our head. We can do that without needing a computer. But we want to do this for more complicated data. So, it's kind of nice to compare the two, but let's just take a look at that and what that looks like in a graph. So, in a linear regression model, we have our distance to the speed and we have our m equals the ve slope of the line. And we'll notice that the line has a plus slope. And as speed increases, distance also increases. Hence, the variables have a positive relationship. And so, your speed of the person, which equals y= mx plus c, distance traveled in a fixed interval of time. And we could very easily compute either following the line or just knowing it's three times 10 m/s that this is roughly 102 km distance that this third bicus has traveled. One of the key definitions on here is positive relationship. So the slope of the line is positive. As distance increase, so does speed increase. Let's take a look at our second example where we put distance is a constant. So we have speed equals 10 m/s. They have a certain distance to go and it takes him a 100 seconds to travel that distance. And we have our second bicyclist who's still doing 20 meters per second. Since he's going twice the speed, we can guess he'll cover the distance in about half the time, 50 seconds. And of course, you could probably guess on the third one 100 / 30 since he's going three times the speed. You can easily guess that this is 33.33 seconds time. We put that into a linear regression model or a graph. If the distance is assumed to be constant, let's see the relationship between speed and time. And as time goes up, the amount of speed to go that same distance goes down. So now your m equals a minus v slope of the line. As the speed increases, time decreases. Hence, the variable has a negative relationship. Again, there's our definition. Positive relationship and negative relationship dependent on the slope of the line. And with a simple formula like this um and even a significant amount of data let's uh see with the mathematical implementation of linear regression and we'll take this data. So suppose we have this data set where we have xy x= 1 2 3 4 5 standard series and the y value is 3 22 43. When we take that and we go ahead and plot these points on a graph, you can see there's kind of a nice scattering and you could probably eyeball a line through the middle of it. But we're going to calculate that exact line for linear regression. And the first thing we do is we come up here and we have the mean of Xi. And remember mean is basically the average. So we added 5 + 4 plus 3 plus 2 plus 1 and divide by five. And that simply comes out as three. And then we'll do the same for y. We'll go ahead and add up all those numbers and divide by five. And we end up with a mean value of y of i equals 2.8 where the x i references it's an average or means value and the yi also equals a means value of y. And when we plot that, you'll see that we can put in the y= 2.8 and the x= 3 in there on our graph. We kind of gave it a little different color so you could sort it out with the dashed lines on it. And it's important to note that when we do the linear regression, the linear regression model should go through that dot. Now, let's find our regression equation to find the best fit line. Remember, we go ahead and take our y= mx plus c. So, we're looking for m and c. So, to find this equation for our data, we need to find our slope of m and our coefficient of c. And we have y = mx + c where m equals the sum of x - x average * y - y average or y means and x means over the sum of x - x means squared. That's how we get the slope of the value of the line. And we can easily do that by creating some columns here. We have xy. Computers are really good about iterating through data. And so we can easily compute this and fill in a graph of data. And in our graph you can easily see that if we have our x value of one and if you remember the x i or the means value is three 1 minus 3 equals a -2 and 2 - 3 = a -1 so on and so forth and we can easily fill in the column of x - x i y - yi and then from those we can compute x - x i^ 2 and x - x i * y - yi and you can guess it that the next step is to go ahead and sum the different columns for the answers we need. So we get a total of 10 for our x - x i^ 2 and a total of 2 for x - x i * y - yi and we plug those in. We get 210 which equals2. So now we know the slope of our line equals2. So we can calculate the value of c. That'd be the next step is we need to know where it crosses the y ais. And if you remember, I mentioned earlier that the linear regression line has to pass through the means value, the one that we showed earlier. We can just flip back up there to that graph. And you can see right here, there's our means value, which is 3, x= 3, and y= 2.8. And since we know that value, we can simply plug that into our formula. y =2x + c. So we plug that in, we get 2.8 8 =2 * 3 + c. And you can just solve for c. So now we know that our coefficient equals 2.2. And once we have all that, we can go ahead and plot our regression line. y =2 * x + 2.2. And then from this equation, we can compute new values. So let's predict the values of y using x= 1 2 3 4 5 and plot the points. Remember the 1 2 3 4 5 was our original x values. So now we're going to see what Y thinks they are, not what they actually are. And we plug those in, we get Y of designated with Y of P. You can see that X= 1 = 2.4, X= 2= 2.6, and so on and so on. So we have our Y predicted values of what we think it's going to be when we plug those numbers in. And when we plot the predicted values along with the actual values, we can see the difference. And this is one of the things that's very important with linear regression in any of these models is to understand the error. And so we can calculate the error on all of our different values. And you can see over here we plotted um x and y and y predict. And we draw a little line so you can sort of see what the error looks like there between the different points. So our goal is to reduce this error. We want to minimize that error value on our linear regression model. Minimizing the distance. There are lots of ways to minimize the distance between the line and the data points like sum of squared errors, sum of absolute errors, root mean square error, etc. We keep moving this line through the data points to make sure the best fit line has the least square distance between the data points and the regression line. So to recap with a very simple linear regression model, we first figure out the formula of our line through the middle and then we slowly adjust the line to minimize the error. Keep in mind this is a very simple formula. The math gets even though the math is very much the same, it gets much more complex as we add in different dimensions. So this is only two dimensions. Y equals MX plus C. But you can take that out to X, Z, Y, J, Q, all the different features in there and they can plot a linear regression model on all of those using the different formulas to minimize the error. Let's go ahead and take a look at decision trees. A very different way to solve problems than the linear regression model. Decision tree is a treeshaped algorithm used to determine a course of action. Each branch of a tree represents a possible decision, occurrence, or reaction. We have data which tells us if it is a good day to play golf. And if we were to open this data up in a general spreadsheet, you can see we have the outlook whether it's rainy, overcast, sunny, temperature, hot, mild, cool, humidity, windy, and did I like to play golf that day? Yes or no? So, we're taking a census and certainly I wouldn't want a computer telling me when I should go play golf or not. But you could imagine if you got up in the night before you're trying to plan your day and it comes up and says tomorrow would be a good day for golf for you in the morning and not a good day in the afternoon or something like that. This becomes very beneficial and we see this in a lot of applications coming out now where it gives you suggestions and lets you know what what would uh fit the match for you for the next day or the next purchase or the next uh whatever you know next mail out in this case is tomorrow a good day for playing golf based on the weather coming in. And so we come up and let's uh determine if you should play golf when the day is sunny and windy. So we found out the forecast tomorrow is going to be sunny and windy. And suppose we draw our tree like this. We're going to have our humidity. And then we have our normal, which is if it's if you have a normal humidity, you're going to go play golf. And if the humidity is really high, then we look at the outlook. And if the outlook is sunny, overcast, or rainy, it's going to change what you choose to do. So if you know that it's a very high humidity and it's sunny, you're probably not going to play golf cuz you're going to be out there miserable, fighting off the mosquitoes that are out joining you to play golf with you. Maybe if it's rainy, you probably don't want to play in the rain. But if it's slightly overcast and you get just the right shadow, that's a good day to play golf and be outside out on the green. Now, in this example, you can probably make your own tree pretty easily. So, it's a very simple set of data going in. But the question is, how do you know what to split? Where do you split your data? What if this is much more complicated data where it's not something that you would particularly understand like studying cancer? They take about 36 measurements of the cancerous cells and then each one of those measurements represents how bulbous it is, how extended it is, how sharp the edges are, something that as a human we would have no understanding of. So how do we decide how to split that data up and is that the right decision tree? But so that's a question that's going to come up. Is this the right decision tree? For that we should calculate entropy and information gain. Two important vocabulary words there are the entropy and the information gain. Entropy. Entropy is a measure of randomness or impurity in the data set. Entropy should be low. So we want the chaos to be as low as possible. We don't want to look at it and be confused by the images or what's going on there with mixed data. And the information gain, it is a measure of decrease in entropy after the data set is split. Also known as entropy reduction. information gain should be high. So we want our information that we get out of the split to be as high as possible. Let's take a look at entropy from the mathematical side. In this case, we're going to denote entropy as I of P of and N where P is the probability that you're going to play a game of golf and N is the probability where you're not going to play the game of golf. Now, you don't really have to memorize these formulas. There's a few of them out there depending on what you're working with. But it's important to note that this is where this formula is coming from. So when you see it, you're not lost when you're running your programming. Unless you're building your own decision tree code in the back. And we simply have a log squar of p over p plus n minus n over p plus n * the log squar of n of p + n. But let's break that down and see what actually looks like when we're computing that from the computer script side. Entropy of a target class of the data set is the whole entropy. So we have entropy play golf and we look at this if we go back to the data you can simply count how many yeses and no in our complete data set for playing golf days. In our complete set we find we have five days we did play golf and 9 days we did not play golf. And so our I equals if you add those together 9 + 5 is 14. And so our I equals 5 over 14 and 9 over4. That's our PNN values that we plug into that formula. And you can go with the 5 over 14=.36 9 over 14=64. And when you do the whole equation you get the minus.36 log roo<unk>^2 of.36 -64 log<unk> of 64 and we get a set value. We get 94. So we now have a full entropy value for the whole set of data that we're working with. And we want to make that entropy go down. And just like we calculated the entropy out for the whole set, we can also calculate entropy for playing golf and the outlook. Is it going to be overcast or rainy or sunny? And [snorts] so we look at the entropy. We have P of sunny times E of three of two. And that just comes out how many sunny days yes and how many sunny days no over the total which is five. Don't forget to put the we'll divide that five out later on. uh equals p overcast = 4 comma 0 plus rainy = 2a 3 and then when you do the whole setup we have 5 over4 remember I said there was a total of five 5 over 14 * the i of 3 of 2 + 4 over 14 * the 4 0 and 54 over i of 23 and so we can now compute the entropy of just the part that has to do with the forecast and we get 693 similar Similarly, we can calculate the entropy of other predictors like temperature, humidity and wind. And so we look at the gain outlook. How much are we going to gain from this entropy play golf minus entropy play golf outlook? And we can take the original 0.94 for the whole set minus the entropy of just the rainy day and temperature and we end up with a gain of.247. So this is our information gain. Remember we define entropy and we define information gain. The higher the information gain, the lower the entropy, the better. The information gain of the other three attributes can be calculated in the same way. So we have our gain for temperature equals 0.029. We have our gain for humidity equals.152. And our gain for a windy day equals 0048. And if you do a quick comparison, you'll see the.247 is the greatest gain of information. So that's the split we want. Now let's build the decision tree. So, we have the outlook. Is it going to be sunny, overcast, or rainy? That's our first split because that gives us the most information gain. And we can continue to go down the tree using the different information gains with the largest information. We can continue down the nodes of the tree where we choose the attribute with the largest information gain as the root node and then continue to split each subnode with the largest information gain that we can compute. And although it's a little bit of a tongue twister to say all that, you can see that it's a very easy to view visual model. We have our outlook. We split it three different directions. If the outlook is overcast, we're going to play. And then we can split those further down if we want. So if the over outlook is sunny, but then it's also windy. If it's uh windy, we're not going to play. If it's uh not windy, we'll play. So, we can easily build a nice decision tree to guess what we would like to do tomorrow and give us a nice recommendation for the day. So, we want to know if it's a good day to play golf when it's sunny and windy. Remember the original question that came out, tomorrow's weather report is sunny and windy. You can see by going down the tree, we go outlook sunny, outlook windy. We're not going to play golf tomorrow. So, our little smartwatch pops up and says, I'm sorry, tomorrow's not a good day for golf. It's going to be sunny and windy. And if you're a huge golf fan, you might go, "Uhoh, it's not a good day to play golf." We can go in and watch a golf game at home. So, we'll sit in front of the TV instead of being out playing golf in the wind. Now that we looked at our decision tree, let's look at the third one of our algorithms we're investigating. Support vector machine. Support vector machine is a widely used classification algorithm. The idea of support vector machine is simple. The algorithm creates a separation line which divides the classes in the best possible manner. For example, dog or cat, disease or no disease. Suppose we have a labeled sample data which tells height and weight of males and females. A new data point arrives and we want to know whether it's going to be a male or a female. So we start by drawing a line. We draw decision lines. But if we consider decision line one, then we will classify the individual as a male. And if we consider decision line two, then it'll be a female. So you can see this person kind of lies in the middle of the two groups. So it's a little confusing trying to figure out which line they should be under. We need to know which line divides the classes correctly. But how the goal is to choose a hyper plane and that is one of the key words they use when we talk about support vector machines. Choose a hyper plane with the greatest possible margin between the decision line and the nearest point within the training set. So you can see here we have our support vector. we have the two nearest points to it and we draw a line between those two points and the distance margin is the distance between the hyperplane and the nearest data point from either set. So we actually have a value and it should be equal distant between the two points that we're comparing it to. When we draw the hyperplanes we observe that line one has a maximum distance. So we observe that line one has a maximum distance margin. So we'll classify the new data point correctly. And our result on this one is going to be that the new data point is MEL. One of the reasons we call it a hyper plane versus a line is that a lot of times we're not looking at just weight and height. We might be looking at 36 different features or dimensions. And so when we cut it with a hyper plane, it's more of a three-dimensional cut in the data. Multi-dimensional that cuts the data a certain way. and each plane continues to cut it down until we get the best fit or match. Let's understand this with the help of an example problem statement. I always start with a problem statement when you're going to put some code together. We're going to do some coding now. Classifying muffin and cupcake recipes using support vector machines. So, the cupcake versus the muffin. Let's have a look at our data set. And we have the different recipes here. We have a muffin recipe that has so much flour. I'm not sure what measurement 55 is in, but it has 55, maybe it's ounces, but it has a certain amount of flour, certain amount of milk, sugar, butter, egg, baking powder, vanilla, and salt. And so based on these measurements, we want to guess whether we're making a muffin or a cupcake. And you can see in this one, we don't have just two features. We don't just have height and weight as we did before between the male and female. In here, we have a number of features. In fact, in this, we're looking at eight different features to guess whether it's a muffin or a cupcake. What's the difference between a muffin and a cupcake? Turns out muffins have more flour, while cupcakes have more butter and sugar. [snorts] So, basically, the cupcakes a little bit more of a dessert, where the muffin's a little bit more of a fancy bread. But how do we do that in Python? How do we code that to go through recipes and figure out what the recipe is? And I really just want to say cupcakes versus muffins like some big professional wrestling thing. Before we start in our cupcakes versus muffins, we are going to be working in Python. There's many versions of Python, many different editors. That is one of the strengths and weaknesses of Python is it just has so much stuff attached to it. It's one of the more popular data science programming packages you can use. In this case, we're going to go ahead and use Anaconda in Jupyter Notebook. The Anaconda Navigator has all kinds of fun tools. Once you're into the Anaconda Navigator, you can change environments. I actually have a number of environments on here. We'll be using Python 36 environment. So, this is in Python version 36. Although, it doesn't matter too much which version you use. I usually try to stay with the 3X because they're current unless you have a project that's very specifically in version 2X. 2.7 I think is usually what most people use in the version two. And then once we're in our um Jupyter notebook editor, I can go up and create a new file and we'll just jump in here. In this case, we're doing SPM muffin versus cupcake. And then let's start with our packages for data analysis. And we almost always use a couple there's a few very standard packages we use. We use import oops import numpy that's for number python. They usually denote it as np that's very comma that's very common. And then we're going to import pandas as pd. And numpy deals with number arrays. There's a lot of cool things you can do with the numpy uh setup as far as multiplying all the values in an array in a numpy array. Data array pandas. I can't remember if we're using it actually in this data set. I think we do as an import. It makes a nice data frame. And the difference between a data frame and a numpy array is that a data frame is more like your Excel spreadsheet. You have columns, you have indexes. So you have different ways of referencing it, easily viewing it, and there's additional features you can run on a data frame. And pandas kind of sits on numpy. So they you need them both in there. And then finally, we're working with the support vector machine. So from sklearn, we're going to use the sklearn model. Import SVM, support vector machine. And then as a data scientist, you should always try to visualize your data. Some data obviously is too complicated or doesn't make any sense to the human. But if it's possible, it's good to take a second look at it so that you can actually see what you're doing. Now, for that, we're going to use two packages. We're going to import mapplot library.pipplot as plt. Again, very common. And we're going to import seabor as sns. And we'll go ahead and set the font scale in the SNS. Right in our import line, that's what this U semicolon followed by a line of data. We're going to set the SNS. And these are great because the the seaborn sits on top of map plot library just like pandas sits on numpy. So it adds a lot more features and uses and control. We're obviously not going to get into mattplot library and seabour. It'd be its own tutorial. We're really just focusing on the SVM, the support vector machine from sklearn. And since we're in Jupiter notebook, uh we have to add a special line in here for our mattplot library. And that's your percentage sign or amber sign mattplot library in line. Now, if you're doing this in just a straight code project, a lot of times I use like Notepad++ and I'll run it from there. You don't have to have that line in there because it'll just pop up as its own window on your computer depending on how your computer's set up because we're running this in the Jupyter notebook as a browser setup. This tells it to display all of our graphics right below on the page. So that's what that line is for. I remember the first time I ran this, I didn't know that and I had to go look that up years ago. It's quite a headache. So mapplot library inline is just because we're running this on the web setup and we can go ahead and run this. Just make sure all our modules are in. They're all imported, which is great. If you don't have them, import, you'll need to go ahead and pip. Use the pip or however you do it. There's a lot of other install packages out there, although pip is the most common. And you have to make sure these are all installed on your Python setup. The next step, of course, is we got to look at the data. You can't run a model for predicting data if you don't have actual data. So, to do that, let me go ahead and open this up and take a look. And we have our uh cupcakes versus muffins. And it's a CSV file or CSV meaning that it's commaepparated variable. And it's going to open it up in a nice uh spreadsheet for me. And you can see up here we have the type. We have muffin, muffin, muffin, cupcake, cupcake, cupcake. And then it's broken up into flour, milk, sugar, butter, egg, baking powder, vanilla, and salt. So we can do is we can go ahead and look at this data also in our Python. Let us create a variable recipes equals we're going to use our pandas module read CSV remember is a commaepparated variable and the file name happened to be cupcakes versus muffins. Oops, I got double brackets there. Do it this way. There we go. cupcakes versus muffins. Because the program I loaded or the the place I saved this particular Python program is in the same folder, we can get by with just the file name. But remember, if you're storing it in a different location, you have to also put down the full path on there. And then because we're in pandas, we're going to go ahead and you can actually in line you can do this, but let me do the full print. And you can just type in recipes.head head in the Jupyter notebook. But if you're running in code in a different script, you'd need to go ahead and type out the whole print recipes. And Pandanda's nose is that's going to do the first five lines of data. And if we flip back on over to the spreadsheet where we opened up our CSV file, uh you can see where it starts on line two. This one calls it zero. And then 2 3 4 5 6 is going to match. Go and close that out because we don't need that anymore. And it always starts at zero. And these are it automatically indexes it since we didn't tell it to use an index in here. So that's the index number for the left hand side. And it automatically took the top row as labels. So pandas using it to read a CSV is just really slick and fast. One of the reasons we love our pandas, not just because they're cute and cuddly teddy bears. And let's go ahead and plot our data. And I'm not going to plot all of it. I'm just going to plot the uh sugar and flour. Now, obviously, you can see where they get really complicated if we have tons of different features. And so, you'll break them up and maybe look at just two of them at a time to see how they connect. And to plot them, we're going to go ahead and use Seabor. So, that's our SNS. And the command for that is snplot. And then the two different variables I'm going to plot is flour and sugar. Data equals recipes. The hue equals type. And this is a lot of fun because it knows that this is pandas coming in. So this is one of the powerful things about pandas mixed with seabour and doing graphing. And then we're going to use a pallet set one. There's a lot of different sets in there. You can go look them up for seabour. We'll do a regular fit regular equals false. So, we're not really trying to fit anything. And it's a scatter KWS. A lot of these settings you can look up in Seabor. Half of these you could probably leave off when you run them. Somebody played with this and found out that these were the best settings for doing a Seabor plot. And let's go ahead and run that. And because it does it in line, it just puts it right on the page. And you can see right here that just based on sugar and flour alone, there's a definite split. And we use these models because you can actually look at it and say, "Hey, if I drew a line right between the middle of the blue dots and the red dots, we'd be able to do an SVM and and a hyper plane right there in the middle. Then the next step is to format or pre-process our data. And we're going to break that up into two parts. We need a type label. And remember, we're going to decide whether it's a muffin or a cupcake. Well, a computer doesn't know muffin or cupcake. It knows zero and one. So, what we're going to do is we're going to create a type label. And from this we'll create a numpy array np where and this is where we can do some logic. We take our recipes from our panda and wherever type equals muffin it's going to be zero. And then if it doesn't equal muffin which is cupcakes it's going to be one. So we create our type label. This is the answer. So when we're doing our training model remember we have to have a a training data. This is what we're going to train it with is that it's zero or one. it's a muffin or it's not. And then we're going to create our recipe features. And if you remember correctly from right up here, the first column is type. So we really don't need the type column because that's our muffin or cupcake. And in pandas, we can easily sort that out. We take our value recipes. That's a pandas function built into pandas. values converting them to values. So it's just the column titles going across the top and we don't want the first one. So what we do is since it always starts at zero, we want one colon till the end. And then we want to go ahead and make this a list. And this converts it to a list of strings. And then we can go ahead and just take a look and see what we're looking at for the features. Make sure it looks right. Me go ahead and run that. And I forgot the S on recipes. So, we'll go ahead and add the S in there and then run that. And we can see we have flour, milk, sugar, butter, egg, baking powder, vanilla, and salt. And that matches what we have up here, right? We printed out everything but the type. So, we have our features and we have our label. Now, the recipe features is just the titles of the columns. But we actually need the ingredients. And at this point, we have a couple options. One, we could run it over all the ingredients. And when you're dealing this, usually you do, but for our example, we want to limit it so you can easily see what's going on because if we did all the ingredients, we have, you know, that's what, um, seven, eight different hyperplanes that would be built into it. We only want to look at one so you can see what the SVM is doing. And so we'll take our recipes and we'll do just flour and sugar. Again, you can replace that with your recipe features and do all of them, but we're going to do just flour and sugar. And we're going to convert that to values. We don't need to make a list out of it because it's not string values. These are actual values on there. And we can go ahead and just print ingredients. And you can see what that looks like. Uh, and so we have just the nan of flour and sugar, just the two sets of plots. And just for fun, let's go ahead and take this over here and take our recipe features. And so if we decided to use all the recipe features, you'll see that it makes a nice column of different data. So it just strips out all the labels and everything. We just have just the values. But because we want to be able to view this easily in a plot later on, we'll go ahead and take that and just do flour and sugar. And we'll run that. And you'll see it's just the two columns. So the next step is to go ahead and fit our model. We'll go ahead and just call it model. And it's a SVM. We're using a package called SVC. In this case, we're going to go ahead and set the kernel [clears throat] equals linear. So, it's using a specific setup on there. And if we go to the reference on their website for the SVM, you'll see that there's about there's eight of them here. Three of them are for regression. Three are for classification. The SVC, support vector classification, is probably one of the most commonly used. And then there's also one for detecting outliers and another one that has to do with something a little bit more specific on the model. But SVC and SVR are the two most commonly used standing for support vector classifier and support vector regression. Remember regression is an actual value, a float value or whatever you're trying to work on. And SBC is a classifier. So it's a yes, no, true, false. But for this we want to know 01 muffin cupcake. We go ahead and create our model. And once we have our model created, we're going to do model.fit. And this is very common, especially in the sklearn. All their models are followed with the fit command. And what we put into the fit, what we're training with it is we're putting in the ingredients, which in this case we limited to just flour and sugar, and the type label. Is it a muffin or cupcake? Now, in more complicated data science series, you'd want to split into, we won't get into that today, where you split it into training data and test data. And they even do something where they split it into thirds where a third is used for where you switch between which one's training and test. There's all kinds of things go into that. It gets very complicated when you get to the higher end. Not overly complicated, just an extra step, which we're not going to do today because this is a very simple set of data. And let's go ahead and run this. And now we have our model fit. And uh I got an error here. So let me fix that real quick. It's capital SPC. It turns out I did it lowercase. Support vector classifier. There we go. Let's go ahead and run that. And you'll see it comes up with all this information that it prints out automatically. These are the defaults of the model. You notice that we changed the kernel to linear. And there's our kernel linear on the printout. And there's other different settings you can mess with. We're going to just leave that alone for right now. For this, we don't really need to mess with any of those. So, next we're going to dig a little bit into our newly trained model. And we're going to do this so we can show you on a graph. And let's go ahead and get the separating. and we're going to say uh we're going to use a w for our variable on here and we're going to do model.coreefficient_0. So what the heck is that? Again, we're digging into the model. So we've already got a prediction and a train. This is a math behind it that we're looking at right now. And so the w is going to represent two different coefficients. And if you remember, we had y = mx + c. So these coefficients are connected to that but in two-dimensional it's a plane. We don't want to spend too much time on this because you can get lost in the confusion of the math. So if you're a math wiz this is great. You can go through here and you'll see that we have a equals minus w of 0 over w of 1. Remember there's two different values there. And that's basically the slope that we're generating. And then we're going to build an xx. What is xx? We're going to set it up to a numpy array. There's our np line space. So we're creating a line of values between 30 and 60. So it just creates a set of numbers for x. And then if you remember correctly, we have our formula y equ= the slope * x plus the intercept. Well, to make this work, we can do this as y equals the slope times each value in that array. That's the neat thing about numpy. So, when I do a * xx, which is a whole numpy array of values, it multiplies a across all of them. And then it takes those same values and we subtract the model intercept. That's your uh we had mx plus c. So, that'd be the c from the formula yals mx plus c. And that's where all these numbers come from. A little bit confusing because it's digging out of these different arrays. And then we want to do is we're going to take this and we're going to go ahead and plot it. So plot the parallels to separating hyper plane that pass through the support vectors. And so we're going to create B equals a model support vectors. Pulling our support vectors out there. Here's our y, which we now know is a set of data. And we have uh we're going to create y down = a * xx + b1 - a * b 0. And then model support vector b is going to be set that to a new value. The minus1 setup and y y up = a * xx + b1 - a * b 0. And we can go ahead and just run this to load these variables up. If you wanted to know understand a little bit more of what's going on, you can see if we print y, let me just run that. You can see it's an array. This is a line. It's going to have in this case between 30 and 60. So there's going to be 30 variables in here. And the same thing with y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y up, y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y y down and we'll we'll plot those in just a minute on a graph so you can see what those look like. Just go ahead and delete that out of here and run that. So, it loads up the variables. Nice clean slate. I'm just going to copy this from before. Remember this? Our SNS, our Seabour plot, LM plot, flower, sugar. And I'll just go and run that real quick so you can see what remember what that looks like. It's just a straight graph on there. And then one of the neat things is because Seabor sits on top of pipplot, we can do the pipplot for the line going through. And that is simply plt.plot And that's our xx and y are two corresponding values xy. And then somebody played with this to figure out that the line width equals 2 and the color black would look nice. So let's go ahead and run this whole thing with the pie plot on there. And you can see when we do this, it's just doing flour and sugar on here. Corresponding line between the sugar and the flour and the muffin versus cupcake. Um, and then we generated the U support vectors, the y down and y up. So let's take a look and see what that looks like. So we'll do our plot. And again, this is all against xx, our x value, but this time we have y down. And let's do something a little fun with this. We can put in a k dash dash. That just tells it to make it a dotted line. And if we're going to do the down one, we also want to do the up one. So here's our y up. And when we run that, it adds both sets of line. And so here's our support. And this is what you expect. You expect these two lines to go through the nearest data point. So the dash lines go through the nearest muffin and the nearest cupcake when it's plotting it. And then your SVM goes right down the middle. So it gives it a nice split in our data. And you can see how easy it is to see based just on sugar and flour which one's a muffin or a cupcake. Let's go ahead and create a function to predict muffin or cupcake. I've got my um recipes I pulled off the um internet and I want to see the difference between a muffin or a cupcake. And so we need a function to push that through. And u create a function with def. And let's call it muffin or cupcake. And remember we're just doing flour and sugar today. We're not doing all the ingredients. And that actually is a pretty good split. You really don't need all the ingredients to know it's flour and sugar. And let's go ahead and do an if else statement. So if model predict is of flour and sugar equals zero. So we take our model and we do run a predict. It's very common in sklearn where you have a predict. You put the data in and it's going to return a value. In this case if it equals zero then print you're looking at a muffin recipe. Else if it's not zero that means it's one then you're looking at a cupcake recipe. That's pretty straightforward for function or def for definition. def is how you do that in Python. And of course, if we're going to create a function, you should run something in it. And so, let's run a cupcake. And we're going to send it values 50 and 20. A muffin or a cupcake. I don't know what it is. And let's run this and just see what it gives us. It says, "Oh, it's a muffin. You're looking at a muffin recipe." So, it very easily predicts whether we're looking at a muffin or a cupcake recipe. Let's plot this. There we go. plot this on the graph so we can see what that actually looks like. And I'm just going to copy and paste it from below where we're plotting all the points in there. So this is nothing different than we did before. If I run it, you'll see it has all the points and the lines on there. And what we want to do is we want to add another point and we'll do pltot. And if you remember correctly, we did for our test we did 50 and 20. And then somebody went in here and decided we'll do YO for yellow or it's kind of a orangeish yellow color is going to come up. Marker size nine. Those are settings you can play with. Somebody else played with them to come up with the right setup so it looks good. And you can see there it is graphed. Clearly a muffin. In this case in cupcakes versus muffins, the muffin has won. And if you'd like to do your own muffin cupcake contender series, you certainly can send a note down below and the team at SimplyLearn will send you over the data they used for the muffin and cupcake. And that's true of any of the data. We didn't actually run a plot on it earlier. We had men versus women. You can also request that information to run it on your data setup. So you can test that out. So to go back over our setup, we went ahead for our support vector machine code. We did a predict 40 parts flour, 20 parts sugar. I think it was different than the one we did whether it's a muffin or a cupcake. Hence, we have built a classifier using SPM which is able to classify if a recipe is of a cupcake or a muffin. Which wraps up our cupcake versus muffin. So the key takeaways, what is machine learning? We discussed that with some of the different aspects of machine learning on there. We went into types of machine learning. If you memorize we have supervised, unsupervised and reinforcement learning. We discussed regression line or best fit and we did the building a decision tree and what the logic is behind that. And finally we did classification using SVM support vector machine and we did the code in there. My name is Richard Kersner with the SimplyLearn team. That's get certified, get ahead. We're going to cover mathematics for machine learning. So today's agenda is going to cover data and its types. Then we're going to dive into linear algebra and its concepts, calculus, statistics for machine learning, probability for machine learning, hands-on demos, and of course throwing in there in the middle is going to be your matrixes and a few other things to go along with all this. Data and its types. Data denotes the individual pieces of factual information collected from various sources. It is stored, processed and later used for analysis. And so we see here uh just a huge grouping of information, a lot of tech stuff, money, dollar signs, numbers uh and then you have your performing analytics to drive insights and hopefully you have a nice share your shareholders gathered at the meeting and you're able to explain it in something they can understand. So we talk about datas types of data we have in our types of data we have a qualitative categorical you think nominal or ordinal and then you have your quantitative or numerical which is discrete or continuous and let's look a little closer at those data type vocabulary always people's favorite is the vocabulary words okay not mine uh but let's dive into this what we mean by nominal nominal they are used to label various Let's uh label our variables without providing any measurable value. Uh country, gender, race, hair, color, etc. It's something that you either mark true or false. This is a label. It's on or off. Either they have a red hat on or they do not. Uh so a lot of times when you're thinking nominal data labels, uh think of it as a true false kind of setup. And we look at ordinal. This is categorical data with a set order or a scale to it. Uh and you can think of salary range is a great one. Uh movie ratings etc. You see here the salary range if you have 10,000 to 20,000 number of employees earning that rate is 150. 20,000 to 30,000 100 and so forth. Some of the terms you'll hear is bucket. Uh this is where you have 10 different buckets and you want to separate it into something that makes sense into those 10 buckets. And so when we start talking about ordinal, a lot of times when you get down to the brass bones, again, we're talking true false. Uh so if you're a member of the 10 to 20k range, uh so forth, those would each be either part of that group or you're not. But now we're talking about buckets and we want to count how many people are in that bucket. Quantitative numerical data uh falls into two classes, discrete or continuous. And so data with a final set of values which can be categorized class strength questions answered correctly and runs hit in cricket. A lot of times when you see this you can think integer uh and a very restricted integer i.e. you can only have 100 questions um on a test. So you can it's very discreet. I only have a 100 different values that it can attain. So think usually you're talking about integers but within a very small range. They don't have an open end or anything like that. Uh so discrete is very solid, simple to count, set number. Continuous on the other hand uh continuous data can take any numerical value within a range. So water pressure, weight of a person etc. Usually we start thinking about float values where they can get phenomenally small in their in what they're worth. And there's a whole series of values that falls right between discrete and continuous. Um you can think of the stock market. You have dollar amounts. It's still discreet, but it starts to get complicated enough when you have like, you know, jump in the stock market from $525.33 to $580.67. There's a lot of point values in there. It'd still be called discrete, but you start looking at it as almost continuous because it does have such a variance in it. Now uh we talk about n we did we went over nominal and ordinal uh almost true false charts and we looked at quantitative and numerical data which we're starting to get into numbers. Discrete you can usually a lot of times discreet will be put into it could be put into true false but usually it's not. Uh so we want to address this stuff and the first thing we want to look at is the very basic which is your algebra. So we're going to take a look at linear algebra. You can remember back when your uklidian geometry. Uh we have a line. Well, let's go through this. We have linear algebra is the domain of mathematics concerning linear equations and their representations in vector spaces and through matrices. I told you we're going to talk about matrices. Uh so a linear equation is simply um uh 2x + 4 y - 3 z = 10. Very linear. 10 x + 12.4 4 y = z. And now you can actually solve these two equations by combining them. Uh, and that's where we're talking about a linear equation. And the vectors we have a + b equals c. Now we're starting to look at a direction. And these values usually think of an xyz plot. Um, so each one is a direction. And the actual distance of like a triangle A is C. And then your matrix can describe all kinds of things. Um, I find matrixes uh confuse a lot of people, not because they're particularly difficult, but because of the magnitude and the different things they're used for. And a matrix is a chart or a um, you know, think of a spreadsheet, but you have your rows and your columns. And you'll see here we have a * b= c. Very important to know your counts. Uh, so depending on how the math is being done, what you're using it for, making sure you have the same rows and number of columns or a single number, there's all kinds of things that play in that that can make matrixes confusing. Uh, but really it has a lot more to do with what domain you're working in. Uh, are you adding in multiple polomials where you have like uh uh ax^2 plus b y plus, you know, you start to see that can be very confusing versus a very straightforward matrix. And let's just go a little deeper into these because these are such primary this is what we're here to talk about is these different math uh mathematical computations that come up. So we're looking at linear equations. Let's dig deeper into that one. An equation having a maximum order of one is called a linear equation. Uh so it's linear because when you look at this we have uh ax plus b= c which is a one variable. We have two variable ax plus b y= c ax plus b y + z c cz z = d and so forth. But all of these are to the power of one. You don't see x squ. You don't see x cubed. So we're talking about linear equations. That's what we're talking about. And their addition. If you have already dived into say neural networks, you should recognize this ax plus b y plus cz um setup plus the intercept uh which is basically your your neural network each node adding up all the different inputs. And we can drill down into that most common formula is your y = mx plus c. So you have your uh y equals the m which is your slope. Your x value plus c which is your um y intercept. They kind of labeled it wrong here. Threw me for a loop. But the the c would be your y intercept. So when you set x equal to 0, y equals c. And that's that's your y intercept right there. Uh and that's they they just had reversed value of y. When x equals 0, it equals the y intercept which is c. and your slope gradient line which is your m. So you get your y = 2x + 3. And there's lots of easy ways to compute this. This why this is why we always start with the most basic one when we're solving one of these problems. And then of course the one of the most important takeaways is the slope gradient of the line. Uh so the slope is very important that m value. Uh in this case we went ahead and solved this. If you have y = 2x + 3 you can see how it has a nice line graph here on the right. So matrixes a matrix refers to a rectangular representation of an array of numbers arranged in columns and rows. So we're talking m rows by n columns. Here a1 is denotes the element of the first row in the first column. Similarly a12 and it's really pronounced a11 in this particular setup. So it's row one column one. A12 is a of row one column 2. uh first row and second column and so on. And there's a lot of ways to denote this. I've seen these as like a capital letter a smaller case a for the top row or I mean you can see where they can go all kinds of different directions as far as the value. You just take a moment to realize there's need to be some designation as far as what row it's in and what column it's in. And we have our uh basic operations. We have addition. So when you think about addition, you have uh two matrices of 2x two and you just add each individual number in that matrix and then when you get to the bottom you have uh in this case the solution is 12 10 + 2 is 12 5 + 3 is 8 and so on. And the same thing with subtraction. Now again you're counting matrices you want to check your um dimensions of the matrix. the shape. You'll see shape come up a lot in programming. So, we're talking about dimensions. We're talking about the shape. If the two shapes are equal, this is what happens when you add them together or subtract them. And we have multiplication. When you look at the multiplication, you end up with a very uh slightly different setup going. Now, if we look at our last one, we're um uh we're like, why this always gets to me when we get to matrices. They don't really say why you multiply matrices. Um, you know, my first thought is 1 * 2, 4 * 3. But if you look at this, we get 1 * 2 + 4 * 3, 1 * 3 + 4 * 5, uh, 6 * 2 + 3 * 3, 6 * 3 + 3 * 5. If you're looking at these matrices, uh, think of this more as an equation. And so we have uh if you remember when we back up here for our multiple line equations, let's just go back up a couple slides where we were looking at uh two variable. So this is a two variable equation. ax plus b y= c. Um and this is a way to make it very quick to solve these variables and that's why you have the matrix and that's why you do the multiplication the way they do. And this is the dotproduct of uh 1* 2 + 4 * 3 1 * 3 + 4 * 5 uh 6 * 2 + 3 * 3 6 * 3 + 3 * 5 and it gives us a nice little 14 23 21 and 33 over here which then can be used and reduced down to a sample um formula as far as solving the variables as you have enough inputs. Uh and then in matrix operations when you're dealing with a lot of matrices. Uh now keep in mind multiplying matrices is different than finding the product of two matrices. Okay? So we're talking about multiplication. We're talking about solving uh for equations. When you're finding the product, you are just finding 1* 2. Keep that in mind because that does come up. I've had that come up a number of times where I am altering data and I get confused as to what I'm doing with it. uh transpose flipping the matrix over it's diagonal comes up all the time where you have you still have 12 but instead of it being uh 128 it's now 1214 821 you're just flipping the columns in the rows. Uh and then of course you can do an inverse um changing the signs of the values across this main diagonal. And you can see here we have the inverse a to the minus one and ends up with uh instead of 12 8 14 12 it's now -22 -2 vectors uh vector just means we have a value and a direction and we have down four numbers here on our vector. uh in mathematics a one-dimensional matrix is called a vector. Uh so if you have your x plot and you have a single value that values along the x- axis and it's a single dimension. If you have two dimensions you can think about putting them on a graph. You might have x and you might have y and each value denotes a direction. And then of course the actual distance is going to be the hypothesis of that triangle. Uh and you can do that with three dimensionals x y and z. uh and you can do it all the way to nth dimensions. So when they talk about the k means uh for categorizing and how close data is together they will compute that based on the Pythagorean theorem. So you would take uh the square of each value, add them all together and find the square root. And that gives you a distance as far as where that point is, where that vector exists or an actual point value. And then you can compare that point value to another one. And makes a very easy comparison versus comparing uh 50 or 60 different numbers. And that brings us up to gene vectors and I gene values. uh igene vectors the vectors that don't change their span while transformation and I gene values the scalar values that are associated to the vectors conceptually you can think of the vector as your picture you have a picture it's um uh two dimensions x and y and so when you do those two dimensions and those two values or whatever that value is um that is that point but the values change when you skew it. And so if we take and we have a vector A and that's a set value. Uh B is um your is your you have A and B which is your gene vector. Two is the gene value. So we're altering all the values by two. That means we're um maybe we're stretching it out one direction making it tall. Uh if you're doing picture editing um that one of the places this comes in. But you can see when you're transforming uh your different information, how you transform it is then your hygiene value. And you can see here uh vector after line transition uh we have 3 a is the hygiene vector. Three is the gene value. So a doesn't change. That's whatever we started with. That's your original picture. And three uh is skewing one direction and maybe uh b is being skewed another direction. And so you have a nice tilted picture because you altered it by those by the hygiene values. So let's go ahead and pull up a demo on linear algebra. And to do this, I'm going to go through my trusted Anaconda into my Jupiter notebook and we'll create a new uh notebook called linear algebra. Since we are working in Python uh we're going to use our numpy. I always import that as np or numpy array. probably the most popular um module for doing matrices and things in given that this is part of a series. I'm not going to go too much into numpy. Uh we are going to go ahead and create two different variables. A for a numpy array 105 and b 29. We'll go ahead and run this. And you can see there's our two arrays 105 29. And I went and added a space there in between so it's easier to read. And since it's the last line, we don't have to put the print statement on it unless you want. We can simp but we can simply do a + b. So when I run this, uh, we have 10 15 29 and we get 30 24, which is what you expect. 10 + 20, 15 + 9. You could almost look at this addition as being um, just adding up the columns on here coming down. And if we wanted to do it a different way, we could also do a t plus b t. Remember that t flips them. And so if we do that, we now get them. Uh we now have 30 24 going the other way. We could also do something kind of fun. There's a lot of different ways to do this. Uh as far as a plus b, I can also do a + b. T. And you're going to see that that will come out the same. the 30 24 whether I transpose a and b or transpose them both at the end and likewise we can very easily subtract two vectors I can go a minus b and we run that and we get - 106 now remember this is the last line in this particular section that's so I don't have to put the print around it um and just like we did before we can transpose either the individual or we can transpose the main setup up and then we get a - 106 going the other way. Now, we didn't mention this in our notes, but you can also do a scalar multiplication. Let me just put down scaler so you can remember that. Uh what we're talking about here is I have uh this array here u and if I go a time u uh we'll take the value two, we'll multiply it by every value in here. So 2 * 30 is 60. 2 * 15 and just like we did before um this happens a lot because when you're doing matrices you do need to flip them. You get 6030 coming this way. So in numpy uh we have what they call dotproduct and uh with this this in a twodimensional vectors it is the equivalent of two matrix multiplication. And remember we were talking about matrix multiplication uh where it is the well let's walk through it. We'll go ahead and start by defining two um numpy arrays. We'll have uh 10 20 256 or our u and our v. Uh and then we're going to go ahead and do if we take the values uh and if you remember correctly an array like this would be 10 * 25 + 20 * 6. We'll go ahead and uh print that. There we go. And then we'll go ahead and do the np dot of u comma v. And we'll find when we do this, we go and run this uh we're going to get uh 370 370. So this is a strain multiplication where they use it to solve uh linear algebra uh when you have multiple numbers going across. And so this could be very complicated. We could have a whole string of different variables going in here. But for this we get a nice uh value for our dot multiplication. And we did um addition earlier which is just your basic addition. Uh and of course a matrix you can get very complicated on these or in this case we'll go ahead and do um let's create two complex matrixes. This one is a matrix of um you know 1210 46 431. We'll just print out A so you can see what that looks like. Here's print A. We print A out. You can see that we have a um 2x3 layer matrix for A. And we can also put together always kind of fun when you're playing with print values. Uh we could do something like this. We could go in here. There we go. Uh we could print a. We have it end with uh equals a run. And this kind of gives it a nice look. Uh here's your matrix. That's all this is. Comma n means it just tags it on the end. That's all all that is doing on there. And then we can simply add in what is a plus b. And you should already guess because this is the same as what we did before. There's no difference. Uh we do a simple vector addition. We have 12 + 2 is 14. 10 + 8 is 18. And so on. And just like we did the uh matrix addition, we can also do a minus b and do our matrix subtraction. And we look at this uh we have what? 12 - 2 is 10. 10 - 8 um where are we? [laughter] Oh, there we go. 8 min confusing what I'm looking at. I should have reprinted out the original numbers. Uh but we can see here 12 - 2 is of course 10. 10 - 8 is 2. Uh 4 - 46 is - 42 and so forth. So same as a subtraction as before, we just call it matrix subtraction. It's identical. Now if you remember up here we had uh scalar addition where we're adding just one number to a matrix. You can also do scalar multiplication. Uh and so simply if you have a single value a and you have b which is your array we can also do a time b. When we run that uh you can see here we have 2 * 4 is 8. Uh 5 * 4 is 20 and so forth. You're just multiplying the four across each one of these values. And this is an interesting one that comes up. A little bit of a brain teaser is matrix and vector multiplication. And so when we're looking at this uh we are just do a regular arrays. It doesn't necessarily have to be a numpy array. We have a which has our um array of arrays and b which is a single array. And so we can from here do the dot ab and this is going to return two values. And the verse value is that it's you could say it's like uh um we're doing the this array b array first with a and then with a second one. And so it splits it up. So you have a matrix of vector multiplication and you can mix and match. When you get into really complicated uh backend stuff, this becomes more common because you're now you got layers upon layers of data and so you you'll end up with a matrix and a set of uh vector matrices. Do you want to multiply? Now, keep in mind that if you're doing data science, a lot of times you're not looking at this. This is what's going on behind the scenes. So, if you're in um the scikit looking at sklearn where you're doing linear regression models, this is some of the math that's hidden behind the scenes that's going on. Other times, you might find yourself having to do part of this and manipulate the data around so it fits right and then you go back in and you run it through the scikit. And if we can do um up here where we did a uh matrix and vector multiplication, we can also do matrix to matrix multiplication. And if we run this where we have the two matrixes, uh you can see we have very complicated array that of course comes out on there for our dot. And just to reiterate it, we have our transpose a matrix which is your T. And so if we create a matrix A and then we do transpose it, you can see how it flips it from 5 10 15 20 2530 to 5 15 25 10 20 30 uh rows and columns and certainly with the math uh this comes up a lot. Um it also comes up a lot with XY plotting. When you put it into piplot, you have one format where they're looking at pairs of numbers and then they want all of X's and all Y's. So you know the transpose is an important tool both for your math and for plotting and all kinds of things. Another tool that we didn't discuss uh is your identity matrix. Uh and this one is more definition. Uh the identity matrix um we have here one where we just did uh two. So it comes down as one 0 0 1 uh one 0 0 1 0. It creates a diagonal of one. And what that is is when you're doing your identities, you could be comparing all your different features to the different features and how they correlate. And of course, when you have uh feature one compared to feature one to itself, it is always one. Uh where usually it's between zero one depending on how well correlates. So when we're talking about identity matrix, that's what we're talking about right here is that you create this preset matrix and then you might adjust these numbers depending on what you're working with and what the domain is. And then another thing we can do uh kind of wrap this up, we'll hit you with the most complicated uh um piece of this puzzle here is an inverse um a matrix. And let's just go ahead and put the um it's a lengthy description. >> [laughter] >> Let's go and put the description. This is straight out of the uh the website for um Numpy. Uh so given a square matrix A, here's our square matrix A, which is 2100 0 1 0 1 2 1. Keep in mind 3x3. It's square. It's got to be equal. It's going to return the matrix A inverse satisfying dot A um A inverse. So here's our matrix multiplication. Um and then of course it equals the dot uh yeah a inverse of a um with an identity shape of uh a dotshaped zero. This is just reshaping the identity. That's a little complicated there. Uh so we go and have our here's our array. Uh we'll go ahead and run this and you can see what we end up with is we end up with uh an array 0.5 minus.5 and so forth with our 211 going down 2100 0 1 0 1 2 1. Um getting into a little deep on the math understanding when you need this is probably really is is what's really important when you're doing data science versus uh handwriting this out and looking up the math and handwriting all the pieces out. you do need to know about the linear algorithm inverse of a. Uh so if it comes up, you can easily pull it up or at least remember where to look it up. We took a look at the algebra side of it. Let's go ahead and take a look at the calculus side of uh what's going on here with the machine learning. So calculus, oh my goodness, and differential equations. You got to throw that in there because that's all part of the bag of tricks, especially when you're doing large neural networks, but also comes up in many other areas. The good news is most of it's already done for you in the back end. Uh so when it comes up, you really do need to understand from the data science, not data analytics. Data analytics means you're digging deep into actually solving these math equations. U and a neural network is just a giant differential equation. Uh so when we talk about calculus, uh we're going to go ahead and understand it by talking about cars versus time and speed. uh so helps to calculate the spontaneous rate of change. Uh so suppose we plot a graph of the speed of a car with respect to time. So as you can see here going down the highway probably merged into the highway from an on-ramp. So I had to accelerate so my speed went way up uh stuck in traffic merged into the traffic. Traffic opens up and I accelerate again up to the speed limit and uh maybe it peters off up there. So you can look at this as as um the speed versus time. I'm getting faster and faster because I'm continually accelerating. And if I hit the brakes, it go the other way. So the rate of change of speed with respect to time is nothing but acceleration. How fast are we accelerating? The acceleration is the area between the start point of x and the end point of delta x. Uh so we can calculate a simple if you had x and delta x we could put a line there and that slope of the line is our acceleration. Now that's pretty easy when you're doing linear algebra but I don't want to know it just for that line and those two points. I want to know it across the whole of what I'm working with. That's where we get into calculus. So when we talk about the distance between x and delta x, it has to be the smallest possible near to zero in order to approximate the acceleration. Uh so the idea is that instead of I mean if you ever did took a basic calculus class, they would draw bars down here and you would divide this area up um let's go back up a screen. you divide this area of this time period up into maybe 10 sections and you'd use that and you could calculate the acceleration between each one of those 10 sections kind of thing. Uh and then we just keep making that space smaller and smaller until delta x is almost uh infantismally small. And so we get a function of a uh equals a limit as h goes to zero of a function of a plus h minus a function of a over h. And that is you're computing the slope of the line. We're just computing that slope under smaller and smaller and smaller samples. Uh and that's what calculus is. Calculus is the integral. You can see down here we have our nice uh integral sign. Looks like a giant s. And that's what that means is that we've taken this down to as small as we can for that sampling. Uh so we're talking about calculus. We're finding the area under the slope is the main process in the integration. Similar small intervals are made of the smallest possible length of x plus delta x where delta x approaches almost an infantismly small space. And then it helps to find the overall acceleration by summing up all the lengths together. Uh so we're summing up all the accelerations from the beginning to the end. And so here's our integral. we sum of a of x * d ofx= a + c. Uh that is our basic calculus here. So when we talk about multivvariant calculus, uh multivariate calculus deals with functions that have multiple variables and you can see here we start getting into some very complicated equations. Um uh change in w over change of time equals change of w over change of z. the differential of Z to DX, differential of X to DT, it gets pretty complicated. Uh, and it really translates into the multivariate integration using double integrals. And so you have the the sum of the sum of f ofxy of D of A equals the sum from C to D and A to B of F ofX of Y DX DY equals uh the sum of A to B sum of C to D of FXY DY DX. understanding the very specifics of everything going on in here and actually doing the math is usually calculus one, calculus 2, and differential equations. Uh so you're talking about three fulllength courses to dig into and solve these math equations. What we want to take from here is we're talking about calculus. Uh we're talking about summing of all these different slopes. And so we're still solving a linear uh expression. We're still solving y = mx + b, but we're doing this for infantismally small x's. And then we want to sum them up. That's what this integral sign means. The sum of a of x d of x= a plus c. And when you see these very complicated uh multivariate differentiation using the chain rule uh when we come in here and we have the change of w to the change of t equals the change of w dz uh and so forth. That's what's going on here. That's what these means. We're basically looking for the area under the curve which really comes to how is the change changing and speed's going up. How is that changing? And then you end up with a multiple layer. So if I have three layers of neural networks, how is the third layer changing based on the second layer changing which is based on the first layer changing? And you get the picture here that now we have a very complicated uh multivariate integration um with integrals. The good news is we can solve this uh mathematically and that's what we do when you do neural networks and reverse propagation. Uh so the nice thing is that you don't have to solve this on paper unless you're a data analysis and you're working on the back end of integrating these formulas and building the script to actually build them. So we talk about applications of calculus. Uh it provides us the tools to build an accurate predictive model. Um so it's really behind the scenes we want to guess at what the change of the change of the change is. That's a little goofy. I I know I just threw that out there. It's kind of a meta term. But if you can guess how things are going to change, then you can guess what the new numbers are. Multivariate calculus explains the change in our target variable in relation to the rate of change in the input variables. So there's our multiple variables going in there. If uh one variable is changing, how does it affect the other variable? And then in gradient descent, calculus is used to find the local and global maxima. And this is really big. Uh we're gonna actually going to have a whole section here on gradient descent because it is really I mean I talked about neural networks and how you can see how the different layers go in there but gradient descent is one of the most key things for trying to guess the best answer to something. So let's take a look at the code behind gradient descent. And uh before we open up the code, let's just do real quick uh gradient descent. Let's say we have a curve like this. And most common is that this is going to represent your error. Oops. [snorts] Error. There we go. Error. Uh hard to read there. And I want to make the error as low as possible. And so what I'm looking at it is I want to find this line here which is the minimum value. So we're looking for the minimum and it does that by uh sampling there and then it based on this it guesses it might be someplace here and it goes hey this is still going down. It goes here and then goes back over here and then goes a little bit closer and it's just playing a high low until it gets to that spot, that bottom spot. And so we want to minimize the error in uh on the flip note, you could also want to be maximizing something. You want to get the best output of it. Uh that's simply uh minus the value. Uh so if you're looking for where the peak is, this is the same as a negative for where the valley is and looking for that valley. Uh that's all that is. And this is a way of finding it. So the cool thing is um all the heavy lifting is done. Um I actually ended up putting together one of these a while back is uh when I didn't know about sidekick and I was just starting. Boy, it's a long while back and uh is playing high low. How do you play high low? not get stuck in the valleys, uh figure out these curves and things like that. Well, you do that and the back end is all the calculus and differential equations to calculate this out. The good news is you don't have to do those. Uh so instead, we're going to put together the code and let's go ahead and see what we can do with that. So, uh guys in the back put together a nice little piece of code here, which is kind of fun. uh some things we're going to note and this is this is really important stuff because when you start doing your data science and digging into your machine learning models uh you're going to find these things are stumbling blocks. Uh the first one is current x where do we start at? Uh keep in mind your model that you're working with is very generic. So whatever you use to minimize it the first question is where do we start? Um, and we started at this cuz the algorithm starts at x= 3. So, we arbitrarily picked five. Learning rate is uh how many bars to skip going one way or the other. Uh, in fact, I'm going to separate that a little bit because these two are really important. Um, if we're dealing with something like this where we're talking about um uh well, here's our here's the function we're going to use our um gradient of our function. Um 2 * x + 5. Keep it simple. So that's a function we're going to work with. So if I'm dealing with increments of a th00and 0.1 is going to be a very long time. And if I'm dealing with increments of 0.001, uh 0.1 is going to skip over my answer. So I won't get a very good answer. Um and then we look at precision. This tells us when to stop the algorithm. So again, very specific to what you're working on. uh if you're working with money and you don't convert it into a float value uh you might be dealing with 0.01 which is a penny. That might be your precision you're working with. Um and then of course the previous step size max iterations uh we want something to cut out at a certain point. Usually that's built into a lot of minimization functions. And then here's our actual uh formula we're going to be working with. And then we come in, we go while previous step size is greater than precision and its is less than max itters. Say that 10 times fast. Um, we're just saying if it's uh if we're if we're still greater than our precision level, we still got to keep digging deeper. Um, and then we also don't want to go past a thou or whatever this is, a million or 10,000 uh running. That's actually pretty high. Um we almost never do max iterations more than like a 100 or 200. Rare occasions you might go up to four or 500 if it's depending on the problem you're working with. Uh so we have our previous equals our current. That way we can track timewise. Uh the current now equals the current minus the rate times the formula of our previous x. So now we've generated our new version. Uh previous step size equals the absolute current previous. Uh, so we're looking for the change in x errors equals iterations + one. That's how we know to stop if we get too far. And then we're just going to print the local minimum occurs at x on here. And if we go ahead and run this, uh, you can see right here it gets down to this point and it says, hey, um, local minimum is minus 3.3222 for this particular series we created. Uh, and this is created off of our formula here. Lambda x2* x plus 5. Now, when I'm running this stuff, uh, you'll see this come up a lot in uh, with the sklearn kit. And and one of the nice reasons of breaking this down the way we did is I could go over those top pieces. Uh, those top pieces are everything when you start looking at these minimization toolkits and built-in code. And so from um we'll just do it's actually docs.ip.org and we're looking at the scikit. There we go. Um optimize minimize. You can only minimize one value. You have the function that's going in. This function can be very complicated. Uh so we used a very simple function up here. It could be there's all kinds of things that could be on there. And there's a number of methods to solve this as far as how they shrink down. Uh and your x knot. There's your there's your start value. So your function, your start value. Um there's all kinds of things that come in here that we can look at which we're not going to. Um optimization automatically creates constraints bounds. Some of this it does automatically, but you really the big thing I want to point out here is you need to have a starting point. You want to start with something that you already know is mostly the answer. Uh if you don't, then it's going to have a heck of a time trying to calculate it out. Or you can write your own little script that does this and and does a high low guessing and tries to find the max value. That brings us to statistics. What this is kind of all about is figuring things out. Lot of vocabulary and statistics. Uh so statistics, well, I guess it's all relative. It's definitely not an ed class. Uh so a bunch of stuff going on. Statistics statistics concerns with the collection, organization, analysis, interpretation and presentation of data. That is a mouthful. Um so we have from end to end we're does it come from is it valid? What does it mean? How do we organize it? Um how do we analyze it? Then you got to take those analysis and interpret it into something that uh people can use. kind of reduce it to understandable. Um, and nowadays you have to be able to present it. If you can't present it, then no one else is going to understand what the heck you did. So, we look at the terminologies. Uh, there is a lot of terminologies depending on what domain you're working in. So clearly if you're working in um a domain that deals with viruses and tea cells and and how does you know where does that come from and you're studying the different people then you're going to have a population if you are working with um mechanical gear u you know a little bit different if you're looking for the wobbling statistics uh to know when to replace a rotor on a machine or something like that uh that can be a big Well, you know, we have these huge fans that turn in our sewage processing systems. And so those fans, they start to wobble and hum and do different things that the sensors pick up. At one point, do you replace them? Instead of waiting for it to break, in which case it cost a lot of money. Instead of replacing a bushing, you're replacing the whole fan unit. Uh an interesting project that came up for our city a while back. Uh so population all objects are measurements whose properties are being observed. Uh so that's your population all the objects. It's easy to see it with people because we have our population and large. Um but in the case of the sewer fans we're talking about how the fan units. That's the population of fans that we're working with. You have a parameter a matrix uh that is used to represent a population or characteristic. You have your sample, a subset of the population studied. You don't want to do them all because then you don't have a if you come up with a conclusion for everyone. You don't have a way of testing it. So, you take a sample. Uh sometimes you don't have a choice. You can only take a sample of what's going on. You can't u study the whole population. And a variable, a metric of interest for each person or object in a population. Types of sampling. We have probabilistic approach. uh selecting samples from a larger population using a method based on the theory of probability and we'll go into a little bit more deeper on these. We have random systematic stratified and then you have nonprobabilistic approach selecting samples based on the subjective judgment of the researcher rather than random selection. Uh it has to do with convenience trying to reach a quota um or snowball. Uh and they're very biased. That's one of the reasons you'll see this big stamp on it says biased. Uh so you got to be very careful on that. So probabilistic sampling. Uh when we talk about a random sampling, we select randomized samples from each group or category. So we it's as random as you can get. Uh we talk about systematic sampling. We're selecting randomized samples from each group or category with a fixed periodic interval. Uh so we kind of split it up. This would be like a time setup or our different categories. And you might ask your question, what is a category or a group? Uh if you look at I'm going to go back a window. Let's say we're studying um economics of different of an area. Um we know pretty much that based on their culture, where they came from, they might need to be separated. And so uh and when I say separated, I don't mean separated from their their uh place where they live. I mean, as far as the analysis, we want to look at the different groups and make sure they're all represented. So, if we had like an 80% uh of a group that is uh say Hispanic and or Indian and also in that same area, we have 20% 20% who are it's called our expatriots. They left America and they're nice and uh your Caucasian group. We might want to sample a group that is representative of both. Uh so we're talking about stratified sampling and we're talking about groups. Those are the groups we're talking about. And it brings us to stratified sampling. Selecting approximately equalized samples from each group or category. Uh this way we can actually separate the categories and give us an insight into the different cultures and how that might affect them in that area. Uh so you can see these are very very different kind of depends on what you're working with um as far as your data and what you're studying. And so we can see here just to go a little bit more we'd have selecting 25 employees from a company of 250 employees randomly. Don't care anything about them. What groups are in which office are in nothing. Um and we might be selecting one employee from every 50 unique employees in a company of 250 employees. And then we have selecting one employee from every branch in the company office. So we have all the different branches. There's our group or our categories by the branch. And the category could depend on what you're studying. So it has a lot of variation on there. You see this kind of grouping and categorizing is also used to generate a lot of misinformation. Uh so if you only study one group and you say this is what it is, then everybody assumes that's what it is for everybody. And so you got to be very careful of that and it's very unethical thing to kind of do. So types of statistics uh we talk about statistics we're going to talk about descriptive and inferential statistics. There are so many different terms in statistics to break it up. Uh so we so we're talking about a particular setup. So we're talking about descriptive and inferential uh statistics. you the base of the word describe is pretty solid. You're describing the data. What does it look like with inferial statistics? We're going to take that from the small population to a large population. So, if you're working with a drug company, uh you might look at the data and say these people were helped by this drug. They did uh 80% better as far as their health or 80% better survival rate than the people um who did not have the drug. So we can infer that that drug will work in the greater populace and will help people. So that's where you get your inferential. Uh so we are predicting how it's going to affect the greater population. So descriptive statistics it is used to describe the basic features of data and form the basis of quantitative analysis of data. So we have a measure of central tendencies. We have your mean, median and mode. And then we have a measure of spread like your range, your interquartile range, your variance, and your standard deviation. And we're going to look at all these a little deeper here in a second. Uh but one of them you can think of is um how the data difference differences, you know, what's the max min range, all that stuff is your spread. And anything that's just a single number is usually your central uh tendencies, measure of central tendencies. So we talk about the mean. It is the average of the set of values considered. What is the average outcome of whatever is going on? And then your median separates the higher half and the lower half of data. Uh so where's the center point of all your different data points? So your mean might have some a couple really big numbers that skew it uh so that the average is much higher than if you took those outliers out where the median would by separating the high from the low might give you a much lower number. You might look at and say, "Oh, that's that's odd. Why is the average so much higher than the median?" Well, it's because you have some outliers or why is it so much lower? And then the mode is the most frequent appearing value. Uh this is really interesting if you're studying economics and how people are doing. You might find that the most common um income like in the US was at one point 24,000 a year where the average was closer to 80,000 and it's like wow what a difference. Well, there's some people have a lot of money and so that skews that way up. So the average person is not making that kind of money. And then you look at the median income and you're like well the median income is a little bit closer to the average. Uh so it does create a very interesting way of looking at the data. Again these are all uh central tendencies single numbers you can look at for the whole spread of the data. And we look at the measure of central tendencies. The mean is the average marks of a students in a classroom. So here we have the mean sum of the marks of the students total number of students. And as we talked about the median, uh if we have 0ero through 10 and we take half the numbers and put them on one side of the line, half the numbers on the other side of the line, uh we end up with five in the middle. And then the mode, what mark was scored by most of the students in a test. In a simple case where most people scored like an 82% and got certain problems wrong, easy to figure out. uh not so easy when you have different areas where like you have like the um oh let's go back to economy a little bit more difficult to calculate if you have a large group the scores that makes 30,000 and a slightly bigger group that makes 26,000. So what do you put down for the mode? Uh certainly there's a number of ways to calculate that and there's actually a different variations depending on what you're doing. So now we're looking at a measure of spread uh range. What's the difference between the highest and the lowest value? First thing you want to look at, you know, it's we had everybody in the test scored between 60 and 100%. Somebody got 100% or maybe 60 to 90%. It was so hard that a lot of people could not get 100%. Um, and you have your interquartile range. Quartortiles divide a rankordered data set into four equal parts. Very common thing to do as part of all the basic packages. Whether you're working in uh data frames with pandas, whether you're working in scala, whether you're working in R, um you'll see this come up where they have range, your min, your max, and then it'll have your interquartile range. How does it look like in each quarter of data? Variance measures how far each number in the set is from the mean and therefore from every other number in the set. uh so you have like a how much turbulence is going on in this data and then the standard deviation it is the measure of the variance or the dispersion of a set of values from the mean and you'll usually see uh if I'm doing a graph I might have the value graphed um and then based on the the error I might graph graph the standard deviation and the error on the graph as a background so you can see how far off it is uh so standard deviation's used a lot So measurement of spread uh marks of a student out of a 100 uh we have here from 50 to 63 or 50 to 90 uh so the range maximum marks minimum marks we have 90 to 45 and the spread of that is 45 90 minus 45 and then we have the interquartile range using the same marks over there you can see here where the median is and then there's the first quarter the second quarter and the third quarter based on splitting it apart by those values And to understand the variance and standard deviation, we first need to find out the mean. Uh so here's our our you know calculating the average there. We end up at approximately 66 for the average. And then we look at that the variance once we know the means we can do equals the marks minus the mean squared. Why is this squared? Uh because one, you want to make sure it's you don't have like if you if you're putting all this stuff together, you end up with an error as far as one's negative, one's positive, one's a little higher, one's a little lower. Uh so you always see the squared value and over the total observations. And so the standard deviation equals the square root of the variance, which is approximately 16. And if you were looking at um a predictable model, you would be looking at the deviation based on the error. How much error does it have? Uh that's again really important to know if you're if your prediction is predicting something, what's a chance of it being way off or just a little bit off. Now that we've looked at the um tools as far as some of the basics for doing your statistics and what we're talking about, let's go ahead and pull up a little demo and show you what that looks like in Python code. Uh so you can get some little hands-on here. For that, let's go back into our Jupyter notebook in Python. Now, almost all of this you can do in NumPy. Last time we worked um in Numpy. This time we're going to go ahead and use pandas. And if you remember from pandas on here, uh, this is basically a data frame, rows, columns. Let's just go ahead and do a print df.head and run that. And you can see we have uh the name Jane, Michael, William, Rosie, Hannah, and their salaries on here. And of course instead of having to do all those hand calculations and add everything together and divide by the total we can do something very simple on this uh like use the command mean in pandas. And so if I go ahead and do this print df pick our column salary because we want to find the means of that collery. We want to find the means of that column. Uh and we go and print this out and you can see that the uh average income on here is 71,000. Uh, and let's just go ahead and do this. We'll go ahead and put in uh means. And if we're going to do that, we also might want to find the median. And the median is uh very similar except it actually is just median. Uh we're used to means and average. It's kind of interesting that those are they use the two different words. Uh there can be in some computations slight differences but for the most part the means is the average. Uh and then the median oops let's put a median here. DF salary that way it displays a little better. We can see the median is 54 um000. So the halfway mark is significantly below the average. Why? Because we have somebody in here who makes 189,000. Darn you, Rosie, for throwing off our numbers. Uh, but that's something you'd want to notice. This is this is the difference between these is huge. And so is what is the meaning behind that when you're studying a populace and looking at uh the different data coming in. And of course, we also want to find out, hey, what's the most uh common income that people make in this little tiny sample? And so, we'll go ahead and do the mode. And you can see here with the mode uh it's at 50,000. So this is this is very telling that most people are making 50,000. The middle point is at 54,000. So half the people are making more than that. What that tells me is that if the most common income is way is below the median, then there's a few there's a you know there's a a lot of high salaries going up, but there's some really low salaries in there. And so this trend which is very common in statistic you when you're analyzing the economy and different people's income is pretty common. And the bigger difference between these is also very important when we're studying statistics. Uh and when you hear someone just say hey the average income was you might start asking questions at that point. Why aren't you talking about the median income? Why aren't you talking about the mode the most common income? What are you hiding? Uh and if you're doing these analysis, you should be looking at these saying, "Hey, why why are this discrepancies? Why are these so different?" And of course, with any uh analysis, it's important to find out the minimum and the maximum. So, we'll go ahead. It's just simply uh um min'll pull up your minimum and then do max pulls up the maximum. pretty straightforward on as far as um translating it and knowing what your you know what the your lowest value and what your highest value is here. Um which you'll use to generate like a spread later on. And real quick on no mode mode uh note that it puts mode zero. Like I said, there's a couple different ways you can compute the mode. Um although the standard one's pretty good. We can of course do the range which is your max minus your min. So now we have a range of 149,000 between the upper end and the lower end. And you might want to be looking up the individual values on all of these. But it turns out there is a describe feature in pandas. And so in pandas we can actually do df salary describe. And if we do this you can see we have that there's seven uh setups. Here's our mean. Um, our standard deviation, which we didn't compute yet, which would just be a STD. And you got to be a little careful because when it computes it, it it looks for axes and things like that. Uh, we have our minimum value, and here's our cortiles, uh, our maximum value, and then of course the name salary. Uh, so these are the these are the basic statistics. You can pull them up and just describe. This is a dictionary. So I could actually do something like um in here I could actually go uh count and run. And now it just prints the count. Uh so because this is a dictionary, you can pull any one of these values out of here. It's kind of a quick and dirty way to pull all the different information and then split it up depending on what you need. Now if I just walked in and gave you this information um in a meeting, at some point you would just kind of fall asleep. That's what I would do anyway. Um, so we want to go ahead and and see about graphing it here. And we'll go ahead and put it into a histogram and plot that graph on it of the salaries. And let's just go ahead and put that in here. So we do our map plot inline. Remember that's a Jupiter's notebook thing. Uh, a lot of the new version of the mattplot library does it automatically, but just in case I always put it in there. Uh, import mattplot library piplot as plt. That's my plotting. And then we have our data frame. Uh I don't I guess I really don't need to respell the data frame. Maybe we could just remind oursel what's in it. So we'll go ahead and just uh print DF. That way we still have it. And then we have our salary. DF salary salary.plot history title salary distribution color gray. Uh plot AXV line salary the mean value. So, we're going to take the mean value um color violet line style dash. This is just all making it pretty. Uh what color dash line width of two that kind of thing. And the median. And let's go ahead and run this just so you can see what we're talking about. And so up here we are taking on our plot. Um so here's the data. Here's our our data frame printed out so you can see it with the salaries. We're looking at the salary distribution and just look at this the way they're the salary is distributed. Um you have our in this case we did let's see we had red for the median. We have violet for our average or mean and you can just see how it really here's our outlier. Here's our person who makes a lot of money. Here's the um average and here's the median. Um and so as you look at this you can say wow um based on the average it really doesn't tell you much about what people are really taking home. All it does is tell you how much money is in this you know what the average salary is. So some of the things you want to take away in addition to this is that it's very easy to plot um an AXV line. These are these up and down lines for your markers. Um, and as you display display the data, I mean, you can add all kinds of things to this and get really complicated. Keeping it simple is pretty straightforward. I look at this and I can see we have a major outlier out here. We can definitely do a histogram and stuff like that. Um, but you know, picture's worth a thousand words. What you really want to make sure you take away is that we can do a basic describe which pulls all this information out and we can print any of the individual information from the describe uh because this is a dictionary. And so if we want to go ahead and look up um the mean value, we can also do describe mean. So if you're doing a lot of statistics, uh being able to doesn't have the print on there, so it's only going to print um the last one, which happens to be the mean. Uh you can very easily reference any one of these. And then you can also, if you're doing something a little bit more complicated and you don't need just the basics, you can come through and pull any one of the individual um references from the co from the pandas on here. So now we've had a chance to describe our data. Uh let's get into inferential statistics. Inferential statistics allows you to make predictions or inferences from data. And you can see here we have a nice little picture movie ratings and um if we took this group of people and said hey how many people like the movie dislike it can't say and then you ask just a random person who comes out of the movie who hasn't been in this study uh you can infer that 55% chance of saying liked 35% chance of saying disliked or a 10 or 11% chance of can't say. So that that's real basics of what we're talking about is you're going to infer that the next person is going to follow these statistics. Uh so let's look at point estimation. Uh it is a process of finding an approximate value for a population's parameter like mean or average from random samples of the population. Let's take an example of testing vaccines for COVID 19. Uh vaccines and flu bugs, all that. It's a pretty big thing of how do you test these out and make sure they're going to work on the populace. A group of people are chosen from the population. Medical trials are performed. Results are generalized for the whole population. So here's a protected here's our small group up here where we've selected them. We run medical trials on them and then the results work for the population. You nice diagram with the arrows going back and forth and the very scary co virus in the middle of one. And let's take a look at the applications of inferial statistics. Very central is what they call hypothesis testing uh and the confidence interval which go with that. And then as we get into probability, we get into our binomial theorem, our normal distribution and central limit theorem. Hypothesis testing. Hypothesis testing is used to measure the plausibility of a hypothesis assumption by using sample data. Now when we talk about theorems, theory, hypothesis, uh keep in mind that if you are in a philosophy class, theory is the same as hypothesis where theorem is a scientific uh statement that is something that has been proven although it is always up for debate because in science we always want to make sure things are up to debate. So a hypothesis is the same as a phil philosophical class calling a theory where theory in science is not the same. Theory in science says this has been well proven. Gravity is a theory. Uh so if you want to debate the theory of gravity try jumping up and down. If you want to have a theory about why the economy is collap collapsing in your area that is a philosophical debate. Very important. I've heard people mix those up and it is a pet peeve of mine. When we talk about hypothesis testing, the steps involved in hypothesis testing is first we formulate a hypothesis. We figure out the right test to test our hypothesis. We execute the test and we make a decision. And so when you're talking about hypothesis, you're usually trying to disprove it. If you can't disprove it and it works for all the facts, then you might call that a theorem at some point. So in a use case uh let's consider an example. We have four students who were given a task to clean a room every day. Sounds like working with my kids. They decided to distribute the job of cleaning the room among themselves. They did so by making four chits which has their names on it and the name that gets picked up has to do the cleaning for that day. Rob took the opportunity to make chits and wrote everyone's name on it. So here's our four people. Nick, Rob, Ilia, Ilia, and Summer. Now Rick, I imlia and summer are asking us to decide whether Rob has done some mischief in preparing the chits i.e whether Rob has written his name on one of the chit. For that we will find out the probability of Rob getting the cleaning job on first day, second day, third day and so on till 12 days. The probability of Rob getting the job decreases every day. I.e. his turn never comes up then definitely he has done some mischief while making the chits. So the probability of Rob not doing work on day one is uh three out of four. There's a 75 chance that he didn't do work. Uh two days 34s * 34 equals.56. 3 days you have 34 34 34s which equals 42. Uh when you get to day 12 it's 0032 which is less than 0.05. Remember this 0.05 005. Uh, that comes up a lot when we're talking about um certain values when we're looking at statistics. Rob is cheating as he wasn't chosen for 12 consecutive days. That's a very high probability when on day 12 he still hasn't gotten the job cleaning the room. So, we come up to our important important terminologies. We have null hypothesis. a general statement that states that there is no relationship between two measured phenomenon or no assoc association among the groups. Alternative hypothesis contrary to the null hypothesis it states whenever something is happening a new theory is preferred instead of an old one. And so the two hypothesis go hand in hand. Uh so your null this is always interesting in in talking about data science and the math behind it. It's about proving that the things have no correlation. Null hypothesis says these two have zero relation to each other. Where the alternative hypothesis says hey we found a relation. This is what it is. We have p value. The p value is a probability of finding the observed or more extreme results when the null hypothesis of a study question is true. And the t value it is simply the calculated difference represented in units of standard error. The greater the magnitude of t, the greater the evidence against the null hypothesis. And you can look at the t value as being specific to the test you're doing where the p value is derived from your t value and you're looking for what they call the 5% or the 0.05 showing that it has a high correlation. So digging in deeper, let's assume that a new drug is developed with the goal of lowering the blood pressure more than the existing drug. And this is a good one because uh the null value here isn't that you don't have any drug. The null value here is that it's better than the existing drug. The new drug doesn't lower the blood pressure more than the existing drug. Now if we get that uh that says our null hypothesis is correct. There is no correlation and the new drug is not doing its job. The alternative hypothesis the new drug does significantly lower the blood pressure more than the existing drug. Uh yay we got a new drug out there. And that's our alternative hypothesis or the H1 or HA. And we look at the p value results from the evidence like medical trials showing positive results which will reject the null hypothesis. And again they're looking for um a 0.05 or 5%. And the t value comparing all the positive test results and finding means of different samples in order to test hypothesis. So this is specific to the test. How uh what percentage of increase did they have? And this leads us to the confidence intervals. Uh a confidence interval is a range of values we are sure our true values of observations lie in. Let's say you asked a dog owner around you and asked them how many cans of food do you buy for your uh per year for your dog. Through calculations you got to know that the on average around 95% of the people bought around 200 to 300 cans of food. Hence we can say that we have a confidence interval of 230 where 95% of our values lie in that data spread. Uh and this the graph really helps a lot. So you can start seeing what you're looking at here where you have the 95% you have your peak in this case it's a normal distribution. So you have the nice bell curve equal on both sides. It's not asymmetrical. And 95% of all the values lie within a very small range. And then you have your outliers the 2.5% going each way. So we touched upon hypothesis. Uh we're going to move into probability. Uh so you have your hypothesis. Once you've generated your hypothesis, we want to know the probability of something occurring. Probability is a measure of the likelihood of an event to occur. Any event can be predicted with total certainty and can only be predicted as a likelihood of its occurrence. So any event cannot be predicted with total certainty. It can only be predicted as a likelihood of its occurrence. Uh score prediction. how good you're going to do in whatever sport you're in, weather prediction, stock prediction, if you've studied physics and chaos theory, even the location of the chair you're sitting on has a probability that it might move 3 ft over. Granted, that probability is one in like uh I think we calculated as under one in trillions upon trillions. So, it's the better the probability, the more likely it's going to happen. There are some things that have such a low probability that we don't see them. So we talk about a random variable. Uh random variable is a variable whose possible values are numerical outcomes of a random phenomena. So uh we have the coin toss. How many heads will occur in the series of 20 coin flips? Probably you know the on average there are 10, but you really can't know because it's very random. How many times a red ball is picked from a bag of balls if there's equal number of of red balls and blue balls and green balls in there. How many times the sum of digits on two dice uh result or five each? Um so you know there's how often you're gonna roll two fives on your pair of dice. So in a use case uh let's consider the example of rolling two dice. We have a random variable outcome equals y. You can take values 2 3 4 5 6 7 8 9 10 11 12. So we have a random variable in a combination of dice and instead of looking at how many times um both dice will roll five. Let's go ahead and look at a total sum of five. And you have in as far as your random variables you can have a one four equals 5 4 1 2 3 32. So four of those roles can be four if you look at all the different options you have four of those random rolls can be a five. And if [snorts] we look at the total number, which happens to be 36 different options, uh you can see that we have four out of 36 chance every time you roll the dice that you're going to roll a total of five. You're going to have an outcome of five. And uh we'll look a little deeper as to what that means. Uh but you could think of that at what point if someone never rolls a five or they always roll a five, can you say, "Hey, that person's probably cheating." uh we'll look a little closer at the math behind that but let's just consider this as one of the cases is rolling two dice and gambling. There's also a binomial distribution. It is the probability of getting success or failure as an outcome in an experiment or trial that is repeated multiple times and the key is is by meaning two binomial uh so passing or failing an exam winning or losing a game and getting either head or tails. So if you ever see binomial distribution, it's based on a um true false kind of setup. You win or lose. Let's consider a use case and let's consider the game of football between two clubs, Barcelona and Dortmund. The teams will have to play a total of four matches and we have to find out the chances of Barcelona winning the series. So we look at the total games and we're looking at five different games or matches. Let's say that the winning chance for Barcelona is 75% or 75. That means at each game they have a 75% chance that they're going to win that game and losing chances are 25% or 0.25. Clearly 75 plus 0.25 equals 1. So that accounts for 100% of the game. Probability for getting K wins in n matches is calculated. And we we're talking like so if you have five games uh and you want to know if I play um how many wins in those five games should I get? What's a percentage on those? And the probability for getting k wins in n matches is calculated by px= k= n ck p q to the n minus k. Here p is the probability of success and q is the probability of failure. And so we can do total games of n= 5 where k equals 01 2345. P which is the chance of winning is 75. Q the chance of losing equals 1 - p which equals 1 - 0075 which equals.25. The probability that Barcelona will lose all of the matches can then just plug in the numbers and we end up with a009765625. So very small chance they're going to lose all their matches. And we can plug in uh the value for two matches. Probability that Barcelona will win at least two matches is 00878. And of course we can go on to probability that Barcelona will win three matches the 26 and of course four matches and so on. And it's always nice to take this information um and let let's find the cumulative discrete probabilities for each of the outcomes where Barcelona has won three or more matches x= 3 x= 4 x= 5 and we end up with the p =264 plus 395 + 237 which equals89. In reality the probability of Barcelona winning the series is much higher than 75. And it's always nice to uh put out a nice graph so you can actually see the number of wins to the probability and how that pans out with our binomial case. Continuing in our important terminology, location. The location of the center of the graph depends on the mean value. And uh this is some very important things. So much of the data we look at and when you start looking at probabilities almost always has a normalized look like the graph in the middle. uh but you do have left skewed where the data is skewed off to the left and you have more stuff happening off to the left and you have right skewed data and so when this comes up and these probabilities come up where they're skewed it's really important to take a closer look at that. Uh mostly you end up with a normalized set of data but you got to also be aware that sometimes it's a skewed data and then the height height of the slope inversely depends upon the standard deviation. So you can see down here the standard deviation is really large. it kind of squishes it out. And if the standard deviation is small, then most of your data is going to hit right there in the middle. You're going to have a nice peak. Um, and so being aware of this that you might have a probability that fits certain data, but it has a lot of outliers. So you're if you have a really high standard deviation, um, if you're doing stock market analysis, this means your predictions are probably not going to make you much money. uh where if you have a very small deviation, you might be right on target and set to become a millionaire. Which leads us to the zcore. Zcore tells you how far from the mean a data point is. It is measured in terms of standard deviations from the mean. Around 68% of the results are found between one standard deviation. Around 95% of the results are found between two standard deviations. And you read the symbols. Of course, they love to throw some Greek letters in there. we have mu minus 2 sigma. Mu is just a quick way. It's that kind of funky u. It just means the mean. Uh and then the sigma is the standard deviation. And that's the o with a little arrow off to the right or the little waggly tail going up. The o with a with a line on it. Uh so mu minus 2 sigma is your uh 95% of the results are found between two standard deviations. Central limit theorem. This goes back to the skew. If you remember, we were looking at the skew values on this previous slide. Have left skewed, normalized, and right skewed. When we're talking about it being skewed or not skewed, the distribution of the sample means will be approximately normally distributed, evenly distributed, not skewed. If you take large random samples from the population with the mean mu and the standard deviation sigma with replacement and you can see here um uh of course we have our uh mu minus 2 sigma and the spread down here the mean the median and the mode and so when you're talking about very large populations these numbers should come together and you shouldn't have a skewed value. If you do that's a flag that something's wrong. That's why this is so important to be aware of what's going on with your data, where your samples are coming from, and the math behind it. And if you're going to do all this, we got to jump into conditional probability. The conditional probability of an event A is a probability that the event will occur given the knowledge that an event B has already occurred. And you'll see this as Baze theorem. B A Y S bay. Uh, and this is read. I mean, you have these funky looking little P brackets. A B. This is the probability of A being true while B is already true. And you have the probability of B being true when A is already true. So, P B of A probability of A being true divided by the probability of B being true. And we talk about BA's theorem which occurred back in the 1800s when he discovered this. This is such an important formula and it's really it's not if you actually do the math you could just kind of do um um XY equals J K and then you divide them out and you're going to see the same math but it works with probabilities which makes it really nice. And so if you have a s you might have uh eight or nine different studies going on in different areas different people have done the studies they brought them together. Um if we look at today's co virus the virus spread uh certainly the studies done in China versus the studies the way they're done in the US that data is different in each of those studies but if you can find a place where it overlaps where they're studying the same thing together you can then compute the changes that you need to make in one study to make them equal and this is also true if you have a study of uh um one group and you want to find out more about it. So this formula is very powerful. Uh it really has to do with the data collection part of the math and data science and understanding where your data is coming from and how you're going to combine different studies in different groups. And we'll go ahead and go into a use case. Uh let's find out the chance of a person getting lung disease due to smoking. Uh and this is kind of interesting the way they word this. Um let's say that according to medical report provided by the hospital states that around 10% of all patients they treated suffered lung lung disease. Uh so we have kind of a generic medical report. They further found out uh by a survey that 15% of the patients that visit them smoke. So we have 10% that are lung disease and um 15% of the patients smoke. And finally, 5% of the people continued smoke even when they had lung disease. Uh not the brightest choice um but you know it is an addiction so it can be really difficult to kick. And so we can look at the probability of a uh prior probability of 10% people having lung disease. And then probability b probability that a patient smokes is 15%. Uh and the probability of B um if B then A. The probability of a patient smokes even though they have lung disease is 5%. And probability of A is B probability that the patient will have lung disease if they smoke. And then when you put the formulas together uh you get a nice solution here. You get the probability of A of B probability that the patient will have lung disease if they smoke. And you can just plug the numbers right in and we get a 3.33% chance. Hence, there is a 3.33% chance that a person who smokes will get a lung disease. So, we're going to pull up a little Python code. Always my favorite. Roll up the sleeves. Keep in mind, we're going to be doing this um kind of like the backend way so that you can see what's going on. And then later on we're going to create um we'll get into another demo which shows you some of the tools that are already pre-built for this. Let's start by creating a set. So we're going to create a set with curly braces. This means that our set has um only unique values. So you have a list uh you have your tupils which can never change and then you have um in this case the the set. So 47, you can't create a 47, 4. It'll delete the four out. So it's only unique values. And if you use dictionaries, quick reminder, this should look familiar because it is a dictionary uh where you have a value and that value is assigned to or that key is assigned to a value. Uh so you could have a key value set up as a dictionary. So it's like a dictionary without the value. It's just the keys and they all have to be unique. And if we run this, we have a set of 47. We can also take a list, a regular um setup. And I'm going to go ahead and just throw in another number in here, four, and run it. Uh, and you can see here if I take my list 1 2 3 4, and I convert it to a set, and here it is. My set from list equals set my list. The result is 1 2 3 4. So it just deletes that last four right out of there. And with the sets, you can also go in there and um print here is my set. My set uh three is in the set. And then if you do three in my set, that's going to be a logic function. Uh and one in my set, six is not in the set and so forth. If we run this, we get uh three is in the set true one is in the set false because 357 is another one. Six is in the set uh six is not in the set. So not in my set. You can also use this with a list. We could have just used 357 and it would have um the same response on there is three in usually do if three is in but three in my set is still works on just a regular list. and we'll go ahead and do a little iteration. We're going to do kind of the dice one. Remember, um, uh, 1 2 3 4 5 six. And so, we're going to bring in an iteration tool and import product as product. And, uh, I'll show you what that means in just a second. So, we have our two dice. We have dice A, and it's going to be a set of values. Um, you can only have one value for each one. That's why they put it in a set. And if you remember from range, it is up to seven. So this is going to be 1 2 3 4 5 six. It will not include the seven. And the same thing for our dice B. And then we're going to do is we're going to create a list which is the product of A and B. So what's U A plus B? And if we go ahead and run this uh it'll print that out. And you'll see um in this case when they say product because it's an iteration tool, we're talking about creating a tupole of the two. So we've now created a tupole of all possible outcomes of the dice where dice A is 1 2 3 1 6 and dice B is 1 to six. And you can see 1 1 2 1 3 and so forth. You remember we had a slide on this earlier where we talked about um the different all the different outcomes of a dice. We can play around with this a little bit. Uh we can do in dice equals 2 divi dice faces 1 2 3 4 5 6. Uh another way of doing what we did before and then we can create an event space where we have a set which is the product of the dice faces repeat equals end dice. And we'll go and just run this. And you can see here it just again puts it through all the different possible variables we can have. And then if we wanted to take the same uh set on here and print them all out like we had before uh we can just go through for outcome and event space outcome end equals. So the event space is creating a sequence and as you can see here when we print it out it stacks them versus going through and putting them in a nice line. and we'll go ahead and do something. Um, let's go print. Since we have the end printing with a comma, that just means it's just going to it's not going to hit the return going down to the next line. Uh, and we'll go ahead and do the length of our event space. Uh, that'll be an important variable we're going to want to know in a minute. And of course, if I get carried away with my typing of length, uh, we'll print it twice and it'll give me an error. Uh so we have 36 different possible variations here and we might want to calculate something like um what about the multiple of three? What if we want to have uh the probability of the multiple of three in our setup? And so uh we can put together the code for the outcome in event space of xy = outcome if x + y remainder 3. So, we're going to divide by three and look at the remainder and it equals zero. Then it's a favorable outcome and we're going to pop that outcome on the end there. And we'll turn it into a set. So, the favor outcome equals a set. Not necessary uh because we know it's not going to be repeating itself, but just in case, we'll go ahead and do that. And if we want to print out the outcome, we can go ahead and see what that looks like. And you can see here these are all u multiples of three. Uh 1 + 2 is 3, 5 + 4 is 9, which divided by 3 is 3, and so forth. And just like we looked up the length uh of the one before, let's go ahead and print the length of our f outcome so we can see what that looks like. There we go. And of course, I did forget to add the print in the middle because we're looping through and putting an end on the on the setup on there. So, we're going to put the print in there. And if I run this, you can see uh um we end up with 12. So, we have 36 total options. Uh we have 12 that are multiple that um add up to a multiple of three. And we can easily conver compute the probability of this uh by simply taking the length of our favorable outcome over the length of the event space. And if we print it out, let me put that in there. Probability last line. So we just type it in, we end up with a 3333 chance. And it's roughly a third. And we might want to make this look nice. So let's go ahead and put in another line there. The probability of getting the sum which is a multiple of three is 3333. We can compute the same thing for five dice. And if we do this for five dice and go and run it, you can see we just have a huge amount of choices. So it just goes on and on down here. And we can look at the uh length of the event space. And we have over 7,776 choices. That's a lot of choices. And if we want to ask the question like we did above, uh what is the sum where the sum is a multiple of five but not a multiple of three, we can go through all of these different options. And then uh you can see here uh d1 d2 d3 d4 d5 equals the outcome. And if uh you add these all together and the division by five does not have a remainder of zero but the remainder is also of a division by three is not equal to zero. So the multiple of five is equal to zero but the multiple of three is not. We can just appin that on here and then we can look at that uh favorable outcome. We'll go ahead and set that and we'll just take a look at this. What's our length of our favorable outcome? It's always good to see what we're working with. And so we have 94 out of 776. And then of course we can just do a simple division to get the probability on here. What's the probability that we're going to roll a multiple of five when you add them together? but not a multiple of three. And so we're just going to divide those two numbers. And you can see here we get uh.16255 or 11.62%. And so you can really have a nice visual that this is not really complicated math right here on probabilities. uh it's just how many options do you have and how many of those are you possibly going to be able to um come up with with the solution you're looking for. And this leads us to a confusion matrix. A confusion matrix is a table which is used to describe the performance of a classification model on a set of test data for which the true values are known. And so you'll see on the left we have the predicted and the actual and we have a negative uh false negative positive true positive um and then we have false positive and true negative. And you can think of this as your predicted model. What does that mean? That means if you divided your data and you use twothird of it to create the model, you might then test it against an actual case for the last third to see how well it comes out. How many times was it uh true positive versus uh false positive? It gave a false positive response. And you can imagine in medical uh situations, this is a pretty big deal. You don't want to give a false positive. So you might adjust your model accordingly so you don't have a false positive. say with a co virus test, it'd be better to have a false negative and then go back and get retested than to have 30% false positives where then the test is pretty much invalid. So in a use case uh like cancer prediction, let's consider an example where a cancer prediction model is put to the test for its accuracy and precision. Actual result of a person's medical report is compared with the prediction made by the machine learning model. And so you can see here, here's our actual predicted uh whether they have cancer or not. You know, cancer, a big one. You don't want to have a uh false positive. I mean, a false negative. In other words, you don't want to have it tell you that you don't have cancer when you do. So, that would be something you'd really be looking for in this particular domain. You don't want a false negative. Uh and this is again, you know, you've created a model, you have hundreds of people or thousands of pieces of data that come in. There's a real famous case study where they have the imagery and all the measurements they take. And there's about 36 different measurements they take. And then if you run the a basic model, you want to know just how accurate it is. How many um negative results do you have that are either telling people they have cancer that don't or telling people that don't have cancer that they do. And then we can take these numbers and we can feed them into our accuracy, our precision, and our recall. Uh so accuracy, precision and recall accuracy metric to measure how accurately the results are predicted and this is your um total um true where you got the right results you add them together the true positive the true negative over all the results. So what percentage of them were accurate versus what were wrong. We talk about precision is a metric to measure how many of the correctly predicted cases are actually turned out to be positive. Uh so we have a precision on true positive. Again, if you're talking about like uh COVID testing with the viruses, uh you really want this to be a a high number. You want this true um that to be the center point where you might have the opposite if you're dealing with cancer where you want no false negatives. Uh so this is your metric on here. Precision is your test positive uh true positive plus uh false positive and then your recall how many of the actual positive cases we were able to predict quickly with our model. Uh so test positive is the test positive plus the false negative on there. And we'll want to go ahead and do a demo on the naive baze classifier. Before I get too far into uh naive base classifier because we're going to pull it from the sklearn or the uh scikit um let's go ahead kind of an interesting page here for classifiers. When you go into the sklearn kit there's a lot of ways to do classification and I'll just zoom up in here so you can see some of the titles. Uh there's everything from the nearest neighbor linear um but we're going to be focusing on the naive bays over here. And this is just um a sample data set that they put together. And you can see how some of these have a very different output. The naive base, remember, is set up as probably the most simplified uh calculator or um set of predictions out there. And so what we've been talking about with the true false and stuff like that where there's a uh an belief that there is a independent assumption between the features where the features are very assumed to have some kind of connection uh then we can go ahead and use that for the prediction. And so that's what we're using as a naive bay classifier versus many of the other classifiers that are out there. For this we're going to use uh the social network ads. It's a little data set on here and let me go and just open that up the file. Uh here we go. It has user ID, gender, age, estimated salary, uh purchased. And so we have you can see the user ID male 19 uh estimated salary 19,000 and purchased zero. Uh so it's either going to make a purchase or not. So look at that last one 01. We should be thinking of binomials. we should be thinking of a simple naive base classifier kind of setup. So if we close this out, we're going to go ahead and import our numpy as np. We're nice to have a a good visual of our data. So we'll put in our mapplot library. Here's our pandas, our data frame. Uh and then we're going to go ahead and import the data set. And the data set's going to be we're going to read it from the social network ads.csv. Then we're going to print the head just so you can see it again. Uh even though I showed you it in the file. And X equals the data set I location uh two three values and Y is going to be the four uh column 4. Let me just run this so it's a little easier to go over that. Um you can see right here we're going to be looking at uh 012 is age and estimated salary. So 2 three and that's what I location just means um that we're looking at the number versus a regular location. Uh regular location you'd actually say age and estimated salary. And then column four is did they make a purchase? They purchased something. Uh so those are the three columns we're going to be looking at when we do this. And we've gone ahead and imported these and imported the data. So now our data set is all set with this information in it. And we'll need to go ahead and split the data up. Uh so we need our from the sklearn model selection we can import train test split. Uh this does a nice job. We can set the random state so it randomly picks the data. And we're just going to take uh 25% of it is going to go into the test our x test and our y test and the 75% will go to x train and y train. And that way once we create our model, we can then have data to see just how accurate or how well it has performed with our um prediction. The next step in pre-processing our data is to go ahead and do feature scaling. Now, a lot of this is start to look familiar. If you've done a number of the other modules and setup, you should start noticing that we bring in our data. We take a look at what we're working with. uh we go ahead and split it up into training and testing. Uh in this case, we're going to go ahead and scale it. Scale it means we're putting it between a value of minus one and one uh or someplace in that middle ground there. This way, if you have any huge set, you don't have this huge um setup. If we go back up to here where salary uh salary is 20,000 versus age 35, well, there's a good chance with a lot of the back-end math that 20,000 will skew the results and the estimated salary will have a higher impact than the age instead of balancing them out and letting the calculations weigh them properly. And finally, we get to actually create our naive bay model. Um, and then we're going to go ahead and import the Gazian naive bay. And the Gazian is is uh the most basic one. That's what we're looking at now. It turns out though, if you go to the SK um learn kit, uh they have a number of different ones you can pull in there. There's a um Bernoli. I'm not I've never used that one. Categorical um compliment. And here's our Gazian. Uh so there's a number of different options you can look at. Gazian when you come to the naive bays is the most commonly used. Uh so we're talking about the naive bays that's usually what people are talking about when they when they're pulling this in. And one of the nice things about the gazian if you go to their website um to sklearn the naive bay gazian there's a lot of cool features. One of them is you can do partial fit on here. Um that means if you have a huge amount of data, you don't have to process it all at on you once you can batch it into the Gazian uh NB model. And there's many other different things you can do with it as far as fitting the data and how you um manipulate it. We're just doing the basics. So we're going to go ahead and create our classifier. We're going to equal the Gausian NB. And then we're going to do a fit. We're going to fit our training data and our training solution. So, X-ray, Y train, and we'll go ahead and run this. Uh, it's going to tell us that it it ran the code right there. And now we have our trained classifier model. So, the next step is we need to go ahead and run a prediction. We're going to do our y predict equals the classifier.predict x test. So, here we fit the data and now we're going to go ahead and predict. And now we get to our confusion matrix. Uh so from the sklearn matrix metrics, you can import your confusion matrix just as saves you from doing all the simple math. It does it all for you. And then we'll go ahead and create our confusion metrics with the Y test and the Y predict. So we have our actual and we have our predicted value. And you can see from here, this is the chart we looked at. Here's predicted. So, true positive, false positive, false negative, true negative. And if we go ahead and run this, there we have it. 653725. And in this particular uh prediction, we had 65 uh or predicted the truth as far as a a purchase. They're going to make a purchase, and we guessed three wrong. And then we had 25 we predicted would not purchase, and seven of them did. So, there's our our confusion matrix. At this point, if you were uh with your shareholders or a board meeting, um you would start to hear some snoozing if they were looking at the numbers and you say, "Hey, here's my confusion mat uh matrix." So, let's go ahead and visualize the results. And we're going to pull from the map plot library colors import listed color map. Um, and this is actually my machine's going to throw an error because this is being um because of the way the setup is. I have a newer version on here than when they put together the demo. And we need our um x set and our y set, which is our x train and y train. And then we'll create our x1 x2. And we'll put that into a grid. Uh, and we set our x set minimum stop and our x set max stop. And if you come all the way over here, we're going to step 0.01. This is going to give us a nice line, uh, is what that's doing. And then we're going to plot the contour, uh, plot the x limit, plot the y limit, and put the scatter plot in there. And let's go ahead and run this. Uh, to be honest, when I'm doing these graphs, there's so many different ways to do that. There's so many different ways to put this code together. to show you what we're doing. It's uh a lot easier to pull up the graph and then go back up and explain it. So, the first thing we want to note here when we're looking at the data is this is the training set. And so, we have those who didn't make a purchase. We've drawn a nice area for that that's defined by the naive bay setup. And then we have those who did make a purchase, the green. And you can see that some of the green dots fall into the red area and some of the red dots fall into the green. So even our training set isn't going to be 100%. Uh we couldn't do that. And so we're looking at our different data coming down. Uh we can kind of arrange our x1 x2. So we have a nice plot going on. And we're going to create the um contour. That's that nice line that's drawn down the middle on here with the red green. Um that's where that's what this is doing right here with the reshape and notice that we had to uh do the t if you remember from numpy um if you did the numpy module um you end up with pairs you know x uh x1 x2 x1 x2 next row and so forth you have to flip it so it's all one row you have all your x1's and all your x2s. Um so this what we're kind of looking for right here on this setup. Uh, and then the scatter plot is of course um your scattered data across there. We're just going through all the points that puts these nice little dots onto our setup on here. And we have our estimated salary and our age. And then of course the dots are did they make a purchase or not. And just a quick note, this is kind of funny. You can see up here where it says X set Y set equals uh X-ray Y train, which seems kind of a little weird to do. Um, this is because this was probably originally a definition. Uh, so it's its own module that could be called over and over again. And which is really a good way to do it because the next thing we're going to want to do is do the exact same thing, but we're going to visualize the test set results. Uh, that way we can see what happened with our test group, our 25%. And you can see down here we have um the test set uh and it if you look at the two graphs next to each other, this one obviously has um 75% of the data. So, it's going to show a lot more. This is only 25% of the data. You can see that there's a number that are kind of on the edge as to whether they could guess by age and income they're going to make a purchase or not. Um, but that said, it still is pretty clear. It's pretty good as far as how much the estimate is and how good it does. Now, graphs are really effective for showing people what's going on, but you also need to have the numbers. And so we're going to do from sklearn we're going to import metrics and then we're going to print our metrics classification report from the Y test and the Y predict. And you can see here we have precision uh precision of zeros is 90. There's our recall 96. We have an F1 score and a support. And we have our precision, the recall on getting it right. Uh, and then we can do our accuracy, the macro average, and the weighted average. Uh, so you can see it it pulls in pretty good as far as um how accurate it is. You could say it's going to be about 90% is going to guess correctly um that it that they're not going to purchase. And we had an 89% chance that they are going to purchase. Um, and then the other numbers as you get down have a little bit different meaning, but it's pretty straightforward on here. Here's our accuracy and here's our micro average and the weighted average and everything else you might need. And if you forgot the exact definition of accuracy, it is the true positive, true negative over all of the different setups. Precision is your true positive over all positives, true and false. And recall is a true positive over true positive plus false negative. And we can just real quick flip back there so you can see those numbers on here. Uh here's our precision. Here's our recall. And here's our accuracy on this. My name is Richard Kersner. I'm with the SimplyLearn team. What's in it for you? Well, today we're going to cover what is a neural network, what can neural networks do, how does a neural network work, types of neural networks, and then we're going to jump into a use case to classify between the photos of dogs and cats. And we'll do that on the KAS with the TensorFlow in the back, but it's a Python script. So, that's always my favorite part is when we dive into the actual script. So, what is a neural network? So, hi guys. I heard you want to know what a neural network is. Here we have uh looks like he just went shopping at a red tag sale. My robots back. So as a matter of fact, you have been using neural network on a daily basis. In today's world, it's just amazing how much we use our new technology. We're not even aware of it. When you ask your mobile assistant to perform a search for you, you know, like saying you're Google or Siri or whoever you use, Amazon Web self-driving cars. So that's the newest thing coming out. They're just now trying to make those legal in different states in the US and around the world. Even in the UK, they now have a self-driving cars going up and down the street. It's pretty amazing. These are all neural network driven. Computer games use it. So, a lot of computer games are driven by neural networks in the back end as part of the game system and how it adjusts to the players. And it's also used in processing the map images on your phone. So, every time you do a navigation someplace and it opens it up, they now use neural networks to help you find the quickest way to get there. Neural network. A neural network is a system or hardware that is designed to operate like a human brain. In today's development, this is so important to understand because we don't have anything else to compare it to. I'm sure someday in the future the computer will redefine or the neural network or the AI artificial intelligence will redefine what these mean. But as far as we can today's world in today's commercial development, we have to compare it to what humans do. So it's we want to compare and how it operates to a human brain and how it solves problems like a human does. What can a neural network do? And really we're just going to dive in deeper to we just covered and look at other examples. So what can a neural network do? Well, let's list out the things neural networks can do for you. Translate text. Boy, we got Google Translate and Microsoft has their own translate. They have some really cool. They actually have an earpiece. It's supposed to start translating as you talk. What a cool technology. What a cool time to live. Identify faces. Can you imagine all the uses for facial identification? In the case of our uh sample or our code that we're going to look at later, we'll be identifying dogs and cats. So, not quite as detailed as understanding whose face belongs to who. I I'm waiting for the Google glasses to come out so I can see who's who and the identify faces as I'm walking around. Have a little name tag over them. Not out there yet, but boy, we are close. We can identify the faces and they have all kinds of technologies to bring that information back to us. Recognize speech goes along with the translate text. So now as you're talking into your assistant, it can use that to do commands, turn lights on, all kinds of things you can do with recognizing speech. Read handwritten text. They're starting to translate all these old text documents that they've had in storage instead of doing it individually where somebody's going through each text by themsel in a room. Picture like an old Raiders of the Lost Arc theme where he's in the back, you know, archaeologist studying the text. Now it's fed into a computer. They take a picture. They even use neural networks to take a scroll that is so messed up that they can't undo the scroll and they X-ray it. And then they use that X-ray to translate the text off of it without ever opening the scroll. I mean, just way cool stuff they're starting to do with all this. And of course, control robots. What would be a neural network without bringing in the robots? And we have our own favorite robot in the middle who goes to our red tag cell and goes shopping for us. So, you know, these are just a few of the wonderful things that neural networks are being applied to. It's such an infant stage technology. What a wonderful time to jump in. And there are a lot of other things it goes into. I mean, we could spend just forever talking about all the different applications from business to whatever you can even imagine. They're now applying neural networks to help us understand. So, now we talked a little bit about all the cool things you can do with a neural network. Let's dive in and say, how does a neural network work? So now we've come far enough to understand how neural network works. Let's go ahead and walk through this in a nice graphical representation. They usually describe a neural network as having different layers. And you'll see that we've identified a green layer, an orange layer, and a red layer. The green layer is the input. So you have your data coming in. It picks up the input signals and passes them to the next layer. The next layer does all kinds of calculations and feature extraction. It's called the hidden layer. A lot of times there's more than one hidden layer. We're only showing one in this uh picture, but we'll show you how it looks like in a more detail in a little bit. And then finally, we have an output layer. This layer delivers the final result. So the only two things we see is the input layer and the output layer. Now let's make use of this neural network and see how it works. Wonder how traffic cameras identify vehicles registration plate on the road to detect speeding vehicles and those breaking the law? They got me going through a red light the other day. Well, last month. That's like the horrible thing. They send you this picture of you and all your information because they pulled it up off of your license plate and your picture. I shouldn't have gone through the red light. So, here we are and we have an image of a car and you can see the license plates on there. So, let's consider the image of this vehicle and find out what's on the number plate. The picture itself is 28x 28 pixels and the image is fed as an input to identify the registration plate. Each neuron has a number called activation that represents the grayscale value of the corresponding pixel range. And we range it from zero to one. One for a white pixel and zero for a black pixel. And you can see down here we have an example where one of the pixels is registered as like 082. Meaning it's probably pretty dark. Each neuron is lit up when its activation is close to one. So as we get closer to black on white, we can really start seeing the details in there. And you can see again the pixel shows us one up there. or it's like part of the car and so it lights up. So pixels in the form of arrays are fed to the input layer. And so we see here the pixels of a car image fed as an input and you're going to see that the input layer which is green is one dimension while our image is two-dimension. Now when we look at our setup that we're programming in Python, it has a cool feature that automatically does the work for us. If you're working with an older neural network pattern package, you then convert each one of those rows so it's all one array. So you'd have like row one and then just tack row two onto the end. You can almost feed the image directly into some of these neural networks. The key is though is that if you're using a 28x 28 and you get a picture of this 30x30, shrink the 30x30 down to fit the 28x 28. So you can't increase the number of input in this case green dots. It's very important to remember when you work on neural networks. And let's name the inputs x1, x2, x3, respectively. So each one of those represents one of the pixels coming in. And the input layer passes it to the hidden layer. And you can see here we now have two hidden layers in this image in the orange. And each one of those pixels connects to each one of those hidden layers. And the interconnections are assigned weights at random. So they get these random weights that come through. If x1 lights up, then it's going to be x1 times this weight going into the hidden layer. And we sum those weights. The weights are multiplied with the input signal and a bias is added to all of them. So as you can see here we have X1 comes in and it actually goes to all the different hidden layer nodes or in this case uh whatever you want to call them network setup the orange dots and so you take the value of X1 you multiply it by the weight for the next hidden layer. So X1 goes to hidden layer 1 X1 goes to hidden layer two X1 goes hidden layer 1 node two hidden layer one node three and so on. And the bias a lot of times they just put the bias in as like another green dot or another orange dot and they give the bias a value one and then all the weights go in from the bias into the next node. So the bias can change. We always just remember that you need to have that bias in there. There's things that can be done with it. Generally most of packages out there control that for you so you don't have to worry about figuring out what the bias is. But if you ever dive deep into neural networks, you got to remember there's a bias or the answer won't come out correctly. The weighted sum of the input is fed as an input to the activation function to decide which nodes to fire. And for feature extraction, as a signal flows within the hidden layers, the weighted sum of inputs is calculated and is fed to the activation function in each layer to decide which nodes to fire. So here's our feature extraction of the number plate. And you can see these are still hidden nodes in the middle. And this becomes important. We're going to take a little detour here and look at the activation function. So, we're going to dive just a little bit into the math so you can start to understand where some of the games go on when you're playing with neural networks in your programming. So, let's look at the different activation functions before we move ahead. Here's our friendly red tag shopping robot. And so, one is a sigmoid function. And the sigmoid function which is 1 over 1 + e to the minus x takes the x value and you can see where it generates almost a zero and almost a one with a very small area in the middle where it crosses over and we can use that value to feed into another function. So if it's really uncertain it might have a 0.1 or 2 or 3 but for the most part it's going to be really close to one and really close to this case zero. 0 to one the threshold function. So if you don't want to worry about the uncertainty in the middle, you just say, "Oh, if x is greater than or equal to zero, if not, then x is zero." So it's either zero or one. Really straightforward. There's no in between in the middle. And then you have the what they call the reel relu function. And you can see here where it puts out the value, but then it says, well, if it's over one, it's going to be one. And if it's uh less than zero, it's zero. So it kind of just deadends it on those two ends, but allows all the values in the middle. And again this like the sigmoid function allows that information to go to the next level. So it might be important to know if it's a 0.1 or a minus.1. The next hidden layer might pick that up and say oh this piece of information is uncertain or this value has a very low certainty to it. And then the hyperbolic tangent function and you can see here it's a 1 - e to the -2x over 1 + e the - 2x. And it's very much along the same theme. A little bit different in here in that it goes between minus one and one. So you'll see some of these they go zero to one, but this one goes minus one to one. And if it's less than zero, it's, you know, it doesn't fire and if it's over zero, it fires. And it also still puts out a value. So you still have a value you can get off of that just like you can with the sigmoid function and the relu function. Very similar in use. And I believe the originally used to be everything was done in the sigmoid function. That was the most uh commonly used. And now they just kind of use more the reloo function. The reason is one it processes faster because you already have the value and you don't have to add another compute the 1 / 1 + e to the minus x for each hidden node and the data coming off works pretty good as far as putting it into the next level if you want to know just how close it is to zero. How close is it not to functioning? You know is it minus.1 minus.2 usually they're float values you get like minus point minus.00138 or something. So, you know, important information, but the Reu is most commonly used these days as far as the setup we're using. But you'll also see the sigmoid function very commonly used also. Now that you know what an activation function is, let's get back to the neural network. So, finally, the model would predict the outcome of applying a suitable activation function to the output layer. So, we go in here, we look at this, we have the optical character recognition, OCR, is used on the images to convert it into a text in order to identify what's written on the plate. And as it comes out, you'll see the red node. And the red node might actually represent just the letter A. So there's usually a lot of outputs when you're doing text identification. We're not going to show that on here, but you might have it even in the order. It might be what order the license plates in. So you might have ABCDE E FG, you know, all the alphabet plus the numbers. And you might have the 1 2 3 4 5 6 7 8 9 10 places. So it's a very large array that comes out. It's not a small amount of uh, you know, we show three dots coming in, eight hidden layer nodes, you know, two sets of four. We just show one red coming out. A lot of times this is uh, you know, 28 * 28. If you did 30 * 30, that's, you know, 900 nodes. So 28 is a little bit less than that uh, just on the input. And so you can imagine the hidden layer is just as big. Each hidden layer is just as big if not bigger. Then the output is going to be there's so many digits. You know, it's a lot. There's it's a huge amount of input and output. But we're only showing you just, you know, it' be hard to show in one picture. And so it comes up and this is what it finally gets out in the output as it identifies a number on the plate. And in this case, we have 08-d3858. Error in the output is back propagated through the network and weights are adjusted to minimize the error rate. This is calculated by a cost function. When we're training our data, this is what's used and we'll look at that in the code and we do the data training. So we have stuff we know the answer to and then we put the information through and it says yes that was correct or no because remember we randomly set all the weights to begin with and if it's wrong we take that error how far off are you you know are you off by is it if it was like minus one you're just a little bit off if it's like minus300 was your output remember when we're looking at those different options you know hyperbolic or whatever and we're looking at the could doesn't have an limit on top or bottom it actually just generates it's a number. So if it's way off, you have to adjust those weights a lot. But if it's pretty close, you might adjust the weights just a little bit. And you keep adjusting the weights until they fit all the different training models you put in. So you might have 500 training models and those weights will adjust using the back propagation. It sends the error backward. The output is compared with the original result and multiple iterations are done to get the maximum accuracy. So, not only does it look at each one, but it goes through it and just keeps cycling through these the data making small changes in the network until it gets the right answers. With every iteration, the weights at every interconnection are adjusted based on the error. We're not going to dive into that math because it is a differential equation and it gets a little complicated, but I will talk a little bit about some of the different options they have when we look at the code. So, we've explored a neural network. Let's look at the different types of artificial neural networks. And this is like the biggest area growing is how these all come together. Let's see the different types of neural network. And again, we're comparing this to human learning. So here's a human brain. I feel sorry for that poor guy. So we have a feed for forward neural network. Simplest form of a they call it a ann a neural network. Data travels only in one direction input to output. This is what we just looked at. So as the data comes in, all the weights are added. It goes to the hidden layer. All the weights are added. It goes to the next hidden layer. All the weights are added and it goes to the output. The only time you use the reverse propagation is to train it. So when you actually use it, it's very fast. When you're training it, it takes a while because it has to iterate through all your training data. And you start getting into big data because you can train these with a huge amount of data. The more data you put in, the better trained they get. The applications vision and speech recognition actually they're pretty much everything we talked about a lot of almost all of them use this form of neural network at some level radio basis function neural network this model classifies a data point based on its distance from a center point. What that means is that you might not have training data. So you want to group things together and you create central points and it looks for all the things you know some of these things are just like the other. If you've ever watched the Sesame Street as a kid, that dates me. So, it brings things together and this is a great way if you don't have the right training model, you can start finding things that are connected you might not have noticed before. Applications power restoration systems. They try to figure out what's connected and then based on that they can fix the problem if you have a huge power system. and self-organizing neural network vectors of random dimensions are input to discrete map comprised of neurons. So they basically find a way to draw they call them they say dimensions or vectors or planes because they actually chop the data in one dimension, two dimension, three dimension, four, five, six. They keep adding dimensions and finding ways to separate the data and connect different data pieces together. Applications used to recognize patterns in data like in medical analysis. The hidden layer saves its output to be used for future prediction. Recurrent neural networks. So the hidden layers remember its output from last time and that becomes part of its new input. You might use that especially in robotics or flying a drone. You want to know what your last change was and how fast it was going to help predict what your next change you need to make is to get to where the drone wants to go. Applications text to speech conversation model. So, you know, I talked about drones, but you know, just identifying on Lexus or Google Assistant or any of these, they're starting to add in I'd like to play a song on my Pandora, and I'd like it to be at volume 90%. So, you now can add different things in there, and it connects them together. The input features are taken in batches like a filter. This allows a network to remember an image in parts. Convolution neural network. today's world in photo identification and taking apart photos and trying to you know have you ever seen that on Google where you have five people together this is the kind of thing separates all those people so then it can do a face recognition on each person applications used in signal and image processing in this case I use facial images or Google picture images as one of the options modular neural network it has a collection of different neural networks working together to get the output so wow we just went through all these different types of neural networks. And the final one is to put multiple neural networks together. And I mentioned that a little bit when we separated people in a larger photo and individuals in the photo and then do the facial recognition on each person. So one network is used to separate them and the next network is then used to figure out who they are and do the facial recognition. Applications still undergoing research. This is the cutting edge. you hear the term pipeline and there's actual in Python code and in almost all the different neural network setups out there they now have a pipeline feature usually and it just means you take the data from one neural network and maybe another neural network or you put it into the next neural network and then you take three or four other neural networks and feed them into another one. So how we connect the neural networks is really just cutting edge and it's so experimental. I mean it's almost creative in its nature. There's not really a science to it because each specific domain has different things it's looking at. So if you're in the banking domain, it's going to be different than the medical domain, than the automatic car domain. And suddenly figuring out how those all fit together is just a lot of fun and really cool. So we have our types of artificial neural network. We have our feed forward neural network. We have a radial basis function neural network. We have our Cohen self-organizing neural network, recurrent neural network, convolution neural network, and modular neural network where it brings them all together. And no, the colors on the brain do not match what your brain actually does, but they do bring it out that most of these were developed by understanding how humans learn. And as we understand more and more of how humans learn, we can build something in the computer industry to mimic that, to reflect that. And that's how these were developed. So exciting part, use case problem statement. So this is where we jump in. This is my favorite part. Let's use the system to identify between a cat and a dog. If you remember correctly, I said we're going to do some Python code. And you can see over here, my hair is kind of sticking up over the computer. Cup of coffee on one side and a little bit of old school. A pencil and a pen on the other side. Yeah, most people now take notes. I love the stickies on the computer. That's great. That's that is my computer. I have sticky notes on my computer in different colors. So, not too far from uh today's programmer. So, the problem is is we want to classify photos of cats and dogs using a neural network. And you can see over here we have quite a variety of dogs in the pictures and cats and you know just sorting out it is a cat is pretty amazing. And why would anybody want to even know the difference between a cat and a dog? Okay, you know why? Well, I have a cat door. It'd be kind of fun that instead of it identifying, instead of having like a little collar with a magnet on it, which is what my cat has, the door would be able to see, oh, that's the cat. That's our cat coming in. Oh, that's the dog. We have a dog, too. That's a dog I want to let in. Maybe I don't want to let this other animal in cuz it's a raccoon. So, you can see where you could take this one step further and actually apply this. You could actually start a little startup company idea, self-identifying door. So, this use case will be implemented on Python. I am actually in Python 3.6. It's always nice to tell people the version of Python because that does affect sometimes which modules you load and everything. And we're going to start by importing the required packages. I told you we're going to do this in Kass. So we're going to import from KAS models sequential from the Kass layers conversion 2D or COV2D max pooling 2D flatten and dense. And we'll talk about what each one of these do in just a second. But before we do that, let's talk a little bit about the environment we're going to work in. And uh you know, in fact, let me go ahead and open a uh the website, KASS's website, so we can learn a little bit more about KASS. So here we are on the Kurass website, and it's uh ke.io. That's the official website for KAS. And the first thing you'll notice is that Kurass runs on top of either TensorFlow, CNTK, and I think it's pronounced Thano or Theo. What's important on here is that TensorFlow and the same is true for all these, but TensorFlow is probably one of the most widely used currently packages out there with the KAS. And of course, you know, tomorrow this is all going to change. It's all going to disappear and they'll have something new out there. So, make sure when you're learning this code that you understand what's going on and also know the code. I mean, look, when you look at the code, it's not as complicated once you understand what's going on. The code itself is pretty straightforward. And the reason we like KAS and the reason that people are jumping on it right now and it's such a big deal is if we come down here, let me just scroll down a little bit. They talk about user friendliness, modularity, easy extensibility, work with Python. Python's a big one because a lot of people in data science now use Python, although you can actually access Kass other ways. Is if we continue down here is layers. And this is where it gets really cool. When we're working with KASS, you just add layers on. Remember those hidden layers we were talking about and we talked about the releuation? You can see right here. Let me just up that a little bit in size. There we go. That's big. I can add in an eelu layer and then I can add in a softmax layer in the next instance. We didn't talk about softmax. So you can do each layer separate. Now if I'm working in some of the other kits I use, I take that and I have one setup and then I feed the output into the next one. This one I can just add hidden layer after hidden layer with the different information in it which makes it very powerful and very fast to spin up and try different setups and see how they work with the data you're working on. And we'll dig a little bit deeper in here. And a lot of this is very much the same. So when we get to that part, I'll point that out to you also. Now just a quick side note, I'm using Anaconda with Python in it. And I went ahead and created my own package and I called it the Kass Python 36 because I'm in Python 36. Anaconda is cool that You can create different environments really easily. If you're doing a lot of different experimenting with these different packages, probably want to create your own environment in there. And the first thing, as you can see right here, there's a lot of dependencies. A lot of these you should recognize by now if you've done any of these videos. If not, kudos for you for jumping in today. PIP, install, numpy, sci, the scikitlearn, pillow, and h5py are both needed for the tensorflow and then putting the kass on there. And then you'll see here uh and pip is just a standard installer that you use with Python. You'll see here that we did pip install TensorFlow since we're going to do KAS on top of TensorFlow. And then pip install and I went ahead and used the GitHub. So git plus Git. And you'll see here github.com. This is one of their releases, one of the most current release on there that goes on top of TensorFlow. And you can look up these instructions pretty much anywhere. This is for doing it on Anaconda. Certainly you'd want to install these if you're doing it in Iuntu server setup. you you'd want to get I don't think you need the H5 py and I abuntu but you do need the rest in there because they are dependencies in there and it's pretty straightforward and that's actually in some of the instructions they have on their website so you don't have to necessarily go through this just remember their website on there and then when I'm under my uh Anaconda navigator which I like you'll see where I have environments and on the bottom I created a new environment and I called it KAS Python 36 just to separate everything you can say I have Python 3.5 and Python 36 used to have a bunch of other ones, but it kind of cleaned house recently. And of course, once I go in here, I can launch my Jupyter notebook. Making sure I'm using the right environment that I just set up. This, of course, opens up my um in this case, I'm using uh Google Chrome. And in here, I could go and just create a new document in here. And this is all in your um browser window when you use the Anaconda. Do you have to use Anaconda and Jupyter Notebook? No. You can use any kind of Python editor, whatever setup you're comfortable with and whatever you're doing in there. So, let's go ahead and go in here and paste the code in. And we're importing a number of different settings in here. We have import sequential. That's under the models because that's the model we're going to use as far as our neural network. And then we have layers and we have conversion 2D, max pooling 2D, flatten dense. And you can actually just kind of guess at what these do. We're talking we're working in a 2D photograph. And if you remember correctly, I talked about how the actual input layer is a single array. It's not in two dimensions. It's one dimension. All these do is these are tools to help flatten the image. So, it takes a two-dimensional image and then it creates its own proper setup. You don't have to worry about any of that. You don't have to do anything special with the photograph. You let the carass do it. And we're going to run this. And you'll see right here they have some stuff that is going to be depreciated and changed cuz that's what it does. Everything's being changed as we go. You don't have to worry about that too much. If you have warnings, if you run it a second time, the warning will disappear. And this has just imported these packages for us to use. Jupiter's nice about this that you can do each thing step by step. And I'll go ahead and also zoom in there. A little control plus. That's one of the nice things about being in a browser environment. So, here we are back. Another sip of coffee. If you're familiar with my other videos, you notice I'm always sipping coffee. I always have a in my case latte next to me and espresso. So the next step is to go ahead and initialize. We're going to call it the CNN or classifier neural network. And the reason we call it a classifier is because it's going to classify it between two things. It's going to be cat or dog. So when you're doing classification, you're picking specific objects. You're specific. It's a true or false. Yes, no. It is something or it's not. So first thing we're going to create our classifier and it's going to equal sequential. So their sequential setup is the classifier. That's the actual model we're using. That's the neural network. So we call it a classifier. And uh the next step is to add in our convolution. And let me just do a uh let me shrink that down in size so you can see the whole line. And let's talk a little bit about what's going on here. I have my classifier and I add something. What am I adding? Well, I'm adding my first layer. This first layer we're adding in is probably the one that takes the most work to make sure you have it set correct. And the reason I say that is this is your actual input. And we're going to jump here to the part that says input shape equals 64x 64x3. What does that mean? Well, that means that our pictures coming in and there's these pictures, remember we had like the picture of the car was 128x 128 pixels. Well, this one is 64x 64 pixels. And each pixel has three values. That's where these numbers come from. And it is so important that this matches. I mentioned a little bit that if you have like a larger picture, you have to reformat it to fit this shape. If it comes in as something larger, there's no input notes. There's no input neural network there that will handle that extra space. So, you have to reshape your data to fit in here. Now, the first layer is the most important because after that, KAS knows what your shape is coming in here and it knows what's coming out and so that really sets the stage. Most important thing is that input shape matches your data coming in. And you'll get a lot of errors if it doesn't. You'll go through there and picture number 55 doesn't match it correctly. And guess what it does? It usually gives you an error. And then the activation, if you remember, we talked about the different activations on here. We're using the reelu model. Like I said, that is the most commonly used now because one, it's fast. Doesn't have the added calculations in it. It just says here's the value coming out based on the weights and the value going in. And um from there, you know, it's uh if it's over one, then it's good or over zero, it's good. If it's under zero, then it's considered not active. And then we have this conversion 2D. What the heck is conversion 2D? I'm not going to go into too much detail in this because this has a couple of things it's doing in here, a little bit more in-depth than we're ready to cover in this tutorial. But this is used to convert from the photo cuz we have 64x 64x3 and we're just converting it to two-dimensional kind of setup. So it's very aware that this is a photograph and that different pieces are next to each other. And then we're going to add in uh a second convolutional layer. That's what the cov stands for 2D. So it's these are hidden layers. So we have our input layer and our two hidden layers and they are two-dimensional because we're dealing with a two-dimensional photograph. And you'll see down here that on the last one, we add a max pooling 2D and we put a pool size equals 22. And so what this is is that as you get to the end of these layers, one of the things you always want to think of is what they call mapping and then reducing. Wonderful terminology from the big data. We're mapping this data through all these layers. And now we want to reduce it to only two sets. In this case, it's already in two sets because it's a 2D photograph. But we had, you know, two dimensions by we actually have 64x 64 by3. So now we're just getting it down to a 2x two. Just the two-dimension two-dimensional instead of having the third dimension of colors. And we'll go ahead and run these. We're not really seeing anything in our run script because we're just setting up. This is all set up. And this is where you start playing cuz maybe you'll add a different layer in here to do something else to see how it works and see what your output is. That's what makes KAS so nice is I can with just a couple flips of code put in a whole new layer that does a whole new processing and see whether that improves my run or makes it worse. And finally, we're going to do the final setup, which is to flatten classifier, add a flatten setup. And then we're going to also add a layer, a dense layer, and then we're going to add in another dense layer. And then we're going to build it. We're going to compile this whole thing together. So, let's flip over and see what that looks like. And we've even numbered them for you. So, we're going to do the flattening. And flatten is exactly what it sounds like. We've been working in a two-dimensional array of picture, which actually is in three dimensions because of the pixels. The pixels have a whole another dimension to it of three different values. And we've kind of resized those down to 2x two. But now we're just going to flatten it. I don't want to have multiple dimensions being worked on by tensor and by kas. I want just a single array. So, it's flattened out. And then step four, full connection. So we add in our final two layers. And you could actually do all kinds of things with this. You could actually leave out this some of these layers and play with them. You do need to flatten it. That's very important. Then we want to use the dents again. We're taking this and we're taking whatever came into it. So once we take all those different the two dimensions or three dimensions as they are and we flatten it to one dimension, we want to take that and we're going to pull it into units of 128. They got that. You're going to say where did they get 128 from? You could actually play with that number and get all kinds of weird results. But in this case, we took the 64 + 64 is 128. You could probably even do this with 64 or 32. Usually you want to keep it in the same multiple whatever the data shape you're already using is in. And we're using the activation the re lu just like we did before. And then we finally filter all that into a single output. And it has how many units? One. Why? Because we want to know whether true or false. It's either a dog or a