Regression Mathematics

Key Takeaways

regression final answer hello world it's Suraj and everyone in the entire world needs to understand how regression works it's a data science technique that can quickly give us super powers of insight from a massive sometimes unintelligible dataset using regression we can predict the optimal way to allocate funds in our organization to increase sales predict demand for a product understand the effects of certain treatments on a patient understand why certain factors contribute to poverty or economic prosperity I could literally go on for hours ain't nobody got time for that whether it's in business research or even lifestyle design regression can give us the answers we need from data to improve our lives and the lives of others now there are a lot of different types of regression models in the wild including logistic and stepwise in elastic net it can be overwhelming to try and understand all of them from a single video so we're gonna focus on just a few of the most important ones for our problem we'll play the role of a newly hired data analyst at a corporation called engine using the data set of our marketing expenses and ticket sales for our newly opened Jurassic Park we'll use a regression model to predict how to optimally increase ticket sales next month the data finds away identifying whether or not the correlation between two or more variables represent a causal relationship is rarely easy using human intuition alone but if we could figure that out in a mathematical way it would be so much easier to make business decisions quickly so we can spend the rest of our time playing for tonight to help answer these types of questions data scientists use regression models regressions are used to quantify the strength of the relationship between one variable and other variables that are thought to explain it these days running hundreds of thousands of regressions has become extremely simple in fact we don't even need to know how to code that's right Microsoft formerly run by YouTube sensation Bill Gates has built regression functionality into its excel product and almost all of the major tech companies have some form of drag-and-drop interface that lets us immediately apply regression analysis to a data set we upload but using a programming language like Python can help us intuitively understand the mathematics that's happening behind the scenes and if we understand the mathematics it'll help us build an intuition around when to use which model for whatever task we have so let's start with the first type of regression the simple linear regression this is a model that can show the relationship between two variables more specifically it shows how the variation in the dependent variables can be captured by a change in the independent variable in a business context the dependent variables can also be called the predictor for sales of a product performance pricing or risk the independent variable also called the explanatory variable explains the influence of the dependent variable a simple linear regression model is linear because all the terms in the model are either a constant value or a parameter multiplied by an independent variable the core idea is to figure out this model which when plotted is the line of best fit for the data the best fit line is the one where the total prediction error of all the data points are as small as possible we can consider the error as the distance between the data point and the regression line so in our data sets after we import pandas for data manipulation numpy for matrix math and plot Li for data visualization we can visualize our data first to see that we're spending money on TV radio and newspaper ads each week and the amount of sales of our product changes every week as well yes I said newspaper remember this is the 90s let's say we want to find the relationship between just TV ad money and sales thus will want to use a simple linear regression model we can represent this as the model y equals MX plus B the values of both M and B should be optimal such that the error is as small as possible when plotting there are several different metrics we can use to compute the error so let's use a popular one called the sum of squared error this simply finds the difference between the actual output and the predicted output squares it adds them all up and that's our final error value we then use this error to update the values for B and M using a technique called gradient descent rather than explain how that works I'm going to link you to my video aptly titled linear regression using gradient descent in the video description when we perform this simple linear regression will find that there does seem to be a relationship between these two variables the more we spend on TV ads the higher our sales are makes sense but we probably want a more holistic view of our data meaning we want to take into account the other variables as well like the newspaper and radio ads so we can see how they all affect sales if we try to plot just one other feature the line we have now becomes a plane we're now in 3d the plane is the function that expresses Y as a function of the other two variables this is the genisis of the multiple linear regression model there's now more than one input variable used to predict the Y value a model with two input variables is a longer equation than one expressed with one we can take it a step further and take into account all three input variables and create an equation for that as well alas if we do this we are now limited in our capabilities in terms of what we can visualize in a graph since we are now dealing with more than three dimensions but that's the wild world of data many datasets have tens if not thousands of dimensions if we again use gradient descent to find the equation that best fits our data will receive a model we can interpret specifically the coefficients in the equation we can correlate each learned coefficient with a different feature if all the other predictors are held constant and we change any of these values we can tell what the corresponding increase in the amount of sales would be but which coefficients are the most significant are they all equally significant does this equation explain the meaning of life here we turn to the art of model evaluation notice that all the coefficients are greater than zero that means all the variables have an impact on the sales hmm before we keep evaluating this model any further I have a suspicion we could use a better model my intuition as a digital detective is that TV ads far outweigh the other features in terms of significance and we'll need to prove that our a linear regression model fits the data relatively well I mean it's not extremely off if it was we want to try to instead use a nonlinear regression a nonlinear regression model is one where not all the terms in the model are either a constant value or a parameter multiplied by an independent variable both linear and nonlinear models can fit curvature in data so that's the main factor to consider but since we're doing pretty well so far let's not go there right now it's also important to note that both single and multiple regression models can be either linear or nonlinear these four terms are all different ways of defining regressions before we kick back relax and tweet that we've secured funding for Tesla at 420 we want to try two more types of linear regression models Ridge and lasso regressions any linear or multiple regression model will be sub optimal when there is a high collinearity amongst the feature variables : e era t is a condition in which some of the feature variables are highly correlated its sub optimal because Co linearity tends to interfere in determining the precise effect of each feature variable there are a few different ways of mathematically evaluating this but we'll get to model evaluation techniques later on right now let's just try out Ridge regression to see how it works Ridge regression alleviates collinearity amongst the regression predictor variables in a model again collinear T is where one of the feature variables in a multiple regression model can be linearly predicted from the others with a substantial degree of accuracy if the feature variables are very much correlated the final regression model is going to cause overfitting meaning our model won't be generalized enough to new data points it'd be perfect but only for the existing data points we don't want that so to solve this issue Ridge regression adds a small squared bias factor to the variables this squared bias factor pulls the feature variable coefficients away from this rigidness by introducing a small amount of bias into the model it reduces the variance which means the difference between predicted data points and the mean of Y immensely this trade-off of variance and bias will produce much more useful coefficient estimates when collinearity exists for Ridge regression will introduce grid search CV this is going to allow us to automatically perform cross-validation which will assess how the results of our model will generalize to an independent data set with a range of different parameters in order to find the optimal value of alpha then we can find the best parameter and the best mean squared error using the built-in attributes Thank You scikit-learn our optimal value of alpha and MSE are both slightly better than the ones we got from our multiple regression model this is good but I think we can do even better enter lasso regression lasso regression is similar to Ridge regression in that they both have the same premise in lasso regression we're again adding a bias term to the regression function so that we can reduce the negative Effects of collinearity and consequently the models variance but instead of using a squared bias like Ridge regression does lasso regression uses a literal lasso just kidding I had to do that at least once it instead uses an absolute value bias and this difference has a huge impact on the bias-variance tradeoff lasso overcomes the disadvantage of Ridge regression by not only punishing high values of the coefficients but actually setting them to zero if they aren't relevant in this way we might end up with less features included in our model than we started with but this can be an advantage when we implement lasso it looks similar to Ridge and our resulting alpha and MSE are the best scores of any model we've used looks like lasso is the way to go using this model we can give it values for how much we will spend on different advertising mediums and it will predict the amount of sales we'll make which lets us much more efficiently allocate funds towards the medium we want to use saving us money and thus increasing our revenue luckily there are only three points to remember from this video a regression model is simple if it has only one input and one output but it's multiple if it has multiple inputs and one output a regression model is linear when all the terms in the model are either a constant value or a parameter multiplied by an independent variable else it's nonlinear lastly if a data set has a high collinearity meaning that inputs are highly correlated we can use either Ridge or lasso regression to help ensure more accurate estimates of the regression coefficients what do you want to use a regression model to do next let me know in the comments section and please subscribe for more technology videos for now I've got to find a lasso so thanks for watching