Statistical Learning: 3.Py Interactions, Qualitative Predictors and Other Details I 2023

Key Takeaways

okay so now we're going to go to some more complicated examples and examples where you'll see the power of of model speak thanks Trev okay so a uh you know we've seen how to have design matrices with just a list of column names you know a another common uh modeling tool is an interaction so the way we specify interactions with model spec is just uh instead of just having the column name we put this is a a tupple a twole of the two column names so this tells model spec make me an interaction of the variable lstat and H um so once it's knows how to create the variable lstat and the variable age it can make the interaction between L stat and H okay so let's just uh fit this model and here we have a coefficient it's just another effect in the regression model and in this case because both lat and age are quantitative variables it just multiplies those two together entrywise yes we'll see in a moment you know there are Reg ex problems where not all variables are quantitative in the Boston Housing data actually all of them are but um we'll see uh many are categorical and then uh the interaction would look different but yeah okay so um another example you know another modeling tool that people might use is to rather than fit just a linear effect for a variable might put a polinomial effect so um we've there's a a function poly that allows us to specify um you know polinomial effect in some feature here we'll put a quadratic effect for lstat and a linear effect for age again the model is specified the same way it's a list of terms it's just that this term here poly has actually two columns corresponding to it uh let's fit the model and summarize and we can see that uh the two columns each one gets a separate coefficient estimate and indicated by this is the First Column corresponding and the second column because of course the two degree polinomial has got a quadratic term and a linear term that's right yeah um and if we wanted to actually we could uh uh you could we could consider an interaction between the polinomial and age uh but we'll leave that for for another day um okay so now this example here we've gone from just a linear term for lstat to a quadratic we might be interested in deciding whether there was really um a significant Improvement in fit going from linear to quadratic we can actually see here that the P value is small but there's another method that's commonly used that is the Inova method to compare two different fitted models so this Anova LM function from the stats models what it will do is um Carry Out uh a you know a test an F test comparing two nested models uh and we can use that to decide whether um one model is significantly better than the other this F statistic here 177 that's huge that's huge and it's also should be about 13.3 squared right that's that t statistic should correspond to that um though maybe not exactly I shouldn't I always says it actually no it should in this case that should be yes you know okay um let's see so part of the section that we skipped looked at residuals uh of our early regression model and noted a trend in the residuals plotted as a function of fitted value in this plot actually we can see that uh that trend has been removed um but we didn't see the first plot so let's not dwell too much on it the last example we're going to of model spec we'll we'll use is a um is qualitative predictors or categorical variables so uh in the another data set in the islp package the car seats data tries to predict the sales of car seets as a function of different features um so different features like uh you know the uh competitor's price so forget this maybe a chain like Walmart and maybe we'll compare it competitor's price like Target uh maybe this is income of the uh typical income in the neighborhood of the of that particular store Etc one of these variables is shelf location this is sort of where the car Siege is placed in the in the sh I think low medium high are the levels and so that's you know we could encode low medium high as 0 1 and two but those are kind of arbitrary so this is really a categorical variable I see right um and uh if we look at the column shelf Lo in the uh in the data frame we'll see whoops that's not right uh we'll see that it's been recognized as a category by pendas and so model spec will then uh if we use shelf lo as a um as a predictor it will treat as categorical so here we have we fit a rather complicated model um we first found all the features besides the response sales and we'll add all those features to the model and we added a few interactions just for the fun of it um what we're going to see is for the variable shelf Lo sh location which is categorical it will have uh created more than one column so here oh I guess rather low medium high is good medium and poor I suppose I don't actually know the levels but so it's it's done what it's done is it's made dummy variables um for these three categories and you always need to leave one of them out because they uh aliased with the intercept that's right and and so it's done it all automatically for you so this is another instance of things you would normally have to do yourself to set up the model Matrix this m has Ms model spec has just done it all for you make made it much easier yes and there of course there are various ways you can you can add these two columns so experts you can if experts in specifying these can can do it their own way if you like but we're definitely not going to draw on that okay so that brings us to the the end of the lab on on regression and um next Lab is uh we'll see you next week for classification