Casting Columns & Renaming Columns | Beginning Azure ML | Part 4

Key Takeaways

hello Internet welcome back to the data science Community last episode we found out through data exploration and data visualization that our data set's still a little bit raw and will require some pre-processing before we can properly create machine learning models so in this episode I will go over with you three things the descriptive statistics module how to cast your columns into direct correct Class Type and how to rename your columns okay before we do any pre-processing I want to show you a quick module that you can also use to further explore your data so if you go and expand your uh modules and then select on statistical functions and then go to descriptive statistics and just drag that module in all right so and all you have to do now is feed your data directly into the descriptive statistics model now keep in mind this model is still an empty model there's we have to run our experiment again for it to fully go through so once the cloud has finished uh processing your module you can actually see that there's a check marks in the module itself and if you click on visualize in the module the module is actually no longer an empty shell of itself there's things inside of it but you can tell that there's something not right the mode looks very messed up when it comes to passenger ID right but everything else seems for the most part to be all right let's figure out why it broke all right so if you go under the Titanic Titanic data set itself and go to visualize so now that we're in the data you can see that there are 891 rows and passing passenger ID has 891 unique values so the mode broke because currently passenger ID is listed as numeric and the mode broke because there's an 891 way tie for the highest frequency value so that's a good indicator that passenger ID should have probably been a categorical value so we should recast uh from numerical into categorical survive should also be categorical pclass should also be categorical and sex should be converted from a string to a categorical as well as the embarked okay let's start our first pre-processing of the data so first we're going to do some casting of the data and to do that you would go under data transformation and under manipulation you will find something called a metadata editor so this is a module that that let you basically edit your columns right so metadata simply means data about the data so we're going to edit the data about the data so we're going to basically feed the Titanic data set into the metad DAT editor and let's actually remove this routing that we have from descriptive statistics so if you go into the metadata editor there's a number of things you can do right you can change the data type all right and you can change it from categorical to non-categorical or you can change the fields into a labels or a feature and then you can also change the column names as well so you want to start by selecting The Columns that you want to actually convert into categorical Val so launch the column selector over here and from that if you click on this column box see it it will show up all the columns that were available in your data set so we established last time that passenger ID should have been one survived pclass sex and Embark should all be categorical values so we want to check that so those are the columns that are now selected and now we want to go from categorical from unchanged into categorical and now you want to feed the output of the metadata editor into the descriptive statistics and then run your experiment again once that's completed you can actually go into the output of the metadata editor and then just visualize what's happening in there passenger ID is now categorical survived is now categorical P class is now categorical as well as sex and Embark now that we've ens shared that that works let's label this for good practice so this is categorical casting so now we know exactly what that does now if you notice if you go back into your original data set and visualize that everything's still numerical so this is still your raw data it's untouch still right so once it's only after it gets past this node does it get changed now you can actually go into your descriptive statistics and then visualize it and look it's not broken anymore because passenger idea is now casted correctly now before we move on I want to talk a little bit about the intelligence and how it works inside the metadata editor if you were noticed earlier when we watched the column selector that there was all the columns that were present in the data was already fed into a pre-generated box however if you just had a free floating metadata editor and just tried to launch the column selector you will notice that nothing shows up it's only after that you feed a data set into the metadata editor while the intell activate or a processed module that's outputting data as well so you can see if I fed the rendered metadata editor that we had previously casting categorical that intellisense will also show up however you will notice that this metadata has an exclamation mark next to it which means it's not processed yet which means it's an it's currently an empty module so if I tried to send into another metadata editor you will see that nothing shows up in the intelligence so if you're remembering last video I tried to do data visualization with the survived column before it was casted correctly into categorical data and it broke so now if we go back and ask the same question again did Sex have anything to do whether or not a person lived or died we can properly answer that question because survived is properly casted as categorical so let's do that so let's just compare that with survived and you can tell from this distribution that a disproportionate amount of men died in the Titanic and while we're here visualizing the we might as well go on to our next topic which is renaming columns and that can be done for a number of good reasons right cuz look at this column name CSP and parch right so is CSP stands for siblings and spouses that were with you so how big and how wide was your family that traveled you versus parch which is parent and child so how big was your immediate family how many parents did come with you and how many kids came with you and P class is also a good candidate for rename too right cuz it's right now it's it represents first class second class class third class but I think accommodation class would be a much better name for it so we've selected three columns as good candidates for renaming and each metadata editor can actually only rename one column so that means you need three metadata editors so drag either drag in three or even better if you can select one metadata editor contrl C and then contrl V and you can have as many metadata editors as you want just by copying and pasting so let's get one more out here and basically just line them up and just have them flow into each other like a waterfall so this data set is going to feed into this data set it's going to render it's going to feed into another data set let's launch the column selector and then so we were going to rename P class okay and we're going to rename P class accommodation class and for a bit of style you want this to be a single string so either use camel casing or use underscore snake uh casing because if you ever tried to export this back into a database the space between the column names will actually break the database so it's good to keep good style if you want to keep using your data and then let's move on to the next metadata editor so if you remember correctly what I said earlier the intellisense won't work here because it's being fed from a non- rendered metadata editor so I just happened to remember what the column was and that was C SP and keep in mind that case does matter in this case for column names and then for the bottom one I wanted to rename parch all right and then I wanted to rename parch into parent child and I we want to rename C SP into sibling spouse okay so now that that's all done we can just run it and now we have three metadata editors which are all doing different things so let's actually name that to be considered to other people who are actually going to work on uh the same experiment so this got renamed to accommodation class and this got renamed to SSP into sibling spouse and likewise uh parch got renamed into parent child all right so let's expand all those so now it's very apparent and clear what happened to all of the data and once that's done loading you can go to the very bottom node and visualize what's been done to your data and you will notice that sibling spouse was successfully renamed parent child as well as accommodation class so these are much more clear now when other people are working on this and that concludes the video tutorial on how to use the metadata Editor to rename your columns as well as to recast column so join us next time when we tackle this issue right here so you will notice that age has 177 missing values and that's actually going to cause problems because it's a numerical value and our models expects continuous data to stay continuous if you like what you just saw subscribe to our Channel or leave us a comment let us know if there's a topic you want us to cover and be sure to check us out at data science dojo.com until next time