Python Tutorial: Transforming categorical variables

Key Takeaways

now that we know what are the categorical variables in our dataset we can start transforming them into numerical to transform a categorical variable into numeric we have to understand its type first there are two types of categorical variables ordinal and nominal ordinal variables have two or more categories that can be ranked or ordered in our case there's the salary column or the values clearly have a logical order the second type is nominal where categories do not have any intrinsic or logical order an example of this kind of rebel in our dataset is the count Department as its values clearly do not have any order or rank sales department is not higher than ager or vice versa and so on based on what type of categorical variable you have there are different methods for transforming them for the case of ordinal variables we can encode categories by converting each of them into respective numeric value the three steps to accomplish this task in Python first we have to tell Python that the counsellor is actually categorical this is done by using method called test type which is providing the type of the variable then one spy to note that it's a categorical variable we have to tell the correct order of categories using carry order categories method as you can see in the code this method takes the list as an input where the correct order of categories is provided last but not least we have to use cat codes attributes to encode each category with a numeric value given our order the result will override the old values of salary column the new numeric values as presented in a table the next categorical variable is nominal and there is no order of rank between departments this means that the encoding approach is not useful anymore in this case transformation should be accomplished through so called dummy variables dummy variables are the variables that can only get two values 0 or 1 let's an employee is from the technical department this means if we have a separate column for each department then the mentioned employee will have value of 1 in that column for technical and 0 in all other columns of departments this mess we will have to create a new data frame where each department is a separate column and each row is a separate employee with wounds in front of his/her department and zero in all other places well the task seems to be confusing it's very easy from technical perspective do two very nice function from pandas called gilt dummies well the minute dummy variables one should be cautious of a phenomenon known as dummy trap the latter is the situation when different dummy rebels come with the same information in this example the sample employees from the technical department so it's the only column with a value of 1 in the first table in the second table the last column is dropped but we can still understand that the employees from technical department by looking at all the other departments that have value of zero for that reason whenever in similar situations dummies are created one of them can be dropped as its information is already included in others okay time to put this