Python Data Science Tutorial #12 - Pandas Series

yeah what is going on guys welcome to despite in tutorial series for data science in today's episode we're going to start with pandas because up until now we've covered matted lip & numpy and now we're going to build on top of that and introduce a new module a new library which is called pandas we talked about that in the beginning of the series and pandas is the library that allows us to efficiently manipulate and structure or data so pandas officers to data structures first of all the series and then the data frame and a data frame can be thought of as some sort of SQL table like structure or axial sheet like structure so you have a table you have columns you have rows and you have indices and all that so it's pretty advanced and it allows us it allows us to do a lot of data science operations we can efficiently manipulate data we can sort the data we can export it so you realize it visualize it and all that so panis is a very good library and we're going to start out today with the series so let us get into the code as always we first start by importing the modules that we're going to use and in today's video this is pandas ask PD so we're going to import the pandas module with the alias of PD because that's again the conventional alias that you give to that library and also we're going to import as always matplotlib PI plot but in today's video we're not really going to use map thought of itself because we're not going to plot with the P ot modular with the pipe load module we're going to use pandas for plotting but we need to map up the pipe load module to show the plot so we're going to use pandas for plotting but in the end we have to call PL teetered show to show the plot itself so as I already mentioned panas is built on top of numpy and on top of map of lips so when we plot something with pandas it uses math with lips so it needs to be installed but we're not really calling matplotlib dot pi plot or a PLT dot something and we're also not going to use numpy we're not going to import numpy because pandas uses numpy but we're not calling NP dot any function or any attributes here so let's get to the series how do we create a series in what is a series a series can be thought of as somewhat like a two on table or a little bit like a dictionary so we have basically values and indices key value pairs if you want or index value pairs so let's just create a series here serious equals PD series with a capital S and what we pass here are two collections or two lists and these two lists are first of all the values and then the respective indices so for example I can just say 10 20 30 40 and these are my values the values of the series and then I can say okay they have the indices a B C and D and then I can go ahead and print the series of course Oh around the wrong module mein where is it here and you're going to see that it's a little bit structured like a table so it's not structured like a dictionary with the key value pairs in in curly brackets it's more like a table we have the index column here the key calling here and then we have the values belonging to these keys or indices and then again or also not again we have a data type of integer 64 of course if I change this here to be a string I am a string it's possible it's not a violation of structure we can do that but then the data types object because we have three integers and one string and that's of course are not consistent but I think if I make all of them a string it should work and it should say that the data type is a string I'm not mistaken no it's not however doesn't matter I don't know why it's an object maybe I forgot something however basically the data type shows you what kind of data you have in this data frame or in this series in this case we're talking still about a series and this is basically how it works you have one column for the values one column for the indices or the keys and this can be used of course to create all kinds of structures like maybe a person with the keys name age weight height and so on and then the values and you may ask yourself now why should I use this instead of a dictionary and we're going to talk about that in a second because the series or pandas data structures in general have a lot of different functions since you can do a lot of crazy with them that a dictionary cannot do it all but we're going to get to that in a second but for now let's talk about how to access the individual values so we have the series here and if I want to access for example um let's just convert it back to numbers it actually doesn't matter but I don't like to have numbers and strings then I can just use the square brackets here and say give me the element with the index C and it gives me the value of 30 yeah so by the way it's not it's not about the datatype here so it's always the case that the second one it's always the second collection is always the index collection so if I was to swap the positions here so if I say okay these are my values and this is the second collection I pass this would be then my index collection so it's always the second one we pass is the index collection the first one is the value collection and of course if we want to swap that around we can I think this should be possible I'm not sure but I think we should be able to do it like that no I don't think so no we always have to first specify the positional arguments and any keyword arguments but we can still if we want do it like that would be redundant because it's already indexed but you can also specify the index keyword here another way that we can do that I think is not possible with dictionaries is we can still even though it's a key/value pair based collection you can still access positions or indices in in the form of numbers so I can say series dot I lock which stands for integer integer location I can still access that and say give me the position two for example and then it will give me the position two even though I don't have any index two or something like that I can use the I lock iterator here to say okay give me the third element with the index two of course and this is how we do it like in an ordinary list basically this is I think it's not possible with a dictionary um another thing that we can do here is we can specify a name for a series so we can say the series has a name my series and then when I print the series I can just get all the over view years so I have the keys to values the data type and then I have the name my series so these are the basics of a series in pandas now let's get a little bit more into the series so what can we really deal with that series and why is it superior to a dictionary now first of all we can even convert this into a dictionary if we want doesn't matter we can just go ahead and say print so for typecasting just dig for dictionary and then pass the series and you'll see that it's no problem to convert this series here into a dictionary so we just have the keys and the values of course I think it makes more sense to swap this around again so do you have these values here and these indices but basically we can take any series we want and convert it into a dictionary and with a series we can do a lot more things than with a dictionary because the dictionary is just a dictionary you can call the D values by keys so you can you can access the values by referring to the keys but that's basically it you cannot really call positions you cannot really do any calculations with dictionaries but with serious you can do all of these things so what we can do for example is let's define a series as one and this series has the numbers 1 2 3 4 ah with a b c and d and of course we don't want to semicolon here and then we have a second one with the same indices so we still have a B C and D but now we have different values for example seven five one two and what we can now do what we can not do with dictionary is we can just go ahead and say print as one plus as to that's it so what happens then is we just add the values that have the same index so now we have a equals eight B equals seven C equals four and D equals six so we just added all these values at 1 plus 7 2 plus 5 and so on because they all have the same indices and I'm not sure again but I'm going to try that I think it also works if they don't have the same position so now we have a CBA D and ABCD but I think it should still add one and one so still add a and a and C and C but we're going to check if this happens and of course it happens yeah so we take a and add it to a and we take C and add it to C and then we get the results so the positions are not really important it's all about the index so this is something that you can do with series that you cannot do with dictionaries now another thing is that we have a lot of functions that we can use for example we can just print ask 1 that count to count the amount of elements in this series which is actually quite basic I think you can also do something similar with the length with the length function and dictionaries but we can also just say I'll show me the first two rows or show me the first five rows or show me the first 20 rows and this is something that we're oftentimes going to use with data frames that are pretty similar to series but this is for example the head function and the tail function so these functions give you the last and the first couple of elements in a series so for example s 1 dot hat 6 would give me the first 6 rows of a series as one hat two would give me the first two rows and Tails of course gives me the last row so the last four rows the last two rows and so on so we can just do that and you'll see that the first thing what's that is Wonder tales or tale signal tales Patel yeah so as you can see it gives you the whole information with the datatype and all that and it just gives you just the first two rows and the tail function gives you the last two rows so this is quite useful if you want to get an overview over the over the series or a date frame later on but you don't really want to print the whole thing so that you have too much output you just want to look at the first five rows to see how the thing is structured and this is quite a useful function here that we're going to use in the future now another thing that's quite useful is that we can apply functions on all the values in our series so we can just go ahead and define a function my Square and I pass a number X or a value X and I return X to the power off to that's a simple function and what I can do now with the series is I can just go ahead and say s1 dot apply and refer to my my square function with that calling it just referring to it and then print the result here and you see you'll see that it squares all the numbers so 1 squared is 1 2 squared is 4 3 squared is 9 and 4 squared is 6 16 sorry and this is quite useful because you can just take one function apply to all the values at once and it's not real difficult it works kind of kind of well and you can do the same thing with the square root function and all different functions that you have even more complicated functions of course they have to return something you cannot just print them and then expect something to happen but basically that's it and it's quite quite useful and another thing that we can do that somehow works also with dictionaries but not in such a smooth way as it were works with series is the sorting of a series or the sorting of a collection so we can just go ahead and say print s1 dot sword and then we can choose to either sort by index or sort by values because a series is always sorted by indices or actually not always but in our case it's sorted by the indices here and also the values so in this case it doesn't make any difference but in this case it would make a difference because it's neither sorted by values nor sorted by indices so we can just go ahead and say as to dot sort index and we would get a sorted collection as a result so as you can see we now have ABCD even though we started with CB ad and if I want to sort by value I can just go ahead and say soared values and we would have a different result so it will now be a DBC and one more thing that I want to show you here is when we call a function from a series or a data frame later on we always get a copy return so when I say for example s1 dot sort values what happens is it just returns me a sorted version or just returns sorted version of this series so again in this case doesn't make sense but s2 s2 that sort values just gives a or returns a sorted version of s2 but without applying anything to the actual collection so we have as two sorted but if I do this and then print s2 you're going to see that it's not sorted because it's still it still has the same arrangement here if I want to now apply this one way I can do this of course is just say as two equals as two dots or values but in pandas I also have a special keyword that is called in place and if I say in place equals true this is a parameter if I say in place equals true what happens is that we don't return anything but we applied on the actual series so in this case we would get a sorted version of as - as you can see another very awesome thing about pandas data structures is the de allow us to easily visualize our data so in this case I have a panda series here with eight elements with eight values and eight keys or indices and what I can do is I can just go ahead and say s1 dot plot and then PLT dog show and that's it that already visualizes the values in this series this is not something that you can do with the dictionary because in dictionary you have to take the keys take the values and then somehow prepare them for a pl T dot plot function all that but here you just say s1 dot plot and that's it and this also works with a data frame later on we're going to look at that in the future but this is very simple you just have all the indices on the x-axis and all the values on the y-axis and you can visualize it like that but of course not only with the line chart you can just go ahead and say s1 the plot and now you can choose all kinds of different chart types so for example I can just say s1 dot plot dot bar and it gives me a professional our chart as you can see so again we have labels here on the x-axis and the values on the y-axis and of course if you want a professional chart with different colors with legend with labeling you need to use map of the moor you cannot just use pandas but if you want a basic quick graph you can just go ahead and use the pandas or plotting functions of course I can also use the bar H function for a horizontal bar or horizontal bar chart as you can see and of course one of my favorites the pie chart so s1 dot plot pie and that's it and you get a bar chart with labeling in different colors without percentages though so if you want that you again have to specify some more things but this is pretty basic and it works quite simple so histogram would be another thing that we can do and it would give me a statistical distribution as you can see it looks kind of shitty because you know what you want to visualize here it gives you basically the values and their frequency but it works it's just one function and you have the chart and this is one big advantage of pandas when it comes to data signs now one of the things that I really love about series and pandas in general is the possibilities to export to almost any data type so we can take this series here and just say s 1.2 and then we have all kinds of different possibilities to export so we can go ahead and say to ask you out and if you have an SQL connection you can just pass the series into an SQL connection so basically insert it into the database of course there's a little bit more advanced but what you can also do is you can just say to string so you can just convert the series into a string and then use string functions for example to extract some data out of it or you can go ahead and say to axle for this you need a another library so it if you if you just run this and try to save in an axle file you need to import a module or you need to install a module that you probably don't have unless you've used it in the past but basically it works and you can also export into CSV files sorry so you can just go ahead and say s 1.2 CSV and I can say my series dot CSV and when I run this you'll see that it creates a CSV file with what is that signature is was aligned to data frame CSV of course yeah but basically it works so my series dot CSV and you can see that we have a 1 b 2 c 3 d 4 e 7 and so on we have the series in or csv file and this can be done with all kinds of different data types so we have jason we have dictionary we have latex numpy pickle all kinds of different formats that we can export or series into with just one function and this is also not possible with the dictionary at least not that easily and this is one of the great advantages of pandas so this was the first tutorial about pandas and we talked a lot about the series from next video on we're going to focus more on the data frame because that's the basis or that's the core off can series are nice and fine but we mainly use data frames because data frames offer multiple columns so that's it for the series for today's video and thank you very much for watching if you like this video if you enjoyed it if you learned something hit the like button also feel free to ask questions and give feedback in the comment section down below and of course as always subscribe to this channel if you want to see more and thank you very much for watching soon our next video bye [Music]