A minimalist's guide to slicing and indexing pandas DataFrames

Key Takeaways

hi this is Brandon roar and this is a minimalist guide to slicing and indexing pandas dataframes there are a lot of ways to pull the elements rows and columns from a data frame if you're feeling brave sometime check out the seven part series on pandas indexing linked to below some indexing methods appear very similar but behave very differently the goal of this post is to identify a single strategy for pulling data from a data frame that's straightforward to interpret and produces reliable results and just a heads up these are my own thoughts only there's no guarantee that it's authoritative or even right now in case you wanted to skip to the end here's the bottom line one use dot lok for labels to use dot i lok for positions and three explicitly designate both rows and columns even if it's with a colon we'll step through some examples to illustrate these below is a link to the Python script if you'd like to run them yourself to start with we'll create a small data frame using data from Wikipedia on the highest mountains in the world for each mountain we have its name height in meters year when it was first summited and the range to which it belongs if this is your first exposure to a panda's data frame each mountain and its associated information is a row and each piece of information for instance name or height is a column each column has a name associated with it within pandas also known as a label the labels for our columns are named height in meters summited and mountain range in pandas dataframes each row also has a name now by default this label is just the row number counting starting at zero however you can set one of your columns to be the index of your data frame which means that its values will used as the row labels we'll set our column name as our index it's a common operation to pick out one of the data frames columns to work on to select a column by its label we use the dot look funky can do that makes our commands easy to interpret is to always include both the row index and the column index that we're interested in in this case we're interested in all of the rows so to show this we use a colon then to indicate the column that were interested in we add its label the command mountains dot Lok colon comma summited gets us just the summited column it's worth noting that this command returns a series the pandas data structure that's used to represent a column if instead of a series we just wanted an array of the numbers that are in the summited column we can add dot values to the end of this command that would return an umpire array containing 1953 1954 1955 and 1956 if we would only like to get a single row then we can use the dot location again this time specifying a row label and putting a colon in the column position if we only want a single value for instance the year that k2 is summited then we can specify the labels for both the row and the column the row always comes first well it's true that you can get away with using only one argument in the location it's most straightforward to interpret if you always specify both the row and column even if it's with a colon we don't have to limit ourselves to a single row or a single column using this method here in the row position we pass a list of labels this returns a set of rows rather than just one we can also get a subset of the columns by specifying the start and end column and putting a colon in between in this case height colon summited will give us all of the columns between and including the start point height and the end point summited note that this is different than numerical indexing in numpy where the endpoint is omitted by default also because we've already specified the name column as the index our result will also be returned the name will also be returned in the data frame that we get back in addition we can select rows or columns where a value meets a certain condition in this case we want to find the rows where the values of the summited column are greater than 1954 in the rows position we can put any boolean expression that has the same number of values as we have rows and we could do this for the columns as well if we wished as an alternative to selecting rows and columns by their labels we can alternatively select them by their row and/or column number the ordering of the columns and thus their positions depends on how the data frame is initialized the index column our name column doesn't get counted in this case to select data by its position we use the I'll oak function again the first argument is for the rows and the second argument is for the columns to select all the columns in the zeroth row for instance we write I'll oak 0 comma colon similarly we can select a column by position by putting the column number we want in the column argument of the ILOG function we can pull out a single value by specifying both the position of the row and the column we can pass a list of positions if we want to cherry-pick certain rows and/or a certain call we can also use the : range operator to get a contiguous set of rows or columns by position note that unlike the Lok using labels the I Alok function using positions does not include the endpoint in this case it returns only columns 0 and 1 and does not return column 2 all of this can be summed up as follows one use Lok for the label based indexing to use I look for position based indexing and three explicitly designate both the rows and the columns even if it's with a colon this set of guidelines will give you a consistent and straightforwardly interpretable way to pull data that you need from a panda's data frame good luck with your data munging you