R Tutorial: Visualizing subsets

Key Takeaways

we have seen that visualizing summaries can help us discover and describe overall trends and relationships in the data in this section we will explore some examples of how we can complement summary visualizations with detailed visualizations of smaller subsets of our data while summary visualizations can be very revealing sometimes important insights are covered up in the summarization and we need to look at the data in more detail to discover them for example here we have a summary visualization of the annual return for four stocks from the summary it appears all four stocks had a similar year a return of around 13% however looking at detailed plots of the daily prices we see a very different year for each stock we'll have more fun with this stock data in chapter 3 visualizing large data in detail is challenging because there's too much data to look at a useful technique in this case is to take a manageable subset of the data that has some natural meaning such as for example all data for one stock and visualize and explore as we saw in one of our previous exercises the distribution of the tip amount is zero for all payment types but credit card this is an interesting phenomenon that we want to get to the bottom of with cash payments does the taxi payment system not distinguish between tips and fare where does the total fare amount just not include the amount that was tipped to investigate this question we turn to detailed visualization of a subset of our data we expect rides of the same nature to have similar fare and tip amounts therefore if we can pull out a subset of our data for similar routes we can compare the distributions of fare and tip amount to investigate our question we expect the distributions of total fare for rides paid with cash and card to look similar if both cases include tips here we have extracted a subset of the data for the most popular route from the Upper East Side south to the Upper East Side north of man looking only at these trips and only at cash and credit transactions we have about 5,000 observations let's do a check to ensure that this subset is well-behaved looking at the relationship between total fare versus trip duration we expect the relationship to be cleaner since we are focusing on one simple route even with the data this small we are still over plotting many points and we can alleviate this to a degree using the Alpha parameter to add transparency to the points this looks much cleaner than what we saw for all routes to compare the distribution of payments using card versus cash we can use a quantile plot this displays the ordered values of the data against the quantiles of a uniform distribution and is often more useful than a histogram for comparing distributions we create a quantile plot using GG plot to use geum specifying that the data should be plotted against the uniform distribution in this plot we see that the card and cash distributions have a similar shape but are shifted we also see that the cash payments are made up of several discrete values while card payments are more continuous which we wouldn't be able to see in a histogram due to binning from this we can reasonably conclude that tips are not included in the total reported fare amount for cash payments in the exercise we will see if the two distributions are similar if we remove tips from both let's go