Lab Demo - Explore & Visualize Datasets - Machine Learning with Tensorflow from Google Cloud #11

Key Takeaways

hi I'm Chris rolls and I'm a machine learning specialist here within Google Cloud where I build machine learning models for customers and right now I'm going to walk you through our first lab on exploring and visualizing the baby weight data set all right let's get started on our first lab on exploring and visualizing our baby weight data set so the first thing we need to do is make sure that we have launched quick labs and we have an active data Lab instance open so if you have not done it already open up quick labs and go ahead and click start lab once you do that connect to your data lab instance and once you do you will look at a screen similar to this one and what we want to do is in the training data analyst repo we want to go to courses machine learning deep dive and 0/6 structured so for this we will for this the duration of this section we will proceed throughout all six of these labs but for right now we're going to start on the first lab for exploring and visualizing our data set so let's go ahead and look at that lab great so now with this lab open in data lab we want to start exploring and visualizing the natality data set so what's the goal of this lab well the goal really is to get a handle on our data set right presumably this is a brand new data set we've never seen before and we need to figure out what we actually have in this data set and if we think about our goal our goal here is to given attributes about a baby and the mother trying to determine the weight of the baby so number one this means it's going to be a regression problem so we're trying to figure out we're trying to analyze the baby weight value so let's let's think more about the goal that we're trying to do we're trying to predict the weight of the baby and the first step we want to do before we even dive into model building is make sure we understand the data one thing that you might that you will notice is that this is a relatively large data set we have over a hundred million rows in our bigquery dataset so this probably isn't gonna it's definitely not going to fit into our local instance of our data lab so what are we gonna do well we're gonna have to aggregate our data using the group by clause with bigquery so we can bring our data in locally for visualization and then once we finish this lab then we can actually move on and start to do machine learning so let's walk through this lab I'm going to use the shift-enter keyboard shortcut which will allow you to jump from cell to cell instead of having to click run you can also hit click the Run button as well so if we look at this lab what we're going to need to do is we're going to specify the bucket project in region these will be specific to you and you will set them appropriately so once we do that we're going to set these values we'll set them as environmental variables so we can easily access them later and if you didn't already have that bucket created this command will create that bucket for you so let's actually run some sequel queries and look at the data that we have so you'll look at this example query that we have and again you could have written any query that you wanted but this is a nice starting out query so what this is doing is it's selecting some potentially really interesting features that we can use for a model like weight in pounds the sex the baby mother age and other ones and we are also using a farm fingerprint function which we'll talk more about later and you can see where we're limiting to all births after the year 2000 and so with that we've assigned the result to a query string from there we want to import the bigquery module as BQ and we'll run this command right here which will basically take that string that we've created execute the result in bigquery and return the result back as a panda's data frame so if you look at this we can run D f ked which returns the first first five results and we can analyze I'm right here so now this is interesting but we've only we've limited our results to the top to 100 and we know that there's millions tens of millions over a hundred million examples so this isn't going to get us exactly where we need to go well this is interesting to get a first take we need to go beyond this so what we can do is we can create a generalized function to to query data to query columns of interests that we want and if you think about it there's a couple key things that we're interested in number one we're interested in for each value of that column what is the count as in the number of babies and what is the average weight for that column value and this will become more apparent but this is an example query that you could use to analyze your data set and we'll use the string formatting function which allows us basically said given this column name plug this into the sequel query so whenever we call get distinct values we'll get to column two fields of interest the count and average weight so here we have a bar plot to see to see the the gender of the baby here let's call the distinct values code we already have the result but I'm gonna I'm going to call it again so we can visualize it and what we can see is we can look at the count which is the y-axis by sex so in this case is male false so blue means female green means male for the sex of the baby you can see there's a slight difference here in terms of the count but what we're also more interested in is is there difference in the weight of the baby right because this is what we're ultimately trying to predict it looks like there may be some difference so this is something to definitely note and it may be something that we want to include in our model so here we're going to plot the mother's age with respect to average weight in this case we're gonna actually be making a line plot and the way that we've specified that is it defaults to it defaults to line plots but above we specified that we want to use a bar plot there's other types that we could plot as well including a bar h4 a horizontal bar plot as well and I guess as a quick aside this is using pandas for plotting and pandas is a very concise way to plot the plot data so it's definitely worth getting the handle on so let's look at the results of mother age so again this is the count so you can see sort of the most common mother ages between 20 and 30 years old which makes intuitive sense as well but let's see if there's actually a correlation between mother age and baby weight well it looks like there is a pretty strong relationship now is this a linear relationship no it's not it's actually a nonlinear relationship so an increase in mother's age may or may not cause an increase in baby weight depending on the age of a mother let's also plot plurality the plurality is the number of babies born at once so plurality equals two means twins plurality equals 3 means triplets so let's look at the count by plurality so not surprisingly most mothers have one child at a time as opposed to twins triplets etc and we can also look at the relationship between baby weight which is the y-axis and plurality as well so again this is a pretty strong relationship so if we know the plurality that's going to give us some indication some signal for trying to predict baby weight so this is definitely a good feature that we're gonna want to use in our model and we can also look at gestation weeks what is gestation weeks that's essentially how long the mother was pregnant with the baby and again we're going to be using a bar plot here which is where we say kind equals bar and one thing we can also specify is log y equals true so we're gonna get log axis for the y value if you're ever curious about what arguments you can put into pandas one trick you can do is to shift and hold tab in data lab or hit tab and you'll be able to see documentation on the side and this is really helpful if you're like me and you tend to forget the arguments you can always use this as reference but now we can close that back up and here we can look at the number of babies born by gestation week so we can see that around about 39 weeks seems to be sort of the most common gestation weeks and it falls off in both directions and let's see if there's a correlation in baby weight interesting the longer the gestation weeks the larger the baby weight is I think this makes intuitive its sense as well great all these factors seem to play a part in baby weight this is encouraging this is telling us that there's actually some signal in the data and that now we're actually ready to move on and start creating our first machine learning model for the baby weight and that's it for today's lab we have now completed the initial steps of our end-to-end machine learning problem analyzing a data set and visualizing the data set next we will begin our process of creating our data set for analysis key steps here will include data cleansing feature engineering and creating repeatable samples that we can use for training and evaluating our machine learning model