Tensorboard Explained in 5 Min

Key Takeaways

hello world it's SJ and in this episode we're going to talk about tensor board tensor board is a data visualization tool that comes packaged with tensorflow tensorflow programs can range from super simple like an addition problem to Super complex using thousands of computations and they all have two basic components operations and tensors the idea is that you create a model that consists of a set of operations feed data or tensors into the model and the tensors will flow between operations until you get an output tensor which is your result tensor board was created as a way to help you understand the flow of tensors in your model so that you can debug and optimize it we're going to visualize a classifier that can recognize handwritten digits in tensor board we'll use two examples of the classifier one is a simple version in 60 lines of Python and the other is a more complex version python I love you don't worry if you don't understand every line of code we just want to get a general of what's Happening under the hood so that we can later understand the tensor board visualizations here we go we'll start by importing tensorflow in our input data class which pulls our training data from the web then we'll Define two helper functions in it weights we return a variable that contains randomly generated weight values along a normal distribution our second helper function will create our model A three- layer feed forward neural network we're going to create three layers and Define them via the Nam scope function the name scope function creates highle name spaces for operations in the tensorflow graph that we will later visualize in each layer will'll run a Dropout function and an activation function Dropout is a function that forces our neural network to learn several different representations of patterns so that generalization improves the activation function creates probabilities out of numbers so now that we have our two helper functions we can create variables to store our training and testing images as well as our training and testing labels next we can create input and output placeholders for our data then we'll initialize our weights between each of the three layers using the function we defined earlier we'll Define histogram summaries for each of our weights so we can visualize the distribution of Weights later in tensor board The Next Step is to add Dropout placeholders to our hidden and input layers we can then create our model using the variables we've created now that we have our model we'll create our cost function the cost function is a measure of how good a neural net is with respect to its given training sample and the expected output we want it to decrease over time we'll also measure the accuracy of the network we want that to increase over time we'll later visualize both next we'll create a session to run our graph computation in our session we'll save our model using the summary writer lastly we'll initialize our variables for training and train our model that's it I know that was really fast unlike series comp apprehension you heard me Apple let's visualize that code in tensor board and I'll explain it more summary operations are how tensor board acquires data from our tensorflow runs they are functions like tf. Matrix multiply we created a number of them in our code here in the events tab we can view our scaler summaries accuracy and cost the x-axis shows the time steps and the y- axis shows the accuracy measure or the percentage of correct predictions over time we can blow up our graph for a closer look or view a wider range of data points by expanding the y- AIS we can also drag a rectangle on our graph to zoom in on a certain region if we' like and double click to zoom out as we Mouse over the chart it will produce crosshairs with data values we can change how smooth our line is by adjusting the slider the step option shows time steps the relative option shows the time relative to the first data point that means the number of minutes or hours since the training run was started and the wall option shows the actual time the runs happened we can create more tags as well in the sidebar it can either be an entirely new tag or a tag that will group a bunch of existing summaries together into a new category let's switch to a more complex version of our handwritten character classifier example with even more scalar summaries we can type in it's over 9,000 and since no summary contains those terms it becomes a new category but if we look under Max and mean they both contain summaries for biases so if we type in biases it will create a new category that contains both biases from these two existing categories we can also download our graphs in CSV or Json format for this more complex example we have lines for both training and testing data that way we can compare runs let's move on to images we created an image summary this lets us view the images in both the training and testing folder okay let's switch back to our simple classifier and move on to the graphs tab which lets us inspect our TF model we can see each of the three layers we created via the name scope function let's double click on one of our layer name spaces to expand it the first thing that pops out is the Dropout function which we declared inside of the scope as well as our relo and Matrix multiply function notice the arrows from both nodes pointing to the cost operation the direction of the arrows shows the direction that the tensors are flowing our two placeholder operation nodes X and P keep input hidden which is used for the Dropout function serve as entry points for our tensors they move through each of the three layers through the weights into the cost function every time eventually to the accuracy function and finally to the output placeholder that g gives us our prediction to reduce clutter tensorflow automatically shows nodes with many connections to other nodes in their own area to view in detail if we wanted to we can add them back to the main graph by clicking the add to main graph button in the detail card yo um bringing it back let's take a look at the sidebar we can choose which run we want to display train or test and show the graph at each time step the color option lets us see at each time step what structures are being used which device each operation is running on compute time along a scale and memory usage we can also manually upload a saved tensorflow graph right from the UI if we'd like let's move on to distrib ions we created histogram summaries for all three of our weights between layers and we can view the distribution of each weight here let's switch to our complex classifier the light part shows all the weights across time and the Shaded Parts shows those weights that are actually activated during training these curves represent percentiles like the Max and mean and median the histogram plot allows us to plot any variables we like from our graph it's showing how the values of our weights change with training isn't it beautiful tensor board lets you visualize your data so you can debug and optimize it you can create summary operations in your tensorflow program which takes tensors as inputs and produces outputs tensor board reads these summaries and displays them visually the challenge for this video is to create a tensorflow program that visualizes audio data in some way and winner gets a shout out from me in two episodes more info and links in the description Please Subscribe for more ml videos and for now I've got to go get some sunlight so thanks for watching