Build a 2D convolutional neural network, part 12: Testing loop

Key Takeaways

so our data is trained or at least is in the process of training it is really natural to know how well it's doing what's our accuracy rate how well does it stack up against the whatever last run that we're comparing it against to look at this we have the test script this one's a little bit tighter it's a little bit shorter we did a lot of our hard work already but we start by importing our testing data our ten thousand examples of handwritten images to test on another logger called a confusion logger which we'll see gathers up our actual and predicted labels and produces a confusion matrix and then a convenience function load structure that's just a quick way to load that structure up from a file name we specify our number of testing iterations here we've just manually coded it to be the number of data points we happen to know are in the test set a little bit sloppy but effective we instantiate this testing data block from our testing data class and then we go and load up the structure that we saved and now we can tweak it we can modify it we can add this new testing data block to it and connect it up and if we refer back to our structure diagram port 0 on the testing data connected to the port 0 of the convolution 2d 0 block and port 1 from the testing data output port connected to port zero of the one hot block so we connect that testing data block exactly how the training data block had been connected and we initialize our total loss we initialize our confusion logger and here we did a little tweak so n iter report is the time at which a report is generated to get it to generate a report on the last iteration specified and testing iterations minus one so that when it gets to that n minus one iteration which is what the iterator will count to then it'll generate the report and then we have our testing loop so for each iteration in our number of testing iterations we run the forward pass on our network there is no backward pass this time we're not training anything so computationally it'll be a lot cheaper so we run the forward pass we take our total loss and we add whatever loss we have to it and then we can log the appropriate values to our confusion matrix if we go and look at the confusion matrix log values method specifically we can see that it wants three things it wants first the predicted result in the form of an array of the appropriate number of classes that's one hot it's all zeroes except for a one in the place of the prediction it also wants the actual result another one hot array with the one in the place of the given label and then an optional method which is the mapping from the label name to the position in that array and we can get that conveniently from the one hot block using the get labels method so with these three things it can track actual label predicted label and it can give it the right label name when it generates the plots i did want to call out that even if you pass it predictions that are not one hot if they're valued between zero and one but but could be fractional it can handle that too but the interpretation will be different than a standard confusion matrix so it's if you want to match apples to apples and call it a confusion matrix best to pass the predictions as a one hot array after it goes through all of the testing iterations it takes the total loss and it finds the average so it divides by that number of testing iterations and we get the average loss and then we can use our confusion logger so this confusion matrix that we created and call its calculate accuracy method to find the accuracy so of all of the examples what fraction of them did we get right in this case that's a reasonable way to summarize the results in one number because we know we have the same number of examples in every class because it's such a carefully constructed data set