PyTorch Mobile Runtime for iOS

Key Takeaways

hello and welcome to the pike torch summer hack 2020 I'm Brad Heights and I'm a partner engineer with the PI torch team if you're not sure what a partner engineer is it means that my day job is making sure that developers like yourselves get the most they can out of PI torch and related products in this video I'm going to give you a walk-through of setting up the PI torch mobile runtime in an iOS project to follow along you'll need Xcode version 11 or higher on Mac OS you'll need to have cocoa pods installed and if you want to deploy to a device you'll need an Apple Developer account in an iOS device running at least iOS 12 you should also be using PI torch 1.5 or higher PI torch offers native runtimes for iOS and Android allowing you to bring your machine learning app to mobile devices including the library in your project as a one-liner but there is more to do getting set up for best performance on mobile in this video I'm going to show you how to set up your project to include the PI torch runtime how to export your model to torch script PI torches optimized model representation how to further optimize your model for best performance how to add your model to your Xcode project and how to call the model from your code let's dive in for this demonstration we're going to build an image classifier into an iOS app first we'll create the project I'm going to create a single page app with UIKit UI now close Xcode and switch to your command line so here's the pod file we're going to attach to this project you can see it specifies iOS 12 and PI torch 1.5 so now pod install that and we'll open the workspace now instead of the project it's important when you're using cocoapods and then we'll go ahead and run the project just to make sure all as well and there's the sim nothing there yet no surprise we'll stop that the next thing we'll need is a UI I'm going to have one image of you one button and one textview in my app the image view and text view will have outlets in the view controller and the button will have an attached action rather than making you watch me set up a bunch of storyboard constraints we'll do a little movie magic Yola now I'll show you how to set up and optimize your model for use with pi towards mobile so first thing first we'll import PI torch and torch vision and this is a good place to point out that you should be using PI torch 1.5 or higher for this example in the next cell we're going to create a PI torch model object now for the actual app I have a pre trained optimized model ready to go but for the optimization process I wanted to show you this on a custom model so that you could duplicate the process with your own models this model happens to contain some very common layer types A to D convolutional layer a to D batch norming layer and a rectified linear unit for activation and the forward function just strings those three operations together so now that we have our model how do we optimize it first thing let's get an instance of the model in my get model helper you'll notice that besides just instantiating the model I call MD Val so eval turns off things in the model that you don't want on during inference time training only layers like drop out automated gradient tracking all this training related stuff meets up CPU cycles and we don't need it for inference so we're gonna make sure the model is in eval mode the second thing we're going to do is some layer fusion fusing layers means taking multiple operations and combining them together into a single operation this improves performance in memory footprint now with diffuse modules method that I'm going to show you there are only certain combinations of layers that you can fuse together I'm going to refer you to the documentation for the latest information on that but here we're going to try to use together convolution bachelor-man Darrell you once modules are fused the next one we're going to do is quantize the model PI torch tensors default to using 32-bit floating-point numbers as their underlying type when we quantize we're going to change that to an 8-bit integer this will perform faster and reduce the models footprint both on disk and in memory the I don't think we're going to do is save the model as torch script torch script is an optimized format for your model including both your computation graph and you're learning weights it's meant to be consumed by the PI torch in just-in-time compiler or JIT which is included with the PI torch mobile runtime so once it's exported will save now there are subtleties to layer fusion into quantizing your model that you'll want to be familiar with when you're optimizing your own model for use with PI torch mobile all of this is covered in the PI torch documentation for quantization which I encourage you to check out now I'm going to pull in the resources for this project they include the model so in my case that's an optimized version of mobile nephew trained on the thousand categories of the image net data set we also have a text file that contains the human readable labels of those thousand categories and finally we have an image file for our classifier to work on there should be links to a sample project that includes these resources wherever you got this video now we can flip back to Xcode and I'll add the three files to my project now how do we call pi torch from our code because pi torch is implemented under the hood in C++ with Lib torch we need a wrapper for that C++ library so we're going to go ahead and create a new group in the project we're going to flip back to the command line and I'm going to add these two files torch module dot H and torch module dot mmm again these should be available wherever you've got this video and then Xcode is going to offer to make a bridging header for me and I'm going to let it do that so you can see here in the header there are only two methods that are really interesting one which initializes the PI torch runtime with a model which is store as a file and it's our model of resource in our project and the second one that does the actual prediction when you pass it in a buffer full of image information and you can see the C++ bridge code and torch module mmm and now add my image resource to my image view so we can see what it is we're classifying and speaking of images I'm going to add one more source file and this is going to be in addition to the UI image framework class that allows us to resize it more easily the particular model I'm using because it was trained on image net expects three color images size 224 by 224 pixels so this helper role helped me achieve that we have some stuff to fill in the my viewcontroller first I'm going to add a lazy loaded instance of our module our model you can see I'm loading it from the file and initializing the torch module object with it also I have lazily loaded array of strings the labels those thousand human readable labels for the categories that the model recognizes I'm also setting that up here now you notice we've got an error on the torch module type it suppose we're to go back to the bridging header and make sure we include torch module page if for whatever reason your project isn't recognizing the type go check the bridging header make sure you've included a header for torch module dot H now we're going to fill in the body of our inference buttons action and you can see here we're going to load the image we're going to resize it to do 224 by 224 we're also going to normalize it so just as with the size of the image our model expects the image to be normalized in a certain way and we have a method to do that and finally we take our image buffer and we pass it to module dot predict after that we retrieve the label or the predicted category and display it in our text view so now at last let's run our app yeah you'll see it starts up and there's our kitten we press the infer button and be aware that even if you run this on a laptop it may take a minute to run the first time but there we are our model thinks that this cat is a cat and that is how you get a pi torch model onto iOS thanks for participating in the hackathon and thanks for watching this video