TensorFlow Enterprise: Productionizing TensorFlow with Google Cloud (TF Dev Summit '20)

Key Takeaways

[Music] hello everyone my name is Makoto a software engineer intensive enterprise as a part of Google cloud now that we have seen the great story about tensorflow in production at work and its cool use cases even in space now I'm going to talk about the enterprise radio application of a stencil Enterprise so what does it mean to be tense of enterprise what is so different what is so difficult well after talking to many customers we have identified a couple of key challenges when it when it comes to enterprise grade email first is a scale and a performance when it comes to production grade enterprise applications often often times the size of data the scalable model is beyond what we put into my laptop or workstations as a result you know we need to think about this problem differently second is the manageability when developing a business applications it is better to not to have to worry about any you know nitty gritty details of in factors infrastructure complexity including managing software environment and managing multiple machines in Costanzo what not instead it is desirable to only have to concentrate on the core business logic of your machine learning applications so that it makes the most benefit to your business third is the support if your application is a business critical and a mission-critical the timely resolution to the bugs and issues and a commitment to stable support for application is essential to continue operating your applications Tencent applies brings a solution to those challenges let's take a look at close to your performance in a nutshell with tencel for enterprise we compile and a ship a special build of tensorflow specifically optimized for Google Cloud it is purely based on the open source sense of law but it also contains specialized optimization specifically for Google Cloud machine there are services in a form of purchase and add-ons let's take a look at how it looks like in practice this code trained the model with potentially very large training data there may be a terabyte of data maintain a Google cluster as you see it it is not different from any typical tenth of a chord written in return with a dataset api's except the pass to the training file is pointing to the Google Cloud storage under the food optimized are your readers specifically made for Google Cloud storage it is making this performance evil ways terabyte of training data and makeup training very performant this is another example that with training their assets from beaker a table which is the data warehouse which may maintain 100 millions of data business data data warehouse this example is a little bit more involved but still similar to the standard data feed API that you and all of you are familiar with so that you know that your mother can still train in your familiar ways but under the food the optimized are you of a Google bigquery can read many many millions of rows in parallel in a inefficient way and turn into tensors so that you your training can continue with the performance this is a little comparison of the throughput when large data is read from Google Cloud storage with or without optimization that content of enterprise brought in as you see it there is a nice through pure gain the better IO throughput performance actually translates into the better utilization of processors such as CPUs and GPUs because IO is no longer the bottleneck of the entire training what it means is you took your training finished it faster and your training wall time is shorter and the result your cost of training is actually cheaper because the continued complete cost is the proportion to the wall time that you know you use the compute resources now that you get some idea have some ideas about what kinda optimization we were able to make the tensorflow specifically for Google Cloud let's see how you actually get it how you actually take the benefit of it we do this through a managed services we deliver a tenth of enterprise through a managed environment which we call a deep learning virtually virtual machine images and container images where all the environment is pretty honest and pre-configured on top of standard Linux distributions most important is it has tensor Enterprise build pre-installed together with all the dependencies including device drivers and a dependency Python parties with correct version combinations or whatnot as well as configuration to the other services in Google cloud because this is just a normal virtual machine image and contain images you can actually deploy deploy it in many different ways in Google in cloud regardless of where you deploy it or how you deploy it the tens of enterprise of optimization is just there so you can take the benefit of all the great performance to get started you only have to pick the tencel for enterprise image and a different resources such as CPUs and run what option is GPU start the virtual machine in just one command in the next moment you can already SSH is a machine that has tens of enterprise build already pre-installed and pre-configured and it's ready to use so that you can start immediately start writing your code in the machine if we prefer notebook environment typical lab is hosted and already studied the VM actually the only thing you have to do is you have to point you know you only have to point your browser to the VM and open up the Jupiter lab and open up a notebook so that you can start writing your tests off record taking the benefit of tensor free enterprise once you have the factory model after many iterations of experimentations now is the time to turn your model at the full scale it may not fit into the one machine and you may want to take advantage of the distribute training facility the tenth of offers so that you know so that it can support a large scale of data on the model for this air preform training is a managed service that takes takes care of distribute training clusters and all other infrastructure complexities on behalf of you more importantly it drives the same tensor for enterprise container image which is exactly the same environment you have used to build your action model so you can be confident that your model just trains with full scale of data under the manager training service you simply need to overlay your application code on top of the TF Enterprise container image then issue one command to start distribute training cluster this example is grabbing a ten workers with larger machines per each workers with 8 GB data to each worker to Train potentially build a large dataset for your great applications this example brings up a distribute training cluster with all things of which interpret of observation included with I can work with distributions now that you can train your model in a full enterprise scale it is the time to make it an end-to-end pipeline to continue running it in production taking advantage of a platform pipelines and tenth of my extended airport from pipelines is actually hosted on the kubernetes engine but the mean is it can also drive the exact same things of enterprise container image so that all optimization is still there and you can still be confident that a Europe application the pipeline just runs because it is all the same environment after entering application random production the enterprise grade support becomes essential to meet any risk of interruption operation and also to continue operating your application in a business business critical manner our way to mitigate this risk is to provide long-term support with open source sense of law we typically offer one year of maintenance window for tesla free enterprise we provide three years of support that includes critical bug fixes and security patches as an additional option and optionally you know we may back port that certain functionalities and features from the future leaders of tensorflow as we see demand as of today we have tested Enterprise version 1.15 and 2.1 another long-term supported versions if your business is pushing the boundary of AI and as your business is sitting at the cutting edge of a I wear noble application I use TF is critical to your your business model and also your business a heavily relying on being able to continue innovating on this space we actually want to work with you through the white glove service program we engineer the career as both tensorflow and a Google cloud are willing to work with you work with your engineer that are your data scientist to mitigate any future bugs and issues that we may not have seen yet to support your cutting-edge applications to unblock it as well that and together advance your applications as well as a tenth of law and tenth of enterprise as a whole please check out the website and a fee for detail of this White Glove service program looking ahead we are really excited to keep writing tightly together between tens of 14th and a Google cloud deemed as being the Creator as an expert and owner of the posts products we continue to make the optimization that I've improvements the tensile flaw for a Google cloud that includes better monitoring and a debugging capabilities to your your chance of a code that runs in cloud as well as integration into integration of this capability into your Google Cloud tooling for the better product productivity of your application we're also looking at smoother intuition between tensorflow popular high level API such as Kerris over 1000 students and manage training services as well as even more manage services such as and continue some tens of tens of dev for the purpose of coherent and a joyful developer experiences please stay tuned this concludes my talk about in social enterprise for more information and for details please do check out the website thank you very much [Music]